Curvature domains in V4 of macaque monkey

Jia Ming Hu; Xue Mei Song; Qiannan Wang; Anna Wang Roe

doi:10.7554/eLife.57261

. 2020 Nov 19;9:e57261. doi: 10.7554/eLife.57261

Curvature domains in V4 of macaque monkey

Jia Ming Hu ¹, Xue Mei Song ^1,², Qiannan Wang ¹, Anna Wang Roe ^1,^2,^3,^✉

Editors: Joshua I Gold⁴, Kristine Krug⁵

PMCID: PMC7707819 PMID: 33211004

Abstract

An important aspect of visual object recognition is the ability to perceive object shape. Two basic components of complex shapes are straight and curved contours. A large body of evidence suggests a modular hierarchy for shape representation progressing from simple and complex orientation in early areas V1 and V2, to increasingly complex stages of curvature representation in V4, TEO, and TE. Here, we reinforce and extend the concept of modular representation. Using intrinsic signal optical imaging in Macaque area V4, we find sub-millimeter sized modules for curvature representation that are organized from low to high curvatures as well as domains with complex curvature preference. We propose a possible ‘curvature hypercolumn’ within V4. In combination with previous studies, we suggest that the key emergent functions at each stage of cortical processing are represented in systematic, modular maps.

Research organism: Rhesus macaque

Introduction

Recognizing the shapes of objects requires information about local contour features, such as orientation and curvature. The encoding of contour orientation in the visual system begins in primary visual cortex (V1) with neurons selective for the orientation of visual contours, and, in the second visual area (V2), with neurons selective for cue-invariant contour orientation (that is, independent of whether contours are defined by luminance, motion, color, or depth cues). These two functions are first embodied in the ‘orientation domain’ in V1 (Hubel and Wiesel, 1968; Blasdel and Salama, 1986; Grinvald et al., 1986) and the ‘higher order orientation domain’ in V2 (Ramsden et al., 2001; Chen et al., 2016). These signature domains in V1 and V2 mark initial computational elements of shape representation.

We investigate whether there are functional domains representing contour curvature in area V4 of macaque monkey visual cortex, an important intermediate stage of shape encoding. Although both V2 (Hegdé and Van Essen, 2003; Ito and Komatsu, 2004; Anzai et al., 2007) and V4 are considered intermediate stages of shape representation, a greater body of neural (Pasupathy and Connor, 2001; Hegdé and Van Essen, 2007; Nandy et al., 2016) and computational modeling (Murphy and Finkel, 2007; Pospisil et al., 2018; Wei et al., 2018) evidence points to area V4 as a locus of curvature information processing. V4 neurons are responsive to contour curvature and can be selective for degree of curvature and orientation of curvature (Pasupathy and Connor, 2001; Nandy et al., 2016). However, it is unknown whether such neuronal responses are organized in any way akin to orientation maps in V1 and V2. Previous studies have shown the functional organization of V4 comprises alternating bands of ‘orientation’ and ‘color’ preference (Tanigawa et al., 2010; Li et al., 2013; Ghose and Ts'o, 2017), as well as organization for disparity defined orientation (Fang et al., 2019), motion direction (Li et al., 2013), and spatial frequency (Lu et al., 2018), but maps for curvature organization have not been demonstrated. Here, we hypothesized that orientation bands in V4 are regions of shape representation that include both orientation and curvature domains.

We used intrinsic signal optical imaging, a method that is well-suited for studying functional domain responses, to examine curvature response in monkey V4. We hypothesized that curvature domains would meet the following criteria: (1) Band location in V4: We predicted that curvature domains would be located within the’ orientation bands’ and not the ‘color’ bands. (2) Distinct response preference: We expected that curvature domains should prefer curved stimuli over straight stimuli, and, furthermore, would exhibit selectivity for different degrees of curvature (from low to high) or different orientations of a curve (from 0° to 360°). (2) Curvature maps: Just as orientation and color domains have distinct maps, we predicted that curvature domains should have systematic representation for curvature degree and curvature orientation. (3) Response consistency: We also predicted that curvature domains would exhibit similar response to curved gratings and curved lines of similar curvature. (4) Hypercolumn: Finally, we sought evidence for the possibility of a curvature hypercolumn within V4. Our data suggest that there is a systematic ‘curvature map’ within V4. While curvature domains have previously been identified within V4 (Yue et al., 2014), this study shows such systematic maps at domain scale.

Results

General approach

Our approach is to understand the functional organization of curvature response in V4. We approached this study from a global to local scale in V4 by testing several predictions. First, we hypothesized that, as part of the shape encoding network, curvature domains would fall within the V4 ‘orientation bands’ rather than the ‘color bands’. Furthermore, within the orientation bands, curvature domains should be spatially distinct from the straight orientation domains. Second, analogous to the organization of orientation domains in V1, we predicted that curvature maps would represent curvature degree and curvature orientation in a systematic way. Third, if response is truly selective for curvature, curvature domains should exhibit the same selectivity for curvature content despite differences in detail (e.g. curved gratings vs curved lines). Fourth, we probed the hypothesis that a curvature hypercolumn exists in V4. This concept predicts that a local (e.g. ~1 degree) region of V4 should represent a range of curvature degrees and orientations. We present our findings below and provide a proposal for a curvature hypercolumn in V4.

Curvature domains exist

Using intrinsic optical imaging, we imaged V4 in three hemispheres of two anesthetized macaque monkeys (27 sessions). In addition to conventional straight gratings, we designed curvature stimuli composed of sinusoidal curved gratings (Figure 1) with curvatures ranging from low to high. This stimulus (4 deg in size) comprises a central region (~1 deg) which contains the primary curvature content, while the flanking regions are relatively straight. Below, we will provide evidence that curvature response is attributed to this central region. As shown in Table 1, almost all experiments described in this study were conducted in at least two cases. Consistent with previous studies (Tanigawa et al., 2010; Li et al., 2013; Ghose and Ts'o, 2017), alternating regions (bands) of color preference vs. orientation preference were observed in V4, with small regions of overlap between bands (Figure 2—figure supplement 1). To precisely place our visual stimuli, we also mapped the retinotopy of V4 cortex using 0.2° horizontal and vertical lines and placed the center of visual stimuli (4° in size) on the monitor (Figure 2—figure supplement 2, see Materials and methods). We then zoomed in and obtained functional maps of V4. We used oriented achromatic and isoluminant color gratings to obtain maps for orientation (Figure 2B), color (Figure 2D), and high vs. low spatial frequency (Figure 2E). For all maps, the locations of functional domains were determined by t-value maps (t-map, two-tailed t test, p<0.01) which were calculated by comparing, pixel by pixel, the responses between two different conditions. Timecourses were examined to compare magnitudes of response of statistically significant domains.

Figure 1. — (A) Calculation of curvature index. Width is half of the ellipse width. (B) Constructing curvature grating by periodic repeat. One low (index = 2) and one high (index = 5) curvature grating shown. (C) Luminance profiles of different stimuli in different directions. The profiles at the vertical axis (arrows at top, profile at right) of the two stimuli are similar, but differ in the horizontal axis (arrow at left, profiles below). The higher the curvature degree, the higher the spatial frequency towards the edge of the stimulus. Red square: central region, blue rectangle: flanking regions.

Table 1. Case list.

		Case 1	Case 2	Case 3
Curvature vs. straight map	Three cases (from three hemispheres of two animals)	Figure 2C and Figure 2—figure supplement 5B	Figure 2—figure supplement 4F,G and Figure 2—figure supplement 5C,D	Figure 2—figure supplement 6B
Curvature domain mask	Three cases	Figure 2G red pixels	Figure 5A right panel	Figure 6D
Curvature degree map	Three cases	Figure 3	Figure 3—figure supplement 2A and B	Figure 7—figure supplement 1
Curvature orientation map	Three cases	Figure 3—figure supplement 2C,D	Figure 4 and 6A	Figure 6D
Response consistency (curvature orientation)	Two cases (from two hemispheres of two animals)	NA	Figure 6A–C	Figure 6D–F
Scrambled response	One case	NA	NA	Figure 2—figure supplement 6
Data used for response amplitude calculation	Cases 1 and 3 are from the same monkey. Response to color (Case 1: 50 trials; Case 2: 70 trials; Case 3: 50 trials with two different orientations) Response to high SF (Case 1: 80 trials; Case 2: 120 trials; Case 3: 100 trials with four different orientations) Response to 0, 45, 90, 135 (Case 1: 30 trials; Case 2: 30 trials; Case 3: 30 trials with the corresponding optimal orientations) Response to curvature grating (Case 1: 240 trials; Case 2: 240 trials; Case 3: 240 trials with four different curvature orientations and two different curvature degrees, a/b ratio 2 and 5) Response to flashed curvature (Case 2: 60 trials with two different curvature orientations; Case 3: 120 trials with four different curvature orientations) Number of pixels related to color domain (Case 1: 34986; Case 2: 10886; Case 3: 22560) Number of pixels related to high SF domain (Case 1: 31,480; Case 2: 10,687; Case 3: 29,757) Number of pixels related to 0, 45, 90, 135 orientation domain (Case 1: 33,676, 47,501, 51,798, 43,837; Case 2: 8225, 10,243, 4712, 8891; Case 3: 38,362, 31,651, 41,822, 41,466) Number of pixels related to curvature domain (Case 1: 30,215; Case 2: 18,209; Case 3: 24,351)

Open in a new tab

Figure 2. — (A) View of cortical surface in Case 1. Dotted box: region shown in B-I. L.S., lunate sulcus. S.T.S., superior temporal sulcus, A, anterior, L, lateral. B-I: Case 1. (B) Color-coded orientation preference map. White dashed lines: approximate borders between color and orientation bands. (C) Curvature map: all curved minus all straight gratings. (D) Color preference map. (E) High spatial frequency preference map (4 cycle/deg vs. 0.5 cycle/deg). (F) Curvature domains (gray patches, two-tailed t-test, p<0.01) superimposed on iso-orientation contours. (**G-I**) Overlay of curvature domains (red) and G: color domains (green, from D), H: high spatial frequency domains (green, from E), I: all orientation domains (green). (J) Response time courses of curvature domains from Case 1 (left), Case 2 (middle), and Case 3 (right). Red lines: preferred stimuli. Gray lines: non-preferred stimuli. (K) Response time courses of curvature domains to flashed curved lines. Red timecourses: flashed curved lines. Gray timecourses: flashed straight lines. (L) Summary of response amplitudes for color (Col), high spatial frequency (HSF), orientation (0°, 45°, 90°, 135°), and curvature (Cur) domains shown in J, K. Gray: straight grating. Red: Curved grating. Black: optimal stimulus responses (except for Cur). For Cur, optimal response was to curved gratings (red) and to flashed curved lines (hatched red). Scale bar: 1 mm. Error bars: SEM (timecourses in J, K), SD (histogram in L).

Figure 2—figure supplement 1. — (A) View of cortical surface in Case 1. Dotted box: region shown in B-I. L.S., lunate sulcus. S.T.S., superior temporal sulcus, A, anterior, L, lateral. B-I: Case 1. (B) Color-coded orientation preference map. White dashed lines: approximate borders between color and orientation bands. (C) Curvature map: all curved minus all straight gratings. (D) Color preference map. (E) High spatial frequency preference map (4 cycle/deg vs. 0.5 cycle/deg). (F) Curvature domains (gray patches, two-tailed t-test, p<0.01) superimposed on iso-orientation contours. (**G-I**) Overlay of curvature domains (red) and G: color domains (green, from D), H: high spatial frequency domains (green, from E), I: all orientation domains (green). (J) Response time courses of curvature domains from Case 1 (left), Case 2 (middle), and Case 3 (right). Red lines: preferred stimuli. Gray lines: non-preferred stimuli. (K) Response time courses of curvature domains to flashed curved lines. Red timecourses: flashed curved lines. Gray timecourses: flashed straight lines. (L) Summary of response amplitudes for color (Col), high spatial frequency (HSF), orientation (0°, 45°, 90°, 135°), and curvature (Cur) domains shown in J, K. Gray: straight grating. Red: Curved grating. Black: optimal stimulus responses (except for Cur). For Cur, optimal response was to curved gratings (red) and to flashed curved lines (hatched red). Scale bar: 1 mm. Error bars: SEM (timecourses in J, K), SD (histogram in L).

We then examined response in V4 to straight vs. curved stimuli, consisting of moving curved (Ponce et al., 2017) sinusoidal gratings (four straight orientations: 0°, 45°, 90°, 135°; four curved orientations: curved up, curved down, curved left, curved right), and flashed single curved (curved up, curved down) or straight lines (horizontal, vertical) (see insets in Figure 2J and K). To examine preference for curvature over straight gratings, we subtracted curvature (sum of all four curvature grating maps) minus straight (sum of all four straight grating maps), revealing dark (curvature preferring) domains (Figure 2C, two-tailed t test, p<0.01). These curvature maps were distinct from (straight) orientation maps (overlay of curvature and straight shown in Figure 2I). For simplicity of terminology, we will refer to these curvature vs. straight preferring domains as ‘curvature domains’.

If these domains are indeed curvature domains, we predicted that they should be located within the ‘orientation bands’ and should be spatially distinct from straight orientation maps. We found that curvature domains (Figure 2G–I, red pixels) were distinct from color domains (Figure 2G, green pixels); they were also distinct from high spatial frequency domains (Figure 2H, green pixels), suggesting that these curvature responses are not simply due to the high spatial frequency components of the curved grating. Note also that these curvature domains have little overlap with low spatial frequency domains (Figure 2—figure supplement 3). In addition, as predicted, the curvature domains fell largely within the orientation band and were in close proximity to the straight orientation domains (Figure 2I, green pixels). As shown by the yellow pixels (locations of overlap), there is limited overlap between color and curvature domains (overlay in Figure 2G, Case 1, 4.9% overlap; all three cases, 5.7% overlap) and between high spatial frequency and curvature domains (overlay in Figure 2H, Case 1, 26.1% overlap; all three cases, 13.9% overlap). However, there is much greater overlap between curvature and orientation domains (Figure 2I, Case 1, 54.9% overlap; all three cases, 43.0% overlap), raising the possibility that curvature and orientation domains are different components of a shape information processing architecture. When overlaid on outlines of orientation domains, curvature domains appear distinct from orientation domains, and have a semi-regular distribution within the orientation band (Figure 2F, color: iso-orientation contours, gray patches: curvature domains). Similar modular curvature vs. straight preference maps were obtained in the other two cases (Figure 2—figure supplement 4: Case 2; Figure 2—figure supplement 6: Case 3).

Note that these curved grating stimuli were generated by periodically repeating an elliptical contour (see Figure 1, Materials and methods), resulting in, for high curvature gratings, high spatial frequency content in the flanks of the stimulus. If the curvature response was primarily due to high spatial frequency, then the curvature map and the high spatial frequency preference should have high overlap. However, this was not observed (Figure 2H), suggesting it is unlikely that the curvature response is primarily due to high spatial frequency content. We also examined the difference between high curvature degree and low curvature degree gratings, which have different spatial frequency content; as these maps are quite similar (Figure 2—figure supplement 4), it is unlikely that curvature domain response is due to spatial frequency content. We also observed that maps were stable over time, indicating these maps are unlikely to be artifactual (Figure 2—figure supplement 5). Furthermore, examination of images obtained in response to scrambled curved gratings minus straight gratings did not result in structured maps (Figure 2—figure supplement 6), suggesting that, when compared with straight gratings, it is not the presence of multiple orientations or multiple spatial frequencies alone that produced structured maps. In addition, the sizes of curvature domains (mean = 434 μm for all three cases) fall within the 200–500 μm range of functional domain sizes in V4 (Tanigawa et al., 2010; Li et al., 2013; Ghose and Ts'o, 2017). These data thus suggest that curvature maps are not artifactual and are distinct from previously described functional maps in V4.

We further examined selectivity of response of curvature domains by comparing magnitude of reflectance change in response to curved vs. straight grating stimuli. Reflectance change timecourses were typical of cortical intrinsic signals in V4, characterized by 2–3 s peak times and 0.01–0.03% amplitudes. As expected (Figure 2L, Col), color domains exhibited robust response to isoluminant color gratings (black) but weak response to achromatic gratings (gray); and failed to distinguished curved (red) vs straight (gray) gratings (Wilcoxon test, p=0.06, all three cases). Similarly (Figure 2L, HSF), high spatial frequency preference domains responded strongly to achromatic gratings of high spatial frequency (black) but poorly to low spatial frequency gratings (gray); and failed to distinguish curved (red) vs. straight (gray) gratings (high spatial frequency domains, p=0.77, all three cases) (see Figure 2—figure supplement 7 for supporting graphs). These analyses indicate that curvature processing is not subserved by color or high spatial frequency domains.

We also found that preferences of orientation domains and curvature domains were distinct. As expected (Figure 2L, 0°, 45°, 90°, 135°), straight orientation domains exhibited strongest response to gratings of their respective optimal orientations (black), weak responses to other straight orientations (gray), and weak response to curved gratings (red). Curvature domains, in contrast, exhibited strong preference for curvature stimuli. As shown in Figure 2J left, timecourses of curvature domains revealed strongest amplitudes for curvature gratings (red lines, Case 1), but relatively weak response to straight gratings (gray lines). Similar preferences are evident for both Case 2 (middle) and Case 3 (right). This is quantitatively summarized in (Figure 2L, Cur, red).

To further test whether this differential response is due to curvature, in two cases, we examined response to flashed curved and straight lines. As shown in Figure 2K, using the same pixels from which significant curved grating responses were obtained, we found that response of curvature domains to single flashed curved lines (red lines) was significantly greater than that to single straight lines (gray lines). Thus, curvature domains displayed preference for curved over straight contours for both grating and line stimuli, supporting a curvature-specific response. This curved line response is quantitatively summarized in (Figure 2L, Cur, red hatch). [Note that, in comparison, straight orientation domains and high spatial frequency domains exhibited weak responses to flashed curved lines (Figure 2—figure supplement 8)].

To summarize (Figure 2L), each of the color domains (Col), high spatial frequency domains (HSF), and orientation domains (0°, 45°, 90°, 135°) exhibit preference for their respective optimal stimulus (black bars), one which far exceeds response to curved gratings (red bars) (black vs red bars, Wilcoxon test, p<0.0001). Likewise, curvature domains (Cur) exhibit strong preference for curved gratings (large red bar) and curved lines (hatched red bar) and little response to straight gratings (gray bar). The degree of preference for curvature (Cur red vs gray) parallels the degree of preference of color, high spatial frequency, and orientation domains in V4 for their respective optimal stimulus (black vs gray, black vs red). These data show that (1) curvature domains are strongly selective for curvature stimuli and (2) non-curvature domains exhibit minimal response to curvature stimuli. Thus, curvature domains in V4 exhibit response preferences that are selective for curvature (compared to straight, color, high spatial frequency) and are maintained across two types of curvature stimulus (gratings and lines). Evidence from electrophysiological recording also show the predominance of curvature preference neurons in these domains (Figure 2—figure supplement 9). These curvature preferring domains were not observed in V1 (one case shown in Figure 2—figure supplement 10), consistent with previous studies (Yue et al., 2014; Ponce et al., 2017). As V2 in these cases was not available on the operculum (Figure 2—figure supplement 11), we have no data on V2.

Mapping types of curvature

We predicted the global organization of curvature representation should systematically shift with changing curvature degree and with changing curvature orientation. To address this global question, we first tested whether the response pattern would vary with changes of curvature features. We conducted cross-correlation of images obtained with different curvature degrees and with different curvature orientations. By examining correlations between images, we expected that increasing stimulus similarity should predict increasing map similarity. In this section, we show that this prediction is met. In the next section, we zoom in on a small location region to study the local organization more closely.

Low to high curvature maps

To examine whether there could be a change of curvature domains representing low to high curvatures, we designed a series of curved gratings from low to high curvature (Figure 3A, upwards: 1–6; downwards: 7–12, Case 1; Ponce et al., 2017) and presented these stimuli in two cases. Maps obtained in response to these curvature stimuli exhibited regular structure and domain size consistent with functional maps (see Figure 3A). To examine the possibility of a low to high curvature gradient, we conducted cross correlations between pairs of these images (see Materials and methods). We reasoned that, if a gradient of curvature degree exists, then there should be a gradually changing map similarity as curvature degree changes. Correlation values ranged from 0 (no correlation or minor negative correlation, blue) to 1 (two maps are identical, red) (see color bar). To give the reader an idea of what high map correlations values are, the range of correlation values for four straight orientation maps in V4 shown in Figure 3—figure supplement 1; matched orientation correlations range from 0.66 to 0.9 (yellow, orange, red values). As shown in Figure 3B, higher correlation values tended to occur between curvature maps of similar curvature degree (orange and yellow squares in upper left quadrant and lower right quadrant) and lower correlation values between those of dissimilar curvature degree (green and blue squares in upper left quadrant and lower right quadrant). Within this correlation matrix, the lowest correlation values were obtained between opposing curvatures (upper right quadrant and lower left quadrant) and between curved and straight stimuli (rightmost column). High correlation values were obtained between adjacent 1 or two curvatures (orange and yellow boxes) (e.g compare values between pairs 2–3, 3–4); correlation values dropped off with increasing differences in degree of curvature (green and blue boxes) (e.g. 2–3, 2–4, 2–5). This relationship is plotted in Figure 3C for each of the 6 Curved Up (top graph) and 6 Curved Down (bottom graph) curvature degrees (e.g. Figure 3C top graph red circles: show declining correlation with curvature difference). This declining correlation with curvature difference suggests the presence of a shifting map of curvature degree. Similar results were obtained in a second case (Case 2, Figure 3—figure supplement 2A–B).

Figure 3. — (A) Maps of different curvature degrees. Each map is one curvature minus average of all straight gratings (Case 1). 1–6 and 7–12: upwards and downwards curvatures, respectively, from low to high curvature preference, 13: straight grating. Red dotted square marks the region that is further analyzed in Figure 7A,B. Correlation values for pairs of curvature response maps (from A). Color bar: high (red) to low (blue) correlation values. (C) The more similar the curvature the greater the correlation value. X axis: curvature degree difference. Y axis: correlation value. Color symbols: correlation value for each curvature above with respect to its curvature degree distances (each fit with matching color line; regression values are shown on the right). The fitted lines are not meant to indicate linear fits; rather, they help to see the trends. Scale bar, 1 mm.

Figure 3—figure supplement 1. — (A) Maps of different curvature degrees. Each map is one curvature minus average of all straight gratings (Case 1). 1–6 and 7–12: upwards and downwards curvatures, respectively, from low to high curvature preference, 13: straight grating. Red dotted square marks the region that is further analyzed in Figure 7A,B. Correlation values for pairs of curvature response maps (from A). Color bar: high (red) to low (blue) correlation values. (C) The more similar the curvature the greater the correlation value. X axis: curvature degree difference. Y axis: correlation value. Color symbols: correlation value for each curvature above with respect to its curvature degree distances (each fit with matching color line; regression values are shown on the right). The fitted lines are not meant to indicate linear fits; rather, they help to see the trends. Scale bar, 1 mm.

Curvature orientation maps

In another experiment (Figure 4A,B, Case 2), we used four orientations of curvature (curved upwards, downwards, leftwards, and rightwards), each at low curvature (1-4) and high curvature (5-8). Similar to maps for straight orientation (Figure 3—figure supplement 1), we expected stronger correlation between maps of similar curvature orientation and lower correlation for dissimilar orientations. This expectation was supported by computed correlation indices for pairs of images (Figure 4C). These correlation values illustrate that high correlation values (orange, yellow) occur only between curvature gratings of the same orientation (e.g. 1–5, 2–6), and low correlation values occur between curvatures of different orientations (green boxes in columns 1–8, e.g. 1–2, 3–4). The finding that the low correlation values occur between these curved and straight gratings (columns 9–12) supports the distinctness of straight vs. curved contour representation. Similar results were obtained in a second case (Case 1, Figure 3—figure supplement 2C–D).

Mapping central part of curvature grating

To more precisely study the organization within curvature domains, we then focused on the cortical region of V4 representing the center of the curvature stimuli (the region of high curvature content, ~1 deg). As shown in Figure 2—figure supplement 2, using thin vertical and horizontal lines, we determined the retinotopic representation of V4 within the imaged field of view. The curvature stimulus was then centered precisely on this location. Note that the region of V1 visible in the same field of view represents a shifted topographic location, so we were unable to map response to the curvature stimulus in both V1 and V4 simultaneously. In these cases, V2 was buried in the lunate sulcus and was not visible.

Organizations within curvature domains

By focusing on the region representing the central part of the curvature stimulus, we were able to discern subdomains of high curvature vs. low curvature preference and largely non-overlapping subdomains of opposing curvature orientation. Figure 5 shows that curvature domains (Figure 5A, Case one and Case 2, white pixels: curvature vs. straight) contain subregions of high curvature preference (red pixels: high curvature vs. low curvature). This is shown by preference (Figure 5B, Case 1) for high (dark red lines) over low (light red lines) curvature and even weaker response to straight gratings (gray lines). Moreover, the response magnitudes of these subregions gradually diminish with curvature degree difference (Figure 5C, dark to light lines plot responses of high to low curvature; left: upwards curvature, right: downwards curvature). Thus, there are high curvature subdomains within curvature domains (Figure 5A), and curvature subdomains exhibit graded response to curvature degree (Figure 5B,C).

Figure 5. — (A) Curvature preferring pixels. Red pixels: high curvature preferring pixels (high curvature vs. Low curvature, a/b ratio = 5 vs. 2, all four orientations, two-tailed t-test, p<0.01). White pixels: (curvature vs. straight). (Case 1: left, Case 2: right, two-tailed t-test, p<0.01). (B) High curvature subdomains (red pixels in A, Case 1) prefer high curvature (dark red lines, four orientations, a/b ratio = 5) over low curvature (light red lines, four orientations, a/b ratio = 2), and straight lines (Gray lines, four orientations). (C) Timecourses of response to 4 degrees of curvature (from high to low, darkest to lightest gray). Scale bar, 1 mm. Error bar: SEM.

Figure 6 illustrates the finding that paired opposing curvature orientations occupy roughly complementary positions within the curvature domains. An example of an UP/DOWN pair is shown (Figure 6A). Figure 6B,C show that pixels that prefer upwards maintain their preference across curved gratings (6B) and two degrees of curved lines (6C). This is also true for downwards pixels (6B,C). A similar result is shown in Figure 6D–F (Case 3). For four curvature orientations, upwards/downwards (Figure 6D left panel) and leftwards/rightwards (Figure 6D right panel) pairs occupy roughly complementary regions within the curvature domains. These preferences are maintained across curved grating and curved line stimuli. Additionally, the graded response for each of the four orientations suggests the presence of a curvature orientation organization (see below). These results reinforce the presence and selectivity of curvature orientation domains. We note that this complementarity is not absolute. Within the curvature domains, the average percentage of pixels responsive to two opposite orientations (overlapped pixels vs. all colored pixels) was 22.4% (Case 1: 32.1%; Case 2: 14.9%; Case 3: 20.3%).

Organization of curvature representation

Given that there are indications of shifting maps for curvature degree, we examined whether we could discern any systematic relationship between domains for straight orientation and curvature degree. We hypothesized that in some parts of V4, there is a gradual shift from straight to curved representation. In another experimental session, as shown in Figure 7A (Case 1), within the central region of curvature representation, we obtained (at a single orientation) curvature minus straight orientation maps to different degrees of curvature. Within this region, there appear to be several progressions of straight to curved domains (four straight-to-UP and four straight-to-DOWN). Overlays of these maps are summarized in Figure 7B (left panel, top row: UP overlays, bottom row: DOWN overlays; white regions: curvature domains). A subset of these overlays are shown in Figure 7B (middle and right panels. UP: top row, DOWN: bottom row). The middle panels isolate straight orientation domains (blue) and the nearest curvature domains (from straight to high curvature, color code at top) (domain clusters are separated for clarity). The right panels demarcate the center of each domain with a color-coded dot (center of mass). We observed a spatial progression from the center of the straight orientation domain (blue dot in center of blue dotted line) to low curvature domain (blue-oranges) to high curvature domain (orange); these colored dots are connected by a line (shaded from blue to orange) to illustrate the progression. Four examples of such progressions for Up are shown in the top panel and four progressions for Down in the bottom panel. Progressions arising from a single orientation domain can lead to two different curvature domains (e.g. UP: leftmost orientation domain, Down: rightmost orientation domain). Additional examples are shown in Figure 7—figure supplement 1. These progressions suggest a possible systematic contour representation in V4.

Figure 7. — (A) Maps of different stimuli minus average of straight gratings (Case 1). Top row: upwards curvatures, bottom row: downwards curvatures. Leftmost: horizontal gratings. (B) Maps of progressions from straight to curved representation. Top row: responses to upwards curvatures (Up). Bottom row: responses to downwards curvatures (Down). *Left panels:* activated regions (two-tailed t-test, p<0.01) corresponding to different curvature stimuli. White regions: curvature domains. Color code: high (orange) to low (blue). *Middle panels:* activated regions corresponding (two-tailed t test, p<0.01, curved vs. average of straight) to respective curvature degrees are outlined by different colors (domain clusters are separated for clarity). Color code (at top): high (orange) to low (blue). [Note that the two leftmost domain progressions in Up panel are associated with the same orientation domain but are separated for clarity.] *Right panels:* Location of the activation center of each domain (indicated by colored dot). White regions: curvature domains. Blue dotted lines: horizontal orientation domains. Shifting progressions from straight orientation (blue dot) to low curvature (blue-orange dot) to high curvature (orange dot) are observed, as indicated by colored line (shaded from blue to orange). (C) Regions that were significantly activated (two-tailed t test, p<0.01) by corresponding stimuli (from Case 1). Red dots: centers of mass. (D) We measured the distance between each curvature domain activation center of mass and its corresponding straight orientation domain center of mass (a/b rations: 1, 2, 4, 5, 7, 10 in Case 1, eight progressions). Gray dots and lines: results from different progressions, black dots: averaged across all eight progressions. Error bar, SD. (E) Summary of curvature domain findings. Left: curvature degree progressions, Right: presence of curvature domains and complex curvature domains. Gray oval: curvature domains.

Figure 7—figure supplement 1. — (A) Maps of different stimuli minus average of straight gratings (Case 1). Top row: upwards curvatures, bottom row: downwards curvatures. Leftmost: horizontal gratings. (B) Maps of progressions from straight to curved representation. Top row: responses to upwards curvatures (Up). Bottom row: responses to downwards curvatures (Down). *Left panels:* activated regions (two-tailed t-test, p<0.01) corresponding to different curvature stimuli. White regions: curvature domains. Color code: high (orange) to low (blue). *Middle panels:* activated regions corresponding (two-tailed t test, p<0.01, curved vs. average of straight) to respective curvature degrees are outlined by different colors (domain clusters are separated for clarity). Color code (at top): high (orange) to low (blue). [Note that the two leftmost domain progressions in Up panel are associated with the same orientation domain but are separated for clarity.] *Right panels:* Location of the activation center of each domain (indicated by colored dot). White regions: curvature domains. Blue dotted lines: horizontal orientation domains. Shifting progressions from straight orientation (blue dot) to low curvature (blue-orange dot) to high curvature (orange dot) are observed, as indicated by colored line (shaded from blue to orange). (C) Regions that were significantly activated (two-tailed t test, p<0.01) by corresponding stimuli (from Case 1). Red dots: centers of mass. (D) We measured the distance between each curvature domain activation center of mass and its corresponding straight orientation domain center of mass (a/b rations: 1, 2, 4, 5, 7, 10 in Case 1, eight progressions). Gray dots and lines: results from different progressions, black dots: averaged across all eight progressions. Error bar, SD. (E) Summary of curvature domain findings. Left: curvature degree progressions, Right: presence of curvature domains and complex curvature domains. Gray oval: curvature domains.

To further examine the details of these progressions, we analyzed the distances between each straight orientation domain and the nearest (same orientation) curvature domains. Specifically, starting from a straight orientation domain (n = 8), we selected the nearest domain of each curvature condition (shown in Figure 7B) and measured the distances between center of mass of each domain and that of the straight domain (leftmost panel in Figure 7C). These values are plotted in Figure 7D (gray lines). This reveals that the curvature preference domains generally progress from low to high curvature with distance across the cortex. Note that some parts of these curvature sequences shift and other parts do not appear to shift. However, on average (black line), across these eight straight-to-curved sequences, there is a general trend of greater distance with increasing curvature. We conducted this analysis for the other two cases (shown in Figure 7—figure supplement 1). Case 2 shows three progressions with a similar tendency to shift (Figure 7—figure supplement 1B) and one without shifts (Figure 7—figure supplement 1C) (graphed in Figure 7—figure supplement 1D). Case 3 shows seven progressions with shifts (Figure 7—figure supplement 1F), and one without (Figure 7—figure supplement 1G) (graphed in Figure 7—figure supplement 1H). In total across the three cases, on average, there is a general overall tendency for straight-to-curved domains to exhibit spatial shifts. However, there are some domains with broad curvature preference. This suggests the presence of a diversity in organization, indicating that curvature representation in V4 is complex.

Curvature domains are not orientation pinwheel centers or sums of orientation components

We considered the possibility that curvature responses could be due to activation of a mixture of oriented responses, such as that found at orientation pinwheel centers. As shown in two cases in Figure 2—figure supplement 12 (Case 1: A-B, Case 2: C-D), the curvature domains (orange regions) do not co-localize with orientation pinwheel centers (blue dots). In fact, most of the pinwheel centers are outside the curvature domains entirely. Moreover, as shown by the responses of pinwheel centers to straight and curved stimuli (Figure 2—figure supplement 12E), the pinwheel center responses to curvatures are weak and do not distinguish curvature from straight (Wilcoxon test, p=0.13). This makes it unlikely for pinwheel centers to be locations of curvature response. We also considered the possibility that curvature responses are simply a weighted sum of responses to component orientations. This is unlikely because (1) the straight orientation domains are spatially distinct from the curvature domains (Figure 2). And (2) our analysis of response to component orientations compared with response to curvature shows that curvature domain response to straight component orientations is weak (smaller by roughly an order of magnitude) (Figure 7—figure supplement 2).

Curvature domains are not end-stopping domains

There is strong support for the role of end-stopped cells in curvature representation (Hubel and Wiesel, 1965; Hubel and Livingstone, 1987; Dobbins et al., 1987; Sceniak et al., 2001). The rationale is that straight contours extend into the RF end zones leading to suppression, while curved contours do not, making end-stopped neurons ideal ‘not straight’ candidates for encoding curvature. Electrophysiological study of single unit responses to curvature gratings in V4 finds strong correlation between curvature preference and strength of end-stopping (Ponce et al., 2017). Thus, this study suggests response to these two parameters in V4 are indistinguishable, raising the possibility that the curvature domains recorded here actually represent end-stopping response.

We feel this is unlikely. End-stopping, while an essential contributer to curvature computation, is not sufficient for all curvature encoding. As supported by previous studies in V4 (Nandy et al., 2013; Nandy et al., 2016), the curvature selectivity displayed by many V4 neurons can also be caused by other properties (e.g. complex structures of receptive field). While end-stopped cells may contribute to the curvature domain responses recorded here, curvature domains are not equal to end-stopping domains. As shown in Figure 2—figure supplement 13, comparison of end-stopping maps (1 deg vs 4 deg straight gratings) and curvature maps reveal that they are distinct. Figure 2—figure supplement 13C (overlay of A, end-stopping map and B, curvature map) illustrates that there are some regions of overlap (~33.3%), but 66.7% of domains are in spatially distinct locations. Moreover, quantification of curvature-preferring pixels (Figure 2—figure supplement 13D) reveals that, while there is a preference for the small over the large straight stimulus (compare two light gray lines), the responses to straight stimuli are still weaker than that to curved (black lines). This suggests that, in V4, curvature domains are not simply end-stopped domains.

Discussion

Summary

In summary, we provide evidence for the existence of curvature domains in V4 and show that there is a systematic map of curvature representation in V4. To the organization of curvature representation within V4, we designed stimuli that spanned curvature degree and curvature orientations. Instead of using shapes with complex structures such as hyperbolic/polar gratings (Hegdé and Van Essen, 2003) or forms composed of multiple corners and lines (Pasupathy and Connor, 2002; Brincat and Connor, 2004), these were stimuli with relatively simple curvature content (similar to Ponce et al., 2017). The gradations of curvature degree across our stimulus set revealed that response preference is spatially mapped in certain parts of V4 (from straight to highly curved, Figure 7B and Figure 7—figure supplement 1B,F). We showed that maps for curvature are distinct from those for straight orientation, color, high spatial frequency, orientation pinwheels, and end-stopping. These maps also have the spatial appearance of organized functional maps in terms of domain size and distribution. We find curvature domains exhibit selectivity for degree of curvature (low to high), one which maps in a spatially shifting manner in V4. These progressions are not necessarily linear, as indicated by the distance changes related to curvature degree (Figure 7D, Figure 7—figure supplement 1D,H). In addition to these curvature degree progressions, other types of spatial organizations were also observed (Figure 7—figure supplement 1C,G). Similarity of response preference across both curved gratings and single curved lines provide further evidence for curvature selectivity rather than some other aspect of these stimuli.

Hypercolumn model

While the overall spatial relationship between straight and curved features will take additional study to fully establish, given the data at hand, we propose the model shown in Figure 7E. We suggest that straight (large gray square) and curved (large gray oval) regions in V4 co-exist and occupy roughly complementary territory within the ‘orientation bands’ of V4. Within the curvature domains, there are multiple types of organizations, characterized by sequences of curvature degree as well as domains with complex curvature preference. To our knowledge, no previous study has provided evidence suggesting that curvature is a basis for a hypercolumn in V4. This framework will need to be further tested in future studies.

Testing other possible interpretations

Unlikely to be primarily component orientation responses

We tested the possibilities that the curvature preference domains are actually responses to the weighted sum of component straight orientations. However, we found little support for this: the weighted sum of three primary components produced responses that were an order or magnitude weaker than to the curved stimulus (Figure 7—figure supplement 2). Scrambled stimuli which contain straight components failed to produce structured maps (Figure 2—figure supplement 6). And orientation pinwheels, which contain many straight component responses, were at locations quite distinct from the curvature domains (Figure 2—figure supplement 12). Neither did our data support the possibility that curvature domains are end-stopping domains (Ghose and Ts'o, 1997), as these mapped to spatially distinct locations (Figure 2—figure supplement 13). Thus, it does not appear that curvature domain responses are due to a simple linear combination of local orientation domain responses, but result from a non-linear (perhaps multi-step) integration of straight oriented inputs.

Unlikely to be high spatial frequency domains

We consider the possibility that the curvature responses are actually responses to the high spatial frequency aspects of the curvature gratings. Based on curvature stimuli used in Ponce et al., 2017, we designed curvature gratings composed of ellipsoid curves repeated sinusoidally. As shown in Figure 1, different stimuli with different degrees of curvature created in this fashion have common spatial frequency along the center axis. Towards the flanks of the stimulus, especially for higher curvature, the grating has higher SF content than that on the central axis. However, several observations make our results inconsistent with the hypothesis that these results are due primarily to these high SF aspects of the stimuli. (1) High spatial frequency maps in V4 (first reported by Lu et al., 2018) differed from curvature maps (Figure 1H and L). On average, less than 15% overall from three cases exhibited overlap between high spatial frequency domains and curvature domains. (2) Similar maps were obtained to gratings with different degrees of curvature (Figure 2—figure supplement 4). If the results were due primarily to spatial frequency content, the maps resulting from high and low curvature gratings should differ. (3) Similar maps were obtained to gratings and single lines of the same curvature (Figure 2K,L, Figure 6), although gratings and single lines have different spatial frequency contents. (4) Maps obtained in response to scrambled curvature stimuli (which also contain high spatial frequencies but lack the structural curvature information) failed to produce curvature maps. Thus, while we have not excluded all possibilities, we believe that the bulk of the evidence provides a consistent view. Note also that these curvature domains have little overlap with low spatial frequency domains (Figure 2—figure supplement 3), further indicating that a representation of curvature and spatial frequency content are distinct.

Unlikely to be end-stopping domains

As previous studies suggested, size tuning or end stopping may play an important role in curvature detection (Hubel and Wiesel, 1965; Dobbins et al., 1987) and is not independent from curvature preference features (Ponce et al., 2017). However, other studies have suggested neurons with complex receptive field structures may also account for curvature preferences (Nandy et al., 2013; Nandy et al., 2016). For this reason, while end-stopping likely contributes to curvature response, there are also other relevant parameters contributing to curvature response. (1) Surround suppressed and curvature domains are not the same: Similar to previous studies (Ghose and Ts'o, 1997), we found there were size sensitive regions in V4 (Case 2, Figure 2—figure supplement 13A). However, while there was some overlap, the size sensitive regions were on the whole spatially distinct from the curvature domains (compare Figure 2—figure supplement 13A and B, overlay in C). (2) Weak response to small stimuli. Responses of curvature domains to small stimuli were weak in comparison to response to curvature stimuli (Figure 2—figure supplement 13D), suggesting weak end-stopping. Therefore curvature response can not be fully explained by strong responses to end-stopping. (3) In addition, if end-stopping was the only factor that matters, curvatures with opposite orientations would densely overlap, which is not the case (Figure 6). Thus curvature preference response in V4 is not due to end-stopping alone.

Curvature domains are signature modules of V4

Our results are bolstered by another recent study which also reports the presence of curvature domains in V4 (Tang et al., 2020). Using stimuli that included simple shapes (e.g. circles vs triangles) and shape components, the authors also report that curvature domains are distinct from orientation domains and color domains, and are absent in V1 and V2. Using two-photon imaging, the authors found neurons in curvature domains respond to a diversity of curved shapes and curve parts, and exhibit weak response to straight lines and corners. Within a curvature domain, they find subclusters of neurons with similar shape or curve part preference. Interestingly, some clusters exhibit preference for shapes (circles) over shape parts (curved portions of circles), raising the possibility that some domains in V4 may represent simple shapes. While the authors did not systematically explore the organization of curvature degree or curvature orientation, this elegant study further reinforces the concept that curvature domains are a primary feature of V4 organization.

Significance of modules—cortical modules represent key transformational stages in the visual hierarchy

We are excited that the combination of MRI (Op de Beeck et al., 2008; Popivanov et al., 2015; Yue et al., 2014; Bao et al., 2020), optical imaging (Ghose and Ts'o, 1997; Tanigawa et al., 2010; Lu et al., 2018; Jiang et al., 2019; this study; Tang et al., 2020), and two-photon (Jiang et al., 2019; Tang et al., 2020) scale studies are providing a functional organizational view of shape processing in the ventral visual pathway. While there is much yet unknown about curvature and shape representation, there is enough to see a sketch of the organizational hierarchy. By conducting fMRI mapping in response to a large array of stimuli containing curved (rounded, spheres, faces) vs rectilinear (square shapes, pyramid arrays, buildings) objects, Yue et al., 2014 reported the presence of three ~ 1–2 cm-sized curvature patches in the temporal lobe of macaque. These patches revealed a hierachy of increasing curvature complexity from the ‘posterior curvature-biased patch’ in near-foveal V4 (PCP), to a middle curvature-biased patch (MCP) in posterior STS, to an anterior curvature-biased patch (ACP) in anterior TE.

The region of V4 studied here corresponds well with Yue et al’s PCP patch. Here, zooming in to the several mm scale, we find 200–500 μm-sized curvature degree and curvature orientation domains within (what is likely) Yue’s PCP patch (see Yue Figures 2 and 3). Consistent with the lack of curvature response in V1, V2 (Yue et al., 2014; Hegdé and Van Essen, 2007; Ponce et al., 2017), we found little evidence for curvature response preference in V1. Thus, much as V1 is considered the ‘orientation-emergent’ area, and V2 the ‘cue-invariant orientation-emergent’ area, we concur with the view that V4 is the ‘curvature-emergent’ area. Integration of signals from V4 curvature domains are likely to contribute to additional levels of systematic maps associated with lower (MCP, PFP) and higher (ACP, AFP) order object, face, and body patch representations (Bao et al., 2020). We predict that these higher level patches will also contain systematic columnar maps.

Materials and methods

Data was acquired from three hemispheres of two adult macaque monkeys (one male and one female, Macaca mulatta). All procedures were performed in accordance with the National Institutes of Health Guidelines and were approved by the Zhejiang University Institutional Animal Care and Use Committee.

Animal preparation

Chronic optical chambers were implanted above the area V4d containing lunate sulcus, superior temporal sulcus (see Figure 2A) as described previously (Li et al., 2013). The only difference is that we used transparent glass instead of a nylon chamber. The eccentricity of the visual field corresponding to the exposed V4/V2/V1 was 0–5°. Following the craniotomy surgery, optical images were collected during which basic functional maps as well as curvature responses of V4 were obtained. Monkeys were artificially ventilated and anesthetized with propofol (induction 5–10 mg/kg, maintenance 5–10 mg/kg/hr, i.v.) and isoflurane (0.5–1.5%). Anesthetic depth was assessed continuously via monitoring heart rate, end-tidal CO₂, blood oximetry, and eeg. Rectal temperature was maintained at around 38C°. Animals were paralyzed (vecuronium bromide, induction 0.25 mg/kg, maintenance 0.05–0.1 mg/kg/hr, i.v.) and respirated. Pupils were dilated (atropine sulfate 1%) and eyes were fitted with contact lenses of appropriate curvature to focus on a stimulus screen 57 cm from the eyes.

Visual stimuli for optical imaging

Visual stimuli were created using ViSaGe (Cambridge Research Systems Ltd.) and displayed on a calibrated 27-inch monitor (Philips 272G5D) running at 60 Hz refresh rate. The luminance for white stimuli was 206.52 cd/m² and black was 0.50 cd/m². Full-Screen visual grating stimuli were used to locate color preference domains in V4. Red/green isoluminance and black-white sine-wave drifting grating stimuli were presented at two different orientations (45° and 135°) with the same spatial frequency (0.5 or one cycles/°), temporal frequency (2 or 4 Hz) and mean luminance level. To acquire orientation maps and spatial frequency (SF) maps, gratings with four different orientations (0°, 45°, 90°, 135°, see Figure 2J) and two different SFs (0.5, 4 cycles/deg) were presented.

Curvature stimuli were designed based on the formula of ellipse, X²/a² + Y²/b² = 1, where a and b represent the length of the long axis and short axis, respectively. Different a/b ratios were used (2, 5, see Figure 5B; 1, 4, 7, 10, see Figure 5C) to create different curvature degrees. The ellipses (see Figure 1) were cropped and duplicated to generate a curvature grating template. SF (along the long axis) of the curvature stimuli was varied by changing the spacing of the curves. The curvature grating was drifted by moving the stimulus window along the curved grating template (see Figure 1). Drifting straight gratings were also created by this process and had the same luminance, drift speed, and spatial frequency (at center axis of curvature) as curvature stimuli. For curvature maps, the position of the stimuli on the monitor was first determined. As a control for curvature, scrambled versions of a curvature grating were created by dividing the grating into 64 subunits and randomly rearranging the locations of these subunits (Figure 2—figure supplement 6A).

Retinotopic mapping

As we used vecuronium bromide during the experiment, the two eyes did not look at the same position of the monitor, thus for curvature mapping, we covered one eye and only presented the curvature stimuli to one eye. In every experiment session, we first mapped the RF location of V4 region and presented the stimuli at that location (~1–5 deg eccentricity in V4). For placement of stimuli on the monitor, we mapped the retinotopy of V4 using a series of 0.2° width horizontal or vertical dashed lines (SF = 1 cycles/°) located at different positions to determine the stimulus center (locating the stimulus center at the imaged V4 center, Li et al., 2013, also see Figure 2—figure supplement 2). As the receptive field sizes of V4 neurons at our recording position are around 3–5 degrees (Chen et al., 2014), our stimuli were set to 4° and presented monocularly during the experiment. Flashed curved lines were also 4° in size (for each line, the length was 4°). The center of straight lines or the vertex of curved lines were fixed at the position that we chose during the retinotopic mappings. The stimuli were presented at 4 or 8 Hz interleaved with black screen.

Optical imaging

The brain was imaged through the implanted glass. Images of cortical reflectance changes (intrinsic hemodynamic signals) corresponding to local cortical activity were acquired (Imager 3001, Optical Imaging Inc, German town, NY) with 632 nm illumination. Image size was 1080 × 1308 pixels representing 14.4 × 17.4 (case 2) or 8.7 × 10.5 (cases 1 and 3) mm field of view (see Figure 2—figure supplement 1). Visual stimuli were presented in a random order. Each stimulus was presented for 3.5 (for color and high spatial frequency stimuli) or 4.5 (curvature and corresponding straight stimuli) seconds. The imaging data were stored in a block fashion. Each block contained the imaging data recorded from all the stimulus conditions (presented one time). For functional domains, each stimulus was presented at least 30 times. For retinotopic mapping, each stimulus was presented 10 times. Imaging started 0.5 s before the stimulus onset and ended till the stimulus offset with a sampling rate of 4 Hz. In V4, cortical fields of view were ~1–5 deg eccentricity; in V1, locations were at about 1–2 deg eccentricity. For all three cases, very little V2 was available on the surface and so was unavailable for imaging (see Figure 2—figure supplement 11).

Map-guided cell recording

Under a surgical microscope, a tungsten microelectrode (impedance 1–4 MU at 1 kHz, FHC) was lowered into the cortex (manipulator: Narishige MO-10) targeting the center of a curvature domain. Neural activity was amplifified at 1 k or 10 k gain (Model 1800, A-M Systems) and digitized at a sampling rate of 30 kHz (Blackrock microsystems). Each stimulus was normally tested for 15–30 trials.

Data analysis

Functional maps

With the following formula, $Δ R_{i} = (\bar{R_{i 1}} - \bar{R_{i 2}}) \times \sqrt{N} / S_{i}$ , we assessed the response differences between two comparison groups. $\bar{R_{i 1}}$ and $\bar{R_{i 2}}$ are the mean dR/R values ( $d R / R = \frac{R_{8 - 16} - R_{1 - 3}}{R_{1 - 3}}$ , R_8-16 is the averaged response from frame 8 to 16, R_1-3 is the averaged response from frames 1 to 3) in the two compared conditions of pixel i, respectively, N is the number of trials, and S_i is the standard deviation of $(R_{i 1} - R_{i 2})$ . Color preference maps were obtained by comparing red/green and white/black grating images, spatial frequency maps by comparing high and low spatial frequency images, and orientation maps by comparing two orthogonal orientation images (0° vs. 90° or 45° vs. 135°). For curvature maps, we compared the sum of curved gratings with the sum of straight gratings (four orientations: 0°, 45°, 90°, 135°). For single condition maps, we compared the selected condition with the averaged of straight gratings (four orientations: 0°, 45°, 90°, 135°). Maps were high-pass filtered (Gaussian filter, 10 pixel diameter) and low-frequency noise was reduced by convolving the map with a 100–150 pixel diameter circular filter and subtracted from the original maps. The orientation preference angle maps were calculated based on the single condition maps, and each pixels were assigned with a unique color to represent the preferred orientation (Bosking et al., 1997).

Locating positions of functional domains

Functional domains were identified by selecting the pixels with significant dR/R difference (two-tailed t test, p<0.01) between two comparison conditions (color domains, red/green versus white/black, ∆dR/R < 0; high spatial frequency domains, high spatial frequency versus low spatial frequency, ∆dR/R < 0; 0° orientation domain, 0° versus 90°, ∆dR/R < 0; 45° orientation domain, 45° versus 135°, ∆dR/R < 0; 90° orientation domain, 90° versus 0°, ∆dR/R < 0; 135° orientation domain, 135° versus 45°, ∆dR/R < 0; Curvature domains, curved gratings versus straight gratings, ∆dR/R < 0). In addition to t-test, pixels that belong to curvature domains were further assessed by one-way ANOVA (curved stimuli versus straight stimuli, p<0.05 with post-hoc Tukey Kramer correction) test. Domain size was calculated as size = πR², where area size is the size of each separated patch (pixel number ×area of each pixel), R is the average of long and short axis of patch. Patches smaller than 0.2 mm (diameter) were excluded from this analysis, as the reliability of these small patches is less secure. For each activated region, the geometrical center was calculated as its activation center (see Figure 7).

Timecourse

For time courses, the value of each pixel was calculated first using the following functions: dR/R=(Fx-F0)/F0, where F0 is the average reflectance value of the first two frames (taken before visual stimuli onset), Fx is the reflectance value corresponding to frame X (X = 1, 2, 3...). To examine the response of a domain, we generated domain timecourses by averaging timecourse of all significant pixels within the domain (compared to blank, p<0.01, two-tailed t test; see Figures 2 and 5–7). To compare the responses of a domain to two stimulus conditions, we conducted a nonparametric test (Wilcoxon test, see Figure 2L) for values within frames 9–18. For some comparisons, the average of response groups were compared (e.g. all curved vs all straight).

Similarity

To evaluate the similarity between two single condition maps, we extract the responses in the imaged area V4 (regions that could be activated by curvature stimuli during the experiment) and calculated the correlation coefficient values between the two response maps (see Figures 3 and 4). To evaluate the relationship between curvature degree difference and response similarity, we use the Regression function from Matlab. In this analysis, we use the condition number to represent the relative curvature degree of each curvature (see Figure 3). The curvature degree difference is based on the relative distance between two curvature degrees.

Statistical analysis

For all statistical comparisons of functional maps, we use two-tailed t test or ANOVA. We used the Wilcoxon rank sum for the comparisons between response amplitudes in two conditions when data failed to show a normal distribution.

Acknowledgements

This research was conducted at Zhejiang University and was supported by the National Key R & D Program of China (2018YFA0701400 to AWR), the National Natural Science Foundation of China (8191101288, 31627802, 81430010 to AWR), and the Fundamental Research Funds for the Central Universities (2019XZZX003-20 to AWR, 2020FZZX001-05 to SXM). No US funds (NIH, OHSU) were used to support these experiments or preparation of this manuscript.

Funding Statement

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Contributor Information

Anna Wang Roe, Email: annawang@zju.edu.cn.

Joshua I Gold, University of Pennsylvania, United States.

Kristine Krug, University of Oxford, United Kingdom.

Funding Information

This paper was supported by the following grants:

National Key R & D Program of China 2018YFA0701400 to Anna Wang Roe.
National Natural Science Foundation of China 8191101288 to Anna Wang Roe.
National Natural Science Foundation of China 31627802 to Anna Wang Roe.
National Natural Science Foundation of China 81430010 to Anna Wang Roe.
The Fundamental Research Funds for the Central Universities 2019XZZX003-20 to Anna Wang Roe.
The Fundamental Research Funds for the Central Universities 2020FZZX001-05 to Xue Mei Song.

Additional information

Competing interests

No competing interests declared.

Author contributions

Data curation, Software, Formal analysis, Validation, Investigation, Visualization, Methodology, Writing - original draft, Writing - review and editing.

Data curation, Investigation, Writing - original draft.

Investigation.

Conceptualization, Supervision, Funding acquisition, Validation, Writing - original draft, Project administration, Writing - review and editing.

Ethics

Animal experimentation: All procedures were performed in accordance with the National Institutes of Health Guidelines and were approved by the Zhejiang University Institutional Animal Care and Use Committee with the approved protocols (Permit Number:zju20160242).

Additional files

Transparent reporting form

elife-57261-transrepform.docx^{(59.7KB, docx)}

Data availability

All data generated or analysed during this study are included in the manuscript and supporting files.

References

Anzai A, Peng X, Van Essen DC. Neurons in monkey visual area V2 encode combinations of orientations. Nature Neuroscience. 2007;10:1313–1321. doi: 10.1038/nn1975. [DOI] [PubMed] [Google Scholar]
Bao P, She L, McGill M, Tsao DY. A map of object space in primate inferotemporal cortex. Nature. 2020;583:103–108. doi: 10.1038/s41586-020-2350-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
Blasdel GG, Salama G. Orientation selectivity, preference, and continuity in monkey striate cortex. Nature. 1986;321:579–585. doi: 10.1523/JNEUROSCI.12-08-03139.1992. [DOI] [PMC free article] [PubMed] [Google Scholar]
Bosking WH, Zhang Y, Schofield B, Fitzpatrick D. Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. The Journal of Neuroscience. 1997;17:2112–2127. doi: 10.1523/JNEUROSCI.17-06-02112.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
Brincat SL, Connor CE. Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nature Neuroscience. 2004;7:880–886. doi: 10.1038/nn1278. [DOI] [PubMed] [Google Scholar]
Chen M, Yan Y, Gong X, Gilbert CD, Liang H, Li W. Incremental integration of global contours through interplay between visual cortical Areas. Neuron. 2014;82:682–694. doi: 10.1016/j.neuron.2014.03.023. [DOI] [PubMed] [Google Scholar]
Chen M, Li P, Zhu S, Han C, Xu H, Fang Y, Hu J, Roe AW, Lu HD. An orientation map for motion boundaries in macaque V2. Cerebral Cortex. 2016;26:279–287. doi: 10.1093/cercor/bhu235. [DOI] [PMC free article] [PubMed] [Google Scholar]
Dobbins A, Zucker SW, Cynader MS. Endstopped neurons in the visual cortex as a substrate for calculating curvature. Nature. 1987;329:438–441. doi: 10.1038/329438a0. [DOI] [PubMed] [Google Scholar]
Fang Y, Chen M, Xu H, Li P, Han C, Hu J, Zhu S, Ma H, Lu HD. An orientation map for Disparity-Defined edges in area V4. Cerebral Cortex. 2019;29:666–679. doi: 10.1093/cercor/bhx348. [DOI] [PubMed] [Google Scholar]
Ghose GM, Ts'o DY. Form processing modules in primate area V4. Journal of Neurophysiology. 1997;77:2191–2196. doi: 10.1152/jn.1997.77.4.2191. [DOI] [PubMed] [Google Scholar]
Ghose GM, Ts'o DY. Integration of color, orientation, and size functional domains in the ventral pathway. Neurophotonics. 2017;4:031216. doi: 10.1117/1.NPh.4.3.031216. [DOI] [PMC free article] [PubMed] [Google Scholar]
Grinvald A, Lieke E, Frostig RD, Gilbert CD, Wiesel TN. Functional architecture of cortex revealed by optical imaging of intrinsic signals. Nature. 1986;324:361–364. doi: 10.1038/324361a0. [DOI] [PubMed] [Google Scholar]
Hegdé J, Van Essen DC. Strategies of shape representation in macaque visual area V2. Visual Neuroscience. 2003;20:313–328. doi: 10.1017/S0952523803203102. [DOI] [PubMed] [Google Scholar]
Hegdé J, Van Essen DC. A comparative study of shape representation in macaque visual Areas v2 and v4. Cerebral Cortex. 2007;17:1100–1116. doi: 10.1093/cercor/bhl020. [DOI] [PubMed] [Google Scholar]
Hubel DH, Livingstone MS. Segregation of form, color, and Stereopsis in primate area 18. The Journal of Neuroscience. 1987;7:3378–3415. doi: 10.1523/JNEUROSCI.07-11-03378.1987. [DOI] [PMC free article] [PubMed] [Google Scholar]
Hubel DH, Wiesel TN. Receptive fields and functional architecture in two nonstriate visual Areas (18 and 19) OF the cat. Journal of Neurophysiology. 1965;28:229–289. doi: 10.1152/jn.1965.28.2.229. [DOI] [PubMed] [Google Scholar]
Hubel DH, Wiesel TN. Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology. 1968;195:215–243. doi: 10.1113/jphysiol.1968.sp008455. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ito M, Komatsu H. Representation of angles embedded within contour stimuli in area V2 of macaque monkeys. Journal of Neuroscience. 2004;24:3313–3324. doi: 10.1523/JNEUROSCI.4364-03.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
Jiang R, i M L, Tang S. Discrete neural clusters encode orientation, curvature and 761 corners in macaque V4. bioRxiv. 2019 doi: 10.1101/808907. [DOI]
Li P, Zhu S, Chen M, Han C, Xu H, Hu J, Fang Y, Lu HD. A motion direction preference map in monkey V4. Neuron. 2013;78:376–388. doi: 10.1016/j.neuron.2013.02.024. [DOI] [PubMed] [Google Scholar]
Lu Y, Yin J, Chen Z, Gong H, Liu Y, Qian L, Li X, Liu R, Andolina IM, Wang W. Revealing detail along the visual hierarchy: neural clustering preserves acuity from V1 to V4. Neuron. 2018;98:417–428. doi: 10.1016/j.neuron.2018.03.009. [DOI] [PubMed] [Google Scholar]
Murphy TM, Finkel LH. Shape representation by a network of V4-like cells. Neural Networks. 2007;20:851–867. doi: 10.1016/j.neunet.2007.06.004. [DOI] [PubMed] [Google Scholar]
Nandy AS, Sharpee TO, Reynolds JH, Mitchell JF. The fine structure of shape tuning in area V4. Neuron. 2013;78:1102–1115. doi: 10.1016/j.neuron.2013.04.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
Nandy AS, Mitchell JF, Jadi MP, Reynolds JH. Neurons in macaque area V4 are tuned for complex Spatio-Temporal patterns. Neuron. 2016;91:920–930. doi: 10.1016/j.neuron.2016.07.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
Op de Beeck HP, Deutsch JA, Vanduffel W, Kanwisher NG, DiCarlo JJ. A stable topography of selectivity for unfamiliar shape classes in monkey inferior temporal cortex. Cerebral Cortex. 2008;18:1676–1694. doi: 10.1093/cercor/bhm196. [DOI] [PMC free article] [PubMed] [Google Scholar]
Pasupathy A, Connor CE. Shape representation in area V4: position-specific tuning for boundary conformation. Journal of Neurophysiology. 2001;86:2505–2519. doi: 10.1152/jn.2001.86.5.2505. [DOI] [PubMed] [Google Scholar]
Pasupathy A, Connor CE. Population coding of shape in area V4. Nature Neuroscience. 2002;5:1332–1338. doi: 10.1038/972. [DOI] [PubMed] [Google Scholar]
Ponce CR, Hartmann TS, Livingstone MS. End-Stopping predicts curvature tuning along the ventral stream. The Journal of Neuroscience. 2017;37:648–659. doi: 10.1523/JNEUROSCI.2507-16.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
Popivanov ID, Jastorff J, Vanduffel W, Vogels R. Tolerance of macaque middle STS body patch neurons to shape-preserving stimulus transformations. Journal of Cognitive Neuroscience. 2015;27:1001–1016. doi: 10.1162/jocn_a_00762. [DOI] [PubMed] [Google Scholar]
Pospisil DA, Pasupathy A, Bair W. 'Artiphysiology' reveals V4-like shape tuning in a deep network trained for image classification. eLife. 2018;7:e38242. doi: 10.7554/eLife.38242. [DOI] [PMC free article] [PubMed] [Google Scholar]
Ramsden BM, Hung CP, Roe AW. Real and illusory contour processing in area V1 of the primate: a cortical balancing act. Cerebral Cortex. 2001;11:648–665. doi: 10.1093/cercor/11.7.648. [DOI] [PubMed] [Google Scholar]
Sceniak MP, Hawken MJ, Shapley R. Visual spatial characterization of macaque V1 neurons. Journal of Neurophysiology. 2001;85:1873–1887. doi: 10.1152/jn.2001.85.5.1873. [DOI] [PubMed] [Google Scholar]
Tang R, Song Q, Li Y, Zhang R, Cai X, Lu HD. Curvature-processing domains in primate V4. eLife. 2020;9:e57502. doi: 10.7554/eLife.57502. [DOI] [PMC free article] [PubMed] [Google Scholar]
Tanigawa H, Lu HD, Roe AW. Functional organization for color and orientation in macaque V4. Nature Neuroscience. 2010;13:1542–1548. doi: 10.1038/nn.2676. [DOI] [PMC free article] [PubMed] [Google Scholar]
Wei H, Dong Z, Wang L. V4 shape features for contour representation and object detection. Neural Networks. 2018;97:46–61. doi: 10.1016/j.neunet.2017.09.010. [DOI] [PubMed] [Google Scholar]
Yue X, Pourladian IS, Tootell RB, Ungerleider LG. Curvature-processing network in macaque visual cortex Curvature-processing network in macaque visual cortex. PNAS. 2014;111:E3467–E3475. doi: 10.1073/pnas.1412616111. [DOI] [PMC free article] [PubMed] [Google Scholar]

eLife. doi: 10.7554/eLife.57261.sa1

Decision letter

Editor: Kristine Krug¹

Reviewed by: Aniruddha Das², Ed Connor³

In the interests of transparency, eLife publishes the most substantive revision requests and the accompanying author responses.

Acceptance summary:

Curvature is a fundamental component of our ability to recognise and discriminate visual objects and therefore interact with the world around us. This optical imaging study compellingly demonstrates a systematic functional architecture of specialised domains for processing and representing curvature in the visual cortex of primates, in particular visual area V4. This finding has profound implications for our understanding of how primates perceive and recognise shape.

Decision letter after peer review:

Thank you for submitting your article "Curvature domains in V4 of Macaque Monkey" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Joshua Gold as the Senior Editor. The following individuals involved in review of your submission have agreed to reveal their identity: Aniruddha Das (Reviewer #2); Ed Connor (Reviewer #3).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

As the editors have judged that your manuscript is of interest, but as described below that substantial additional analyses are required before it is published, we would like to draw your attention to changes in our revision policy that we have made in response to COVID-19 (https://elifesciences.org/articles/57162). First, because many researchers have temporarily lost access to the labs, we will give authors as much time as they need to submit revised manuscripts. We are also offering, if you choose, to post the manuscript to bioRxiv (if it is not already there) along with this decision letter and a formal designation that the manuscript is "in revision at eLife". Please let us know if you would like to pursue this option. (If your work is more suitable for medRxiv, you will need to post the preprint yourself, as the mechanisms for us to do so are still in development.)

Summary:

In this manuscript, Hu and colleagues investigate the functional organisation of important elements for representing object shape in the primate brain. Their focus of investigation is the mapping of curvature in extrastriate visual area V4, an area in which there is substantial evidence for neurons representing intermediate visual features that are more complex than the representations of single orientation in receptive fields of V1 and V2. The authors show that there are clearly regions of V4 that are activated better by curvature stimuli than traditional single orientation stimuli and thus provide potentially exciting and novel data about organization of curvature orientation in area V4. Curvature tuning is widely regarded as a major step toward complex shape perception that arises specifically in V4, the third stage in primate ventral pathway.

However, there are some major concerns about the scope and specificity of the presented data and data analyses as well as the discussion of the results, which currently limit the ability of the reader to fully appreciate to what extent the organisation for curvature: (a) is truly specialised across V4; (b) cannot be explained by end stopped cells, clustering of orientation-tuning or size tuning; and (c) is distinct from the neuronal properties found in earlier visual areas V1 and V2. In general, we suggest the authors present their entire dataset, use the same analyses across the full range of imaging data from V4 and potentially extend to V1 and V2, and use statistical measures to analyse the responses around curvature clusters and pin wheels to establish the existence of gradients for curvature parameters. More detailed comments and suggestions are presented below. We think that these revisions and additional analyses should bring out the evidence of a specialised, spatially organized representation of curvature more clearly.

Essential revisions:

1) Additional analyses to strengthen the case for specialised curvature organisation in V4.

In some parts of the paper (Figures 3 and 5), the authors rely on pixel-wise correlation as a proxy for mapping. But pixel correlation conveys no information about spatial organization, and high correlations are equally compatible with entirely random organization of signals. For example, any pattern (even a scramble) is likely to preferentially activate some neurons by chance and therefore some pattern of activation across the cortex (e.g. Figure 2—figure supplement 6C). If a different pattern (like another scrambled) is presented it is likely that a different pattern is obtained, and if I do any kind of morphing between these images, I am necessarily going to gradually shift the pattern of activation from one to the other. But this says little about the nature of representation. There is also an issue of map noise: within condition consistency is often below 0.7, and it's not clear why.

Only graphical analysis of images showing spatial clustering of similar tuning, spatial relationships between clusters, and/or spatial gradients of tuning changes are meaningful for understanding mapping, and this paper only gets to that stage in Figure 7, and even then only for one relatively small region of one case. The critical results shown there (and the corresponding but unanalyzed results in Figures 6A and S9A,B) are the only convincing evidence of organization within V4 curvature domains, and that organization in turn is the most convincing evidence that curvature per se is represented rather than something correlated with curvature.

Even these plots confound two phenomena. One phenomenon appears to be true clustering of curvature direction tuning. The strongest prediction for curvature direction tuning is that opposite directions (differing by 180 degrees) should be the most separated. This is clearest, surprisingly, in Figures S9A and B, where regions tuned for the up direction are largely complementary to regions tuned for down. This convinces me that the authors have observed organization for curvature direction, and this case in particular deserves to be featured in the main text and analyzed as in Figure 7.

In contrast, especially in Figures 6 and 7, there are many regions where the responses to opposite directions are largely overlapping, and instead it is directions that are 90 degrees different that are most complementary. In those cases, the same straight to curved gradients will be observed for two opposite directions (as is the case for the central three gradients in Figure 7C). The simplest interpretation of this is tuning for component orientations, because curved gratings, especially in the higher curvature domain, share component orientations across 180 degree differences, but have distinct orientations across 90 degree differences. A more complex interpretation, but one supported by previous literature, is that some domains imaged here represent contours with multiple curvatures. This would be consistent with findings that many V4 neurons represent contour fragments containing multiple curvature directions (Pasupathy and Connor, 2001; Carlson et al., 2011). These are frequently curvatures with opposite directions, since many contour fragments contain adjoining convex and concave curves, which out of geometric necessity tend to point in opposite directions. In this sense, the three gradients in the center of 7C, and similar gradients (not analyzed) in Figure 6 may reflect the geometric structure of the natural world.

Our strong suggestion would be to analyze all the available imaging regions across V4 (not just individual cases or smaller regions within cases) with the analyses in Figure 7, making clear distinctions between gradients that are specific to a single curvature direction and those that overlap for two curvature directions, and discussing possible interpretations of the latter case.

2) Additional analyses to elucidate the functional nature of curvature domains and the "pinwheel" structures.

The lack of single unit data here is problematic, especially given the authors' model, because a curvature "region", especially given its proximity to similar orientation regions, could simply be comprised of a mix of oriented cells with no individual cells showing curvature sensitivity. Looking at Figure 5, all curvatures are somewhat correlated with each other and with many orientations; this is clearly not the case for orientation in which there is basically no overlap between orientations. The model the authors present relies on the proximity of (and overlap between) curvature and orientation regions, but a "multi-orientation" region that includes individual neurons with a diversity of orientations tunings, but has no neurons with actual curvature preferences, would be equally plausible and consistent.

Because there is no single unit data, the regions of "curvature" activation could simply result from the co-activation of classically oriented cells (especially near pinwheel centers where such cells are likely to be colocalized). The authors are concerned that the presence of high spatial frequencies in the curvature stimuli is a confound, but there's no attempt to control for the increase in orientation content with curvature. Strong curvature contains a number of orientations, and we would therefore expect to activate multiple orientation columns, but this is not controlled for.

We believe it would be straightforward to control for: on a pixel-by-pixel basis see how responses to a curve could be explained by a weighted sum of the responses to the orientations that it contains. For example, small curvature responses should be similar to single orientation responses, while high curvature responses should be explainable by a mix of orientation responses (which according to map correlation data seems very reasonable). And an exact weighting of what orientations are present in a given curvature stimulus, can be done by looking at the 2-d Fourier transform of the stimulus.

Also, a similar analysis to that described in (point 1 above / Figure 7 of the manuscript) should be done to look for pinwheel structures, i.e. with overlapping contour maps and with gradient lines, and with clear presentation of the statistical significance of gradients presented prominently in the main paper. This may support the idea of pinwheel structures for curvature direction, in a way that would be graphically and statistically convincing. In contrast, the basis for the pinwheel maps in Figure 7D is unclear, so they are not appropriate for inclusion as presented. Even if pinwheels cannot be clearly identified, and if not all images show clear clustering for tuning, the main point of the paper would be established.

3) Extending analyses to available imaging data beyond V4 to strengthen the case that the organisation for curvature in V4 is different from functional representations in V1 and V2.

We were wondering why V1 and V2 were excluded. It looks like the authors are throwing out 2/3 of the data, and a vital component to making the case that there's something "special" going on in V4 is comparing it to other areas. On a similar note, V1 in case 2 looks problematic: there's no blob like activation of color and orientation columns aren't obvious (especially compared to the textbook examples shown in cases 1 and 3). Another exclusion issue: Figures 2 3, and 5 place letters so they obscure a substantial fraction of the imaged region.

A number of pertinent experimental details are also missing: there are 3 animals, but how many sessions, are there analyses in which the observations of multiple sessions are combined, and are there comparisons being made across sessions? Are stimuli presented binocularly? What are the RF locations of the V1, V2, and V4 regions?

Please extend the central analysis applied to V4 to V2 and V1 data and complete the requested information.

4) Discussing the results in the context of previous findings for single neuron tuning to show more clearly what other interpretations of the presented results are possible or can be rejected.

The curvature maps appear convincing: there are clearly regions of neurons that are activated better by curvature stimuli than traditional single orientation stimuli. However a simple explanation, dating back to 1965, of this would be the presence of end-stopped and surround suppression, and this is not addressed. The historical context seems a little off here; Livingstone's findings of 2017 are cited, but somehow Hubel and Wiesel, who in 1965 (J Neurophysiol) suggested that end-stopped, length tuned neurons could be useful for curvature detection, is omitted. There was a nice explicit model of how that could happen by Dobbins et al. in 1987 (Nature). But the manuscript makes no reference to any size tuning or surround suppression, and how that, in combination with classic orientation tuning, could create a cell that responds vigorously to curvature. This is even a greater concern given the well-established prevalence of size tuning, both electrophysiologically (1987) and with optical imaging (1997), in area V4. This has a huge impact on both novelty and interpretability; if you have a region of surround suppression (which we know exist) and it overlaps with an orientation region, there's a "curvature" region in that it will respond to a curve better than extended straight line or grating. But I would argue it's not a "curvature" region, since a short bar or grating without any curvature is actually the optimal stimulus. The absence of any reports of eccentricity or size tuning measurements does not install confidence that this is not what is responsible for these observations.

We think a clearer discussion of these issues would be important for the paper.

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Thank you for resubmitting your article "Curvature domains in V4 of Macaque Monkey" for consideration by eLife. Your revised article has been reviewed by two peer reviewers, and the evaluation has been overseen by a Reviewing Editor and Joshua Gold as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Aniruddha Das (Reviewer #2).

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.

We would like to draw your attention to changes in our revision policy that we have made in response to COVID-19 (https://elifesciences.org/articles/57162). Specifically, when editors judge that a submitted work as a whole belongs in eLife but that some conclusions require a modest amount of additional new analysis/data, as they do with your paper, we are asking that the manuscript be revised to either limit claims to those supported by data in hand, or to explicitly state that the relevant conclusions require additional supporting data.

Our expectation is that the authors will eventually carry out the additional experiments and report on how they affect the relevant conclusions either in a preprint on bioRxiv or medRxiv, or if appropriate, as a Research Advance in eLife, either of which would be linked to the original paper.

Summary:

Revisions for this paper:

The manuscript is largely improved, and the authors have effectively addressed some of the major concerns raised in the initial submission. For instance, the authors addressed the concern that the curvature is fully accounted for by end stopping / surround suppression. They show that short straight lines poorly stimulate “curvature” regions and that “end stopped” patches are substantially distinct from “curvature” patches even though they overlap partially (Figure 2—figure supplement 13). Additionally, while showing that end stopping and curvature are not identical, the authors are careful in stating that end stopping could play an important role in curvature processing. But an important concern appears incompletely addressed. The evidence for the curvature specific domains is strong and novel, however the reviewers felt that the claims regarding the specialised architecture of these domains, namely the ordered, gradual representation of different degrees of curvature and the existence of pinwheels have not been fully supported by the presented results. We suggest either addressing the additional evidence suggested in revisions or tempering these claims and addressing these issues in a subsequent, linked publication.

1) One issue that is not satisfactorily addressed is “curvature pinwheels”. The authors make a point of proposing that pinwheel structure is fundamental to the organization of maps like orientation (in V1) and curvature (V4). While that proposal could arguably be made for V1 (Li, 2019) the authors' own evidence here suggests that pinwheels might be a red herring for curvature in V4.

a) The authors only show pinwheels in one data set (Figure 7B(right hand panel). And there too only the single pinwheel in the upper curvature patch looks reasonable. In the lower curvature patch the upper pinwheel lies very close to the edge of the curvature region making the classification problematic given the ~250-micron FWHM spatial resolution of intrinsic signal optical imaging (Polimeni, PNAS 2005).

b) More importantly, the most striking characteristic in the “curvature orientation” maps appears to be the high degree of overlap between regions of opposite curvature direction (Figure 6D). The net “curvature orientation” vector in these regions of overlap are presumably close to zero vector length. The maps (e.g. Figure 7B) with saturated colors are thus likely misleading as the real pattern likely involves not point singularities of “curvature orientation” (“pinwheels”) but large overlap patches of poor net “curvature orientation” with short “curvature orientation”-specific extensions outwards. Notably the authors did not show “curvature-orientation” contour maps, quantify the local orientation gradient and test it for significance to support their proposal of pinwheels. It is likely that the signal to noise in such maps would be very low.

c) In the Discussion, it may be worth considering if the pattern of large patches of overlap with orientation-specific extensions may be the dominant topological motif, rather than trying to squeeze the map into a straitjacket of pinwheels.

2) With respect to the gradual shift in curvature maps, the response to the point of of pixel-wise map correlation was not adequate: the authors' response states that it merely establishes a curvature map, but any superposition of single condition domains, or differential analysis between different curvatures, establishes this. The manuscript says something more: there is a shifting map, implying some sort of continuity of representation. As stated in the previous comments, this strikes us as poor logic: if we take a low frequency, chromatic, linear grating, at one extreme, and a high frequency achromatic curve at another, the two activation maps will look different, and if I do a pixel wise morphing between those two stimuli that doesn't know anything about high level attributes like color, curvature, and SF, I necessarily have to create activation maps that look intermediate. It may be that subtle non-continuities, like fractures and pinwheels, in functional maps will make the change in correlations non-linear, but I'm not even sure of that. In other words, this analysis has to succeed, and is therefore, at least as it is presented, uninformative. If the authors are interested in the regularity or periodicity of representations, that a correlation analysis of cortical distance vs representational distance, seems more appropriate. For example, the spatial correlation of curvature preference on a pixel-wise basis, or between "centers" of domains. But, on the basis of what is presented, we are not sure how that is actually going turn out. For example, Figure 7 seems to show that 3/4 of the center of masses of "up" and "down" curvatures are basically on top of one another.

Revisions expected in follow-up work:

The reviewers felt that the claims regarding the specialised architecture of these domains, namely the ordered, gradual representation of different degrees of curvature and the existence of pinwheels have not been fully supported by the presented results. We suggest either addressing the additional evidence suggested in revisions or tempering these claims and addressing these issues in a subsequent, linked publication. See previous section for details.

eLife. 2020 Nov 19;9:e57261. doi: 10.7554/eLife.57261.sa2

Author response

Essential revisions:

1) Additional analyses to strengthen the case for specialised curvature organisation in V4.

In some parts of the paper (Figures 3 and 5), the authors rely on pixel-wise correlation as a proxy for mapping. But pixel correlation conveys no information about spatial organization, and high correlations are equally compatible with entirely random organization of signals. For example, any pattern (even a scramble) is likely to preferentially activate some neurons by chance and therefore some pattern of activation across the cortex (e.g. Figure 2—figure supplement 6C). If a different pattern (like another scrambled) is presented it is likely that a different pattern is obtained, and if I do any kind of morphing between these images, I am necessarily going to gradually shift the pattern of activation from one to the other. But this says little about the nature of representation. There is also an issue of map noise: within condition consistency is often below 0.7, and it's not clear why.

Correlations between pairs of curvature stimuli:

1) 0.7 is not low. As shown in Figure 3—figure supplement 1, the comparisons of matched straight orientation maps, which are the best possible correlation indices, range from 0.66 to 0.9. In Figures 3 and 4 (previous Figures 3 and 5), the values for matched curvature maps, based on correlation between two conditions, also fall within this range (0.68-0.91). In our preparations, with the signal sizes recorded in V4 under anesthesia, and with data collected over hours of time, these values are typical. Figure 3—figure supplement 1 provides a benchmark by which to evaluate the value of the correlations. These values have now been incorporated in text.

2) Correlation values are related to curvature content. We agree with the reviewer that, in principle, correlations can result from non-curvature related responses, and even scrambled responses. However, here, we start with the observation that maps are differentially responsive to different degrees of curvature grating (curvature degree, Figure 3; curvature orientation, Figure 4). Then we show that these maps are systematically shifting using map correlations. We spend the rest of the manuscript providing additional support that these maps are truly curvature-specific responses. When considered in its entirety, our data supports the existence of curvature maps in V4.

Only graphical analysis of images showing spatial clustering of similar tuning, spatial relationships between clusters, and/or spatial gradients of tuning changes are meaningful for understanding mapping, and this paper only gets to that stage in Figure 7, and even then only for one relatively small region of one case. The critical results shown there (and the corresponding but unanalyzed results in Figures 6A and S9A,B) are the only convincing evidence of organization within V4 curvature domains, and that organization in turn is the most convincing evidence that curvature per se is represented rather than something correlated with curvature.

Paper organization: In response to the general critique that the maps for curvature domains come late in the paper, we think it is worthwhile to explain our general strategy. Our approach was to understand the organization of curvature response from global to local scale in V4. (1) We first show that response to curvature (preference for the curvature stimulus over the straight stimulus) falls within the “orientation bands” and not the “color” bands, and “curvature” domains are distinct from orientation domains. This finding alone is new (Figure 2). (2) We then examine, at the domain map scale, the organization and relationship with straight orientation maps. This provided some support for the possibility that responses might be related to curvature degree (Figures 3: Case 1, S11: Case 2, shifting correlations related to curvature degree) or to curvature orientation (new Figures 4: Case 2, S11: Case 1, higher when matched). Based on these global signals, V4 can distinguish different curvature degrees and curvature orientations. (3) To further examine whether these correlations are due to the curvature content, we analyzed the topographic region of V4 corresponding to the central (~1 degree) of the curvature stimulus (total stimulus size is 4 deg), which is the part with high curvature content. This revealed substructure within the putative curvature domains with preferential response for curvature degree and orientation (Figures 5, 6). (4) Finally, we made an attempt to provide some hypothesis about the global organization of curvature representation (Figure 7G). We concluded that, when taken all together, our data are consistent with the hypothesis that there is a systematic “curvature map” within V4. This is the first study showing such systematic maps at domain scale (cf. Yue et al., 2014, showed large regions responsive to curvature but did not report maps for curvature degree or curvature orientation. Tang et al: we recently learned of this study under review at eLife and have cited this study).This strategy is now described in the beginning of the Results under “General Approach”.

Even these plots confound two phenomena. One phenomenon appears to be true clustering of curvature direction tuning. The strongest prediction for curvature direction tuning is that opposite directions (differing by 180 degrees) should be the most separated. This is clearest, surprisingly, in Figures S9A and B, where regions tuned for the up direction are largely complementary to regions tuned for down. This convinces me that the authors have observed organization for curvature direction, and this case in particular deserves to be featured in the main text and analyzed as in Figure 7.

Adding Figure S9 to main text: complementary representation of up/down orientation: As the reviewers suggest, we have now added Figure S9 to Figure 6 (the new Figure 6A-C), and describe it in the main text. Figure 6A illustrates a region where opposing orientations (illustrated with both curved gratings and curved lines) are largely non overlapping. Figure 6A-C is now described in the text.

In contrast, especially in Figures 6 and 7, there are many regions where the responses to opposite directions are largely overlapping, and instead it is directions that are 90 degrees different that are most complementary. In those cases, the same straight to curved gradients will be observed for two opposite directions (as is the case for the central three gradients in Figure 7C). The simplest interpretation of this is tuning for component orientations, because curved gratings, especially in the higher curvature domain, share component orientations across 180 degree differences, but have distinct orientations across 90 degree differences. A more complex interpretation, but one supported by previous literature, is that some domains imaged here represent contours with multiple curvatures. This would be consistent with findings that many V4 neurons represent contour fragments containing multiple curvature directions (Pasupathy and Connor, 2001; Carlson et al., 2011). These are frequently curvatures with opposite directions, since many contour fragments contain adjoining convex and concave curves, which out of geometric necessity tend to point in opposite directions. In this sense, the three gradients in the center of 7C, and similar gradients (not analyzed) in Figure 6 may reflect the geometric structure of the natural world.

Curvature domain response not due to component orientations. Our evidence suggests these curvature responses are difficult to attribute simply to component orientations. We test this by examining whether curvature domains exhibit strong response to the 0,45,135 deg components of the curvature stimulus. In Figure 7—figure supplement 2A-D, we show that, while there are locations with robust response to straight stimuli (red lines in B,C: 0,45,135 graphs), the curvature domains exhibit little response to component straight stimuli. Instead, they show strong response only to curvatures; these responses are graded with changing curvature degree (dark to light gray lines). This suggests that the curvature response is not primarily a response to components of the curve. We also show (see “Not due to weighted sum” below) that the curvature response is not a weighted sum of straight responses. We have added text in Results under “Curvature domains are not pinwheel centers or sums of orientation components”.

Some V4 domains may represent contours with multiple curvatures or other invariant responses.

As the reviewer states, V4 neurons can represent contour fragments containing multiple curvature orientations (Pasupathy and Connor, 2001; Carlson et al., 2011). In this paper, we have taken an initial look at the organizations related to simple curvature degree and curvature orientation. We do not understand nor have we explored other possible combinations of different curvature response that may exist in V4. We believe that there may be higher order integrations reflecting combinations of curvatures or cue-invariant curvature response or even simple shapes (cf Tang et al. submitted). This parallels our thinking in V2, where there may be distinct domains responsive to single cue (e.g. motion dot borders) vs invariant multi-cue (e.g. regardless of which type of cue) border domains (Ramsden et al., 2015). Such possibilities will require further study.

There are indications within our data of possible higher order responses that are activated by different combinations of curvature orientations. For example, we illustrate locations where pixels are responsive to both upwards and downwards or both leftwards and rightwards curvature orientations (new Figure 6D brown regions). The percentage of such overlapping pixels is relatively small (overlapping pixels vs. all colored pixels was on average 22.4%; Case 1: 32.1%; Case 2: 14.9%; Case 3: 20.3%). Thus, although there were “complex” pixels responsive to multiple orientations, representation of opposing curvature orientations was mostly non-overlapping.

In sum, we believe that the organization is likely to contain further complexity and requires further study. We indeed embrace the idea that representation of convex and concave “shape fragments” may derive from statistics of natural stimuli (e.g. David et al., 2004 J Neurosci; Movshon and Simoncelli, 2015). Hebbian-based principles may lead to establishment of systematic domain-based maps that reflect statistics of the natural environment. Such domain diversity would be consistent with a mid-tier stage of shape representation.

Our strong suggestion would be to analyze all the available imaging regions across V4 (not just individual cases or smaller regions within cases) with the analyses in Figure 7, making clear distinctions between gradients that are specific to a single curvature direction and those that overlap for two curvature directions, and discussing possible interpretations of the latter case.

As we mentioned above, our strategy was to focus on the region responsive to the central curvature part of our stimulus. Figures 4-7 are focused on this central representation. As shown in Figure 7—figure supplement 2A, we purposely focused on the central region (~1 deg, red box) of the 4 deg curvature stimulus. This is the region that contains the greatest changing curvature across curvature degree stimuli. In addition, the central region experiences relatively constant curvature orientation and spatial frequency across conditions, unlike the flanking parts of the stimulus (blue box), which contain large spatial frequency and orientation differences across conditions). Thus, the non-central regions are distinct in that they contain relatively straight components and a broader mixture of orientations and spatial frequencies across conditions. This has been clarified in Results under “General Approach” and “the central ~1 deg representation” stated where appropriate in the text.

V1/V2 and V4 represent different retinotopic locations in the same field of view. In each experiment, we identified cortical location corresponding to central ~1 deg of the stimulus. To determine the region of cortex corresponding to this central region, we first mapped, monocularly, the RF location of the V4 region (Figure 2—figure supplement 2), using vertical and horizontal 0.2° wide bars (vertical: different x; horizontal: different y, Figure 2—figure supplement 2A,G,M). In each case, the central ~1 deg stimulus covered a cortical area roughly ~4-5mm in diameter; we therefore evaluated curvature response from this small region. In some experiment sessions, we also presented a flashed white square (size=1°) at the determined V4 center (X0, Y0) to estimate the coverage area of the central part of the stimulus (See Figure 2—figure supplement 2H and M). This is now explicitly stated in text and retinotopic mapping methods described in text and Materials and methods under “Retinotopic mapping”.

2) Additional analyses to elucidate the functional nature of curvature domains and the "pinwheel" structures.

The lack of single unit data here is problematic, especially given the authors' model, because a curvature "region", especially given its proximity to similar orientation regions, could simply be comprised of a mix of oriented cells with no individual cells showing curvature sensitivity. Looking at Figure 5, all curvatures are somewhat correlated with each other and with many orientations; this is clearly not the case for orientation in which there is basically no overlap between orientations. The model the authors present relies on the proximity of (and overlap between) curvature and orientation regions, but a "multi-orientation" region that includes individual neurons with a diversity of orientations tunings, but has no neurons with actual curvature preferences, would be equally plausible and consistent.

Because there is no single unit data, the regions of "curvature" activation could simply result from the co-activation of classically oriented cells (especially near pinwheel centers where such cells are likely to be colocalized). The authors are concerned that the presence of high spatial frequencies in the curvature stimuli is a confound, but there's no attempt to control for the increase in orientation content with curvature. Strong curvature contains a number of orientations, and we would therefore expect to activate multiple orientation columns, but this is not controlled for.

Curvature domains are not pinwheel centers. The reviewer raises a good point, concerning the possibility that curvature responses could be due to activation of a mixture of oriented responses, such as that found at orientation pinwheel centers. (1) As shown in the following figure (new Figure 2—figure supplement 12), the curvature domains (orange regions) do not co-localize with orientation pinwheel centers (blue dots). In fact, most of the pinwheel centers are outside the curvature domains entirely. (2) Moreover, as shown by the responses of pinwheel centers to straight and curved stimuli (E), the pinwheel center responses to curvatures are weak and do not distinguish curvature from straight (Wilcoxon test, p=0.13). This makes it unlikely for pinwheel centers to be locations of curvature response. We have added text in Results under “Curvature domains are not pinwheel centers or sums of orientation components”.

Lack of electrophysiology: Given the time-consuming nature of our optical imaging data collection, we were not able to focus on electrophysiological characterization. As the goal of this study was to evaluate the functional organization of V4, we focused our efforts on collecting optical imaging data. However, we did collect a small number of units. During imaging of Case 3, we targeted a few penetrations to curvature domains (Figure 2—figure supplement 9). Although the number of neurons recorded is small, our data are consistent with the curvature maps. We found that single-unit and multi-unit responses (n=26) in curvature domains (n=3) preferred curvature stimuli over straight stimuli and exhibited weak responses to straight stimuli. This suggests that curvature domains are not simply comprised of a mix of orientation cells. This is now described in the text.

We believe it would be straightforward to control for: on a pixel-by-pixel basis see how responses to a curve could be explained by a weighted sum of the responses to the orientations that it contains. For example, small curvature responses should be similar to single orientation responses, while high curvature responses should be explainable by a mix of orientation responses (which according to map correlation data seems very reasonable). And an exact weighting of what orientations are present in a given curvature stimulus, can be done by looking at the 2-d Fourier transform of the stimulus.

Not due to a weighted sum. According to reviewers’ comments, we show the cortical responses to the orientations in different curvature stimuli. As shown in the Figures 7—figure supplement 2B-D, the stimuli with low curvature degrees tend to be similar to the single orientation responses. However, for high curvature degrees, the responses of orientation domains were weak, only curvature domains show strong response. Thus the high curvature responses in curvature domains cannot be explained simply by a weighted sum of these orientation responses (2D fourier transform revealed that there are orientation components around 45° and 135° in the curvature stimuli, Figure 7—figure supplement 2A). To do a quick calculation of a weighted sum of these orientation responses, we chose the responses of 0, 45, 135 orientation domains (as these domains cover the range of the orientations in the curvature stimuli). , here Response is the weighted sum of these orientation responses, Response_i represents the response of corresponding orientation domain (i=0, 45, 135) to curvature, Wi represents the weight of different orientations, here we consider these orientations contribute equally, thus Wi=1/3. For upwards curvature (from Figure 7—figure supplement 2C, a/b ratio=7), Response₀=-0.0026%, Response₄₅=-0.0027%, Response₁₃₅=0.0006%, Wi=1/3, Response=-0.0016%, much weaker (by almost an order of magnitude) than the response in curvature domains -0.0115% (responses were calculated by average of the response amplitudes from the last 10 frames). While we are not able to test the weighted sum of all possible orientations, at least with this simple test, we do not see any indication of component summation. We have added text in Results under “Curvature domains are not pinwheel centers or sums of orientation components” and in Discussion.

In summary: We focused on the regions corresponding to the central ~1 deg of the stimuli (where curvature content changes the most across stimuli). (1) We find that the curvature responses are not due to the component orientation signals, as these are located in the orientation domains. In domains with response to high curvature, the responses to orientation are very weak (Figure 7—figure supplement 2). (2) Orientation pinwheel centers and curvature domains are in different spatial locations (Figure 2—figure supplement 12). Orientation pinwheel centers and curvature domains had different response preferences: pinwheels failed to distinguish between curvature and straight, and their responses to curvatures are weak (Figure 2—figure supplement 9). (3) Electrophysiological recordings in curvature domains show that, at least in the small sample of penetrations, neurons predominantly prefer curvature. Thus the responses here are more likely due to the curvature content rather than orientation component features. Substantial text has been added in Discussion under “Testing other possible interpretations”.

Also, a similar analysis to that described in (point 1 above / Figure 7 of the manuscript) should be done to look for pinwheel structures, i.e. with overlapping contour maps and with gradient lines, and with clear presentation of the statistical significance of gradients presented prominently in the main paper. This may support the idea of pinwheel structures for curvature direction, in a way that would be graphically and statistically convincing. In contrast, the basis for the pinwheel maps in Figure 7D is unclear, so they are not appropriate for inclusion as presented. Even if pinwheels cannot be clearly identified, and if not all images show clear clustering for tuning, the main point of the paper would be established.

We agree that the data regarding potential pinwheels is not sufficient and have now removed any mention of curvature pinwheels from the paper.

3) Extending analyses to available imaging data beyond V4 to strengthen the case that the organisation for curvature in V4 is different from functional representations in V1 and V2.

We were wondering why V1 and V2 were excluded. It looks like the authors are throwing out 2/3 of the data, and a vital component to making the case that there's something "special" going on in V4 is comparing it to other areas. On a similar note, V1 in case 2 looks problematic: there's no blob like activation of color and orientation columns aren't obvious (especially compared to the textbook examples shown in cases 1 and 3). Another exclusion issue: Figures 2 3, and 5 place letters so they obscure a substantial fraction of the imaged region.

Clear V1 responses in Case 2. The reviewers ask about the unclear responses in V1 from Case 2 (Figure 2—figure supplement 1D, E). In this case, the cortex underneath the coverglass was not flat, resulting in a band of cortex that was out of the plane of focus. However, distinct blob maps (Figure 2—figure supplement 1D) and some orientation maps (Figure 2—figure supplement 1E) were obtained in V1. In other sessions, we obtained strong V1 response in this Case (see Author response image 1).

Author response image 1. — A. Color map. B. Orientation map.

A number of pertinent experimental details are also missing: there are 3 animals, but how many sessions, are there analyses in which the observations of multiple sessions are combined, and are there comparisons being made across sessions? Are stimuli presented binocularly? What are the RF locations of the V1, V2, and V4 regions?

Missing Details. (1) We did 27 experimental sessions on three hemispheres of two monkeys. Text has been added. We conducted 6-12 sessions in each hemisphere. Each experimental session was focused on answering different questions. (2) We did obtain repeats of some maps. For example, in Figure 2—figure supplement 5, we compared the curvature maps in two different experiment sessions from the same hemisphere. The maps are consistent across time. Text has been added. (3) As we used vecuronium bromide during the experiment, the two eyes did not look at the same position of the monitor, thus for curvature mapping, we covered one eye and only presented the curvature stimuli to one eye. Described in Materials and methods. (4) In every experiment session, we first mapped the RF location of V4 region and presented the stimuli at that location (~1-5 deg eccentricity in V4). For V1, locations were at about 1-2 deg eccentricity (see Figure 2 — figure supplement1). We did not have much V2 available on the surface in these cases (see Figure 2—figure supplement 11). We have now incorporate this into the Materials and methods.

Please extend the central analysis applied to V4 to V2 and V1 data and complete the requested information.

V2 not available on surface. In these three cases, there is limited amount of V2 on the surface; most of V2 is buried within the lunate sulcus (Figure 2—figure supplement 11). Thus we do not have data from V2. Text has been added.

Most of V1 not imaged due to retinotopic mismatch: See Figure 2—figure supplement 2.

V1 does not contain curvature domains. We examined whether V1 contains curvature maps. In two experimental sessions from Case 3 (see Figure 2—figure supplement 10), we positioned the stimulus center over the exposed V1. We first determined the visuotopic location by mapping a 1 deg stimulus (Figure 2—figure supplement 10A, red dotted line, position of stimulus differs by ~1 deg in Experiment 1 and Experiment 2). In both experimental sessions, when the 4 deg curvature stimulus was centered over this location, V1 failed to show any significant preference for curvature over straight (Figure 2—figure supplement 10B). So, this suggests that, unlike V4, V1 does not contain curvature domains; this finding is consistent with previous studies (Yue et al., 2014; Ponce et al., 2017). Text has been added.

4) Discussing the results in the context of previous findings for single neuron tuning to show more clearly what other interpretations of the presented results are possible or can be rejected.

The curvature maps appear convincing: there are clearly regions of neurons that are activated better by curvature stimuli than traditional single orientation stimuli. However a simple explanation, dating back to 1965, of this would be the presence of end-stopped and surround suppression, and this is not addressed. The historical context seems a little off here; Livingstone's findings of 2017 are cited, but somehow Hubel and Wiesel, who in 1965 (J Neurophysiol) suggested that end-stopped, length tuned neurons could be useful for curvature detection, is omitted. There was a nice explicit model of how that could happen by Dobbins et al. in 1987 (Nature). But the manuscript makes no reference to any size tuning or surround suppression, and how that, in combination with classic orientation tuning, could create a cell that responds vigorously to curvature. This is even a greater concern given the well-established prevalence of size tuning, both electrophysiologically (1987) and with optical imaging (1997), in area V4. This has a huge impact on both novelty and interpretability; if you have a region of surround suppression (which we know exist) and it overlaps with an orientation region, there's a "curvature" region in that it will respond to a curve better than extended straight line or grating. But I would argue it's not a "curvature" region, since a short bar or grating without any curvature is actually the optimal stimulus. The absence of any reports of eccentricity or size tuning measurements does not install confidence that this is not what is responsible for these observations.

End-stopping and surround suppression are not sufficient to explain all the curvature responses. We appreciate the reviewer for raising the important question of whether size tuning may contribute to curvature preference response. As previous studies suggested, size tuning or end stopping may play an important role in curvature detection (Hubel and Wiesel, 1965; Dobbins et al., 1987) and is not independent from curvature preference features (Ponce et al., 2017). However other studies have suggested neurons with complex receptive field structures may also account for curvature preferences (Nandy et al., 2013, 2016). For this reason, while end-stopping likely contributes to curvature response, there are also other relevant parameters contributing to curvature response. (1) Surround suppressed and curvature domains are not the same: Similar to previous studies (Ghose and Ts'o, 1997), we found there were size sensitive regions in V4 (Case 2, Figure 2—figure supplement 13A). However, while there was some overlap, the size sensitive regions were on the whole spatially distinct from the curvature domains (compare Figure 2—figure supplement 13A and B, overlay in Figure 2—figure supplement 13C). (2) Weak response to small stimuli. Responses of curvature domains to small stimuli were weak in comparison to response to curvature stimuli (Figure 2—figure supplement 13D), suggesting weak end-stopping. Therefore curvature response cannot be fully explained by strong responses to end-stopping. (3) In addition, if end-stopping was the only factor that matters, curvatures with opposite orientations would densely overlap, which is not the case (Figure 6). Thus curvature preference response in V4 is not due to end-stopping alone. This is now described in Results under “Curvature domains are not end-stopping domains” and in Discussion under “Unlikely to be end-stopping domains”.

The curvature domain is based on curvature preference, not on surround suppression or end-stopping. The reviewer suggests that the imaged response to a curve may actually be a response to a short bar. We show in Figure 2—figure supplement 13 that this is not the case. The response to the small stimulus (“short bar”) is not strongly relative to the curvature response. While it is possible that some “curvature domains” may be termed “end-stopped domains” (i.e. regions of overlap in Figure 2—figure supplement 13C, 33.3% of orange pixels), at least some curvature domains exhibit poor end-stopping and are consistent with “curvature domain” (66.7% of orange pixels).

We think a clearer discussion of these issues would be important for the paper.

We have now completely rewritten this section and provided a more in depth discussion. It now includes a more in depth summary and the hypercolumn model (Figure 7E) has been moved to discussion. A new section entitled “Testing other possible interpretations” discusses possible contributions from component orientation, high spatial frequency, and end-stopping responses. Relationship of our findings with other studies is now expanded. In addition, we describe the significance of this work as providing a columnar hierarchy for shape representation.

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Revisions for this paper:

The manuscript is largely improved, and the authors have effectively addressed some of the major concerns raised in the initial submission. For instance, ehe authors addressed the concern that the curvature is fully accounted for by end stopping / surround suppression. They show that short straight lines poorly stimulate “curvature” regions and that “end stopped” patches are substantially distinct from “curvature” patches even though they overlap partially (Figure 2—figure supplement 13). Additionally, while showing that end stopping and curvature are not identical, the authors are careful in stating that end stopping could play an important role in curvature processing. But an important concern appears incompletely addressed. The evidence for the curvature specific domains is strong and novel, however the reviewers felt that the claims regarding the specialised architecture of these domains, namely the ordered, gradual representation of different degrees of curvature and the existence of pinwheels have not been fully supported by the presented results. We suggest either addressing the additional evidence suggested in revisions or tempering these claims and addressing these issues in a subsequent, linked publication.

1) One issue that is not satisfactorily addressed is “curvature pinwheels”. The authors make a point of proposing that pinwheel structure is fundamental to the organization of maps like orientation (in V1) and curvature (V4). While that proposal could arguably be made for V1 (Li, 2019) the authors' own evidence here suggests that pinwheels might be a red herring for curvature in V4.

a) The authors only show pinwheels in one data set (Figure 7B(right hand panel). And there too only the single pinwheel in the upper curvature patch looks reasonable. In the lower curvature patch the upper pinwheel lies very close to the edge of the curvature region making the classification problematic given the ~250-micron FWHM spatial resolution of intrinsic signal optical imaging (Polimeni, PNAS 2005).

We agree that the data regarding potential pinwheels is not sufficient and have now removed any mention of pinwheels from the paper.

b) More importantly, the most striking characteristic in the “curvature orientation” maps appears to be the high degree of overlap between regions of opposite curvature direction (Figure 6D). The net “curvature orientation” vector in these regions of overlap are presumably close to zero vector length. The maps (e.g. Figure 7B) with saturated colors are thus likely misleading as the real pattern likely involves not point singularities of “curvature orientation” (“pinwheels”) but large overlap patches of poor net “curvature orientation” with short “curvature orientation”-specific extensions outwards. Notably the authors did not show “curvature-orientation” contour maps, quantify the local orientation gradient and test it for significance to support their proposal of pinwheels. It is likely that the signal to noise in such maps would be very low.

Yes, we agree with the reviewer’s assessment. We removed Figure 7A-D and revised the summary figure in Figure 7E.

c) In the Discussion, it may be worth considering if the pattern of large patches of overlap with orientation-specific extensions may be the dominant topological motif, rather than trying to squeeze the map into a straitjacket of pinwheels.

Yes, we agree. We have simplified the model (Figure 7E) to reflect the data that we present. These include basic curvature domains, the presence of curvature degree progressions, and potential highly overlapped complex curvature domains.

2) With respect to the gradual shift in curvature maps, the response to the point of of pixel-wise map correlation was not adequate: the authors' response states that it merely establishes a curvature map, but any superposition of single condition domains, or differential analysis between different curvatures, establishes this. The manuscript says something more: there is a shifting map, implying some sort of continuity of representation. As stated in the previous comments, this strikes us as poor logic: if we take a low frequency, chromatic, linear grating, at one extreme, and a high frequency achromatic curve at another, the two activation maps will look different, and if I do a pixel wise morphing between those two stimuli that doesn't know anything about high level attributes like color, curvature, and SF, I necessarily have to create activation maps that look intermediate. It may be that subtle non-continuities, like fractures and pinwheels, in functional maps will make the change in correlations non-linear, but I'm not even sure of that. In other words, this analysis has to succeed, and is therefore, at least as it is presented, uninformative. If the authors are interested in the regularity or periodicity of representations, that a correlation analysis of cortical distance vs representational distance, seems more appropriate. For example, the spatial correlation of curvature preference on a pixel-wise basis, or between "centers" of domains. But, on the basis of what is presented, we are not sure how that is actually going turn out. For example, Figure 7 seems to show that 3/4 of the center of masses of "up" and "down" curvatures are basically on top of one another.

We agree with the reviewer that a simple correlation analysis as presented in not sufficient to infer a systematic progression of shifting curvature domains. We have now added, as the reviewer suggested, analyses of the distances in Figure 7 (Case 1) and in Figure 7—figure supplement 1 (Cases 2 and 3). For each case, we focused on the regions that corresponded to the central ~1 deg of the stimulus; we analyzed all the straight domains and the corresponding curvature domains (with same orientation). Starting from a straight orientation domain, we selected the nearest domain of each adjacent condition (shown in Figure 7B) and measured the distances between center of mass of each domain and that of the straight domain (leftmost panel in Figure 7C). These values are plotted in Figure 7D. This illustrates that the curvature preference domains generally progress from low to high curvature with distance across the cortex. It is informative that the progression is not necessarily linear, a point we bring up in Discussion. We also do not suggest that this indicates any continuity of representation, merely that there is a generally increasing distance with curvature change. We thank the reviewer for suggesting this analysis. We also add in the legends of Figure 3 that the fitted lines are not meant to indicate linear fits; rather, they help the reader see the trends of the different groups of points. Analyses of the two other cases are shown in Figure 7—figure supplement 1.

In addition, we also observed examples of a clear separation between straight response and curvature response (Figure 7—figure supplement 1C) and domains of broad curvature degree preference (Figure 7—figure supplement 1G). Thus, not all curvature domains are organized in progressions.

In total across the 3 cases, on average, there is a general overall tendency for straight-to-curved domains to exhibit spatial shifts. However, there are some domains with broad curvature preference. This suggests the presence of a diversity in organization, indicating that curvature representation in V4 is complex.

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Transparent reporting form

elife-57261-transrepform.docx^{(59.7KB, docx)}

Data Availability Statement

All data generated or analysed during this study are included in the manuscript and supporting files.

[bib1] Anzai A, Peng X, Van Essen DC. Neurons in monkey visual area V2 encode combinations of orientations. Nature Neuroscience. 2007;10:1313–1321. doi: 10.1038/nn1975. [DOI] [PubMed] [Google Scholar]

[bib2] Bao P, She L, McGill M, Tsao DY. A map of object space in primate inferotemporal cortex. Nature. 2020;583:103–108. doi: 10.1038/s41586-020-2350-5. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib3] Blasdel GG, Salama G. Orientation selectivity, preference, and continuity in monkey striate cortex. Nature. 1986;321:579–585. doi: 10.1523/JNEUROSCI.12-08-03139.1992. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib4] Bosking WH, Zhang Y, Schofield B, Fitzpatrick D. Orientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. The Journal of Neuroscience. 1997;17:2112–2127. doi: 10.1523/JNEUROSCI.17-06-02112.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib5] Brincat SL, Connor CE. Underlying principles of visual shape selectivity in posterior inferotemporal cortex. Nature Neuroscience. 2004;7:880–886. doi: 10.1038/nn1278. [DOI] [PubMed] [Google Scholar]

[bib6] Chen M, Yan Y, Gong X, Gilbert CD, Liang H, Li W. Incremental integration of global contours through interplay between visual cortical Areas. Neuron. 2014;82:682–694. doi: 10.1016/j.neuron.2014.03.023. [DOI] [PubMed] [Google Scholar]

[bib7] Chen M, Li P, Zhu S, Han C, Xu H, Fang Y, Hu J, Roe AW, Lu HD. An orientation map for motion boundaries in macaque V2. Cerebral Cortex. 2016;26:279–287. doi: 10.1093/cercor/bhu235. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib8] Dobbins A, Zucker SW, Cynader MS. Endstopped neurons in the visual cortex as a substrate for calculating curvature. Nature. 1987;329:438–441. doi: 10.1038/329438a0. [DOI] [PubMed] [Google Scholar]

[bib9] Fang Y, Chen M, Xu H, Li P, Han C, Hu J, Zhu S, Ma H, Lu HD. An orientation map for Disparity-Defined edges in area V4. Cerebral Cortex. 2019;29:666–679. doi: 10.1093/cercor/bhx348. [DOI] [PubMed] [Google Scholar]

[bib10] Ghose GM, Ts'o DY. Form processing modules in primate area V4. Journal of Neurophysiology. 1997;77:2191–2196. doi: 10.1152/jn.1997.77.4.2191. [DOI] [PubMed] [Google Scholar]

[bib11] Ghose GM, Ts'o DY. Integration of color, orientation, and size functional domains in the ventral pathway. Neurophotonics. 2017;4:031216. doi: 10.1117/1.NPh.4.3.031216. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib12] Grinvald A, Lieke E, Frostig RD, Gilbert CD, Wiesel TN. Functional architecture of cortex revealed by optical imaging of intrinsic signals. Nature. 1986;324:361–364. doi: 10.1038/324361a0. [DOI] [PubMed] [Google Scholar]

[bib13] Hegdé J, Van Essen DC. Strategies of shape representation in macaque visual area V2. Visual Neuroscience. 2003;20:313–328. doi: 10.1017/S0952523803203102. [DOI] [PubMed] [Google Scholar]

[bib14] Hegdé J, Van Essen DC. A comparative study of shape representation in macaque visual Areas v2 and v4. Cerebral Cortex. 2007;17:1100–1116. doi: 10.1093/cercor/bhl020. [DOI] [PubMed] [Google Scholar]

[bib15] Hubel DH, Livingstone MS. Segregation of form, color, and Stereopsis in primate area 18. The Journal of Neuroscience. 1987;7:3378–3415. doi: 10.1523/JNEUROSCI.07-11-03378.1987. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib16] Hubel DH, Wiesel TN. Receptive fields and functional architecture in two nonstriate visual Areas (18 and 19) OF the cat. Journal of Neurophysiology. 1965;28:229–289. doi: 10.1152/jn.1965.28.2.229. [DOI] [PubMed] [Google Scholar]

[bib17] Hubel DH, Wiesel TN. Receptive fields and functional architecture of monkey striate cortex. The Journal of Physiology. 1968;195:215–243. doi: 10.1113/jphysiol.1968.sp008455. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib18] Ito M, Komatsu H. Representation of angles embedded within contour stimuli in area V2 of macaque monkeys. Journal of Neuroscience. 2004;24:3313–3324. doi: 10.1523/JNEUROSCI.4364-03.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib19] Jiang R, i M L, Tang S. Discrete neural clusters encode orientation, curvature and 761 corners in macaque V4. bioRxiv. 2019 doi: 10.1101/808907. [DOI]

[bib20] Li P, Zhu S, Chen M, Han C, Xu H, Hu J, Fang Y, Lu HD. A motion direction preference map in monkey V4. Neuron. 2013;78:376–388. doi: 10.1016/j.neuron.2013.02.024. [DOI] [PubMed] [Google Scholar]

[bib21] Lu Y, Yin J, Chen Z, Gong H, Liu Y, Qian L, Li X, Liu R, Andolina IM, Wang W. Revealing detail along the visual hierarchy: neural clustering preserves acuity from V1 to V4. Neuron. 2018;98:417–428. doi: 10.1016/j.neuron.2018.03.009. [DOI] [PubMed] [Google Scholar]

[bib22] Murphy TM, Finkel LH. Shape representation by a network of V4-like cells. Neural Networks. 2007;20:851–867. doi: 10.1016/j.neunet.2007.06.004. [DOI] [PubMed] [Google Scholar]

[bib23] Nandy AS, Sharpee TO, Reynolds JH, Mitchell JF. The fine structure of shape tuning in area V4. Neuron. 2013;78:1102–1115. doi: 10.1016/j.neuron.2013.04.016. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib24] Nandy AS, Mitchell JF, Jadi MP, Reynolds JH. Neurons in macaque area V4 are tuned for complex Spatio-Temporal patterns. Neuron. 2016;91:920–930. doi: 10.1016/j.neuron.2016.07.026. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib25] Op de Beeck HP, Deutsch JA, Vanduffel W, Kanwisher NG, DiCarlo JJ. A stable topography of selectivity for unfamiliar shape classes in monkey inferior temporal cortex. Cerebral Cortex. 2008;18:1676–1694. doi: 10.1093/cercor/bhm196. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib26] Pasupathy A, Connor CE. Shape representation in area V4: position-specific tuning for boundary conformation. Journal of Neurophysiology. 2001;86:2505–2519. doi: 10.1152/jn.2001.86.5.2505. [DOI] [PubMed] [Google Scholar]

[bib27] Pasupathy A, Connor CE. Population coding of shape in area V4. Nature Neuroscience. 2002;5:1332–1338. doi: 10.1038/972. [DOI] [PubMed] [Google Scholar]

[bib28] Ponce CR, Hartmann TS, Livingstone MS. End-Stopping predicts curvature tuning along the ventral stream. The Journal of Neuroscience. 2017;37:648–659. doi: 10.1523/JNEUROSCI.2507-16.2016. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib29] Popivanov ID, Jastorff J, Vanduffel W, Vogels R. Tolerance of macaque middle STS body patch neurons to shape-preserving stimulus transformations. Journal of Cognitive Neuroscience. 2015;27:1001–1016. doi: 10.1162/jocn_a_00762. [DOI] [PubMed] [Google Scholar]

[bib30] Pospisil DA, Pasupathy A, Bair W. 'Artiphysiology' reveals V4-like shape tuning in a deep network trained for image classification. eLife. 2018;7:e38242. doi: 10.7554/eLife.38242. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib31] Ramsden BM, Hung CP, Roe AW. Real and illusory contour processing in area V1 of the primate: a cortical balancing act. Cerebral Cortex. 2001;11:648–665. doi: 10.1093/cercor/11.7.648. [DOI] [PubMed] [Google Scholar]

[bib32] Sceniak MP, Hawken MJ, Shapley R. Visual spatial characterization of macaque V1 neurons. Journal of Neurophysiology. 2001;85:1873–1887. doi: 10.1152/jn.2001.85.5.1873. [DOI] [PubMed] [Google Scholar]

[bib33] Tang R, Song Q, Li Y, Zhang R, Cai X, Lu HD. Curvature-processing domains in primate V4. eLife. 2020;9:e57502. doi: 10.7554/eLife.57502. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib34] Tanigawa H, Lu HD, Roe AW. Functional organization for color and orientation in macaque V4. Nature Neuroscience. 2010;13:1542–1548. doi: 10.1038/nn.2676. [DOI] [PMC free article] [PubMed] [Google Scholar]

[bib35] Wei H, Dong Z, Wang L. V4 shape features for contour representation and object detection. Neural Networks. 2018;97:46–61. doi: 10.1016/j.neunet.2017.09.010. [DOI] [PubMed] [Google Scholar]

[bib36] Yue X, Pourladian IS, Tootell RB, Ungerleider LG. Curvature-processing network in macaque visual cortex Curvature-processing network in macaque visual cortex. PNAS. 2014;111:E3467–E3475. doi: 10.1073/pnas.1412616111. [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Curvature domains in V4 of macaque monkey

Jia Ming Hu

Xue Mei Song

Qiannan Wang

Anna Wang Roe

Roles

Abstract

Introduction

Results

General approach

Curvature domains exist

Figure 1. Curvature grating stimuli.

Table 1. Case list.

Figure 2. Curvature domains exist and are distinct.

Figure 2—figure supplement 1. Large field of view of functional maps in Cases 1–3.

Figure 2—figure supplement 2. V1/V2 and V4 represent different retinotopic locations in the same field of view.

Figure 2—figure supplement 3. Relationship between curvature domain and low SF domain.

Figure 2—figure supplement 4. Low and high curvature maps are highly overlapped (two Cases across two sessions).

Figure 2—figure supplement 5. Stability of curvature response across different days.

Figure 2—figure supplement 6. Imaging of curvature responses and scrambled map in the same cortical region (from Case 3).

Figure 2—figure supplement 7. Supporting graphs for Figure 2L.

Figure 2—figure supplement 8. Straight orientation domains and high spatial frequency domains exhibit weak responses to single curved lines.

Figure 2—figure supplement 9. Neurons in curvature domains prefer curvature.

Figure 2—figure supplement 10. Curvature domains were not observed in V1.

Figure 2—figure supplement 11. Very little V2 is available on surface.

Figure 2—figure supplement 12. Curvature domains are not pinwheel centers.

Figure 2—figure supplement 13. Curvature domains can not be fully explained by end-stopping.

Mapping types of curvature

Low to high curvature maps

Figure 3. Systematic maps of curvature degree.

Figure 3—figure supplement 1. Correlation values between pairs of straight orientation maps obtained in three different runs (from Case 1).

Figure 3—figure supplement 2. Systematic maps of curvature degree and curvature orientation in other cases.

Curvature orientation maps

Figure 4. Systematic maps of curvature orientation.

Mapping central part of curvature grating

Organizations within curvature domains

Figure 5. Subregions of high curvature preference.

Figure 6. Organization of curvature orientation representation in V4 curvature domains (A-C: Case 2, D-F: Case 3).

Organization of curvature representation

Figure 7. Functional organization of curvature in V4.

Figure 7—figure supplement 1. Diversity of curvature domain organization (A-D: Case 2, E-H: Case 3).

Figure 7—figure supplement 2. Curvature domain response is not due to component orientation response.

Curvature domains are not orientation pinwheel centers or sums of orientation components

Curvature domains are not end-stopping domains

Discussion

Summary

Hypercolumn model

Testing other possible interpretations

Unlikely to be primarily component orientation responses

Unlikely to be high spatial frequency domains

Unlikely to be end-stopping domains

Curvature domains are signature modules of V4

Significance of modules—cortical modules represent key transformational stages in the visual hierarchy

Materials and methods

Animal preparation

Visual stimuli for optical imaging

Retinotopic mapping

Optical imaging

Map-guided cell recording

Data analysis

Functional maps

Locating positions of functional domains

Timecourse

Similarity

Statistical analysis

Acknowledgements

Funding Statement

Contributor Information

Funding Information

Additional information

Competing interests

Author contributions

Ethics

Additional files

Data availability

References

Decision letter

Roles

Author response