Disambiguating Necker cube rotation using a location cue: What types of spatial location signal can the visual system learn?

Abstract

The direction of rotation of a wire-frame (Necker) cube, which is perceptually bistable, can be trained to depend on stimulus location (Q. Haijiang, J. A. Saunders, R. W. Stone, & B. T. Backus, 2006). However, it is not known which aspects of “location” are important to this learning. We therefore explored “location” in a series of experiments that separately assessed testing venue, location relative to the observer, and location in the retinal image as types of location signal that could potentially be recruited by the visual system. Subjects were trained using wire-frame cubes with rotation direction disambiguated by depth cues. Training cubes were presented at two locations, rotating in opposite directions. On interleaved test trials, ambiguous monocular cubes were presented at the same two locations. The extent to which test cubes were perceived to rotate according to the trained location-rotation contingency was our measure of location-cue recruitment. We found that only retinal position was recruited as a cue for apparent rotation direction. Furthermore, the learned retinal location cue was robust to ocular transfer. Our findings are consistent with a relatively low-level site of learning, such as MT.

Introduction

The visual system routinely and automatically makes use of “cues” to construct appearance. For example, depth cues inform size percepts, and region luminance contrast informs reflectance percepts. The validity of inferring appearance from such cues is due to their basis in longstanding statistics of the environment, hence their use is likely to optimize our ability to form veridical percepts (Brunswick, 1956; Geisler & Kersten, 2002; Helmholtz, 1910/1925).

It was recently demonstrated, using an ambiguous Necker cube stimulus, that the visual system can also recruit new cues to appearance (Haijiang, Saunders, Stone, & Backus, 2006). Ambiguous stimuli have long been of interest to vision scientists, as they evoke different percepts not according to any change in the proximal stimulus, but instead due to changes in neural activity inherent to the observer (for examples, see Attneave, 1971). In particular, bistable, or “reversible”, ambiguous stimuli have two distinct plausible interpretations; at stimulus onset, either perceptual outcome may result. Such stimuli are valuable to the exploration of the neural basis of perception inasmuch as the perceptual state can be well controlled as an independent variable (through the use of forcing cues) or well measured as an independent variable (through subjective report). The rotating Necker cube is one such stimulus, with the direction of rotation being ambiguous in the proximal stimulus but clearly perceived as one of two dichotomous interpretations following stimulus onset.

Using an associative learning paradigm, the perceived rotation directions of Necker cube stimuli were trained to be contingent on cube location (Backus, 2009; Backus & Haijiang, 2007; Haijiang et al., 2006). These studies demonstrated robust recruitment of location as a cue to rotation direction of the Necker cube. Here, we are concerned with the nature of the location cue—what aspect of “location” is learned by the visual system? Further, it is not known whether learning generalizes to new environments (i.e., new experiment venues). In the following experiments, we address the question of what types of location signal are recruited as a cue.

Experimental approach

As in previous studies (Backus, 2009; Backus & Haijiang, 2007; Haijiang et al., 2006), subjects viewed brief presentations of a rotating wire-frame cube, with training and test trials interleaved. On training trials, the stimulus contained cues to depth, hence rotation direction was disambiguated. Training stimuli appeared at one of two locations, with opposite rotations at each. Test stimuli were presented at the same two locations as training stimuli in any given experiment. Test stimuli contained no cues to depth, hence, the direction of cube rotation was ambiguous, and the perceived direction at onset was a dichotomous decision.

On each trial, we obtained a button-press response from the observer that indicated the perceived direction of rotation on that trial. Whereas performance on unambiguous training trials could be quantified in terms of “percent correct,” thereby giving us an indication of observers’ comprehension and engagement in the task, ambiguous test trials had no correct answer. Instead, the perceived direction of rotation of the cube on test trials enabled us to assess the extent of cue recruitment: If training trials caused no learning, then there would be no average difference in the proportion of trials seen as rotating clockwise at the two oppositely trained locations. Alternatively, if training caused substantial learning, then test cubes presented at one of the two locations would be perceived as rotating clockwise and test cubes presented at the other location would be perceived as rotating counterclockwise, according to the trained location–rotation contingency.

We wished to verify that any difference in perceptual outcome of test trials between the two locations represented long-term learning of the location cue rather than being attributable simply to short-term “priming” effects (e.g., Long & Moran, 2007). To this end, subjects underwent a second testing session on the following day in which the location–rotation contingency of the training cues was reversed. If cue recruitment from Day 1 was a short-term phenomenon, then the outcome of Day 2 test trials would depend only on Day 2 training, and this dependence would be of the same magnitude as that shown for test trials on Day 1. If, however, cue recruitment on Day 1 was long-lasting, then Day 2 training would be less effective and test trials on Day 2 would show a smaller effect of that day’s training.

The above paradigm was applied with respect to three outstanding questions regarding the nature of the recruited location cue. Experiment 1 assessed whether location cues learned in one testing venue, or “environment”, transferred to a different testing venue. Next, we revisited the question of whether the “world location” of the cubes, i.e., their location within the testing room, could be trained as a cue (Experiment 2), and additionally whether this factor contributed to the previously demonstrated location cue (Experiment 3). Finally, Experiment 4 investigated whether the location cue had an eye-specific component, by training a location–rotation contingency in one eye only on Day 1, then assessing longevity of the recruited location cue in either the trained or the untrained eye on Day 2.

To anticipate our findings, our results are consistent with recruitment of an entirely “retinal location” cue for short training regimes as used here. These findings constrain the locus of the training effect to cortical areas that contain neurons selective for retinal location. Such areas are typically selective for low-level stimulus properties. Hence one interpretation of our results is that although the rotating cube is perceived as a coherent unit, training is in fact of the local cube components, perhaps causing rightward- and leftward-moving elements to be registered as near or far, respectively.

General methods

Hardware and software

Experiments were programmed in Python using the Vizard platform version 3.11 (WorldViz, Santa Barbara, CA) on a Dell Precision T3400 computer. Stimuli were rear-projected onto a screen, using either a Christie Mirage S+ 4K projector (Room 1) or an Infocus LP350 projector (Room 2, used in Experiment 1 only).

Cube stimuli

Simulated rotating cube stimuli were light against a dark background. Subjects were initially seated so as to be comfortable at a viewing distance of 1.0 m from the display screen, which was also the distance to the center of the stimulus as specified by vergence and accommodation. Subjects’ heads were not restrained. Cube edges were of 20.0-cm length, hence subtended approximately 11.5 degrees of visual angle when in the frontoparallel plane. The “wire-frame” edges of the cube were solid rectangular parallelepipeds with width and breadth of 0.3 cm, except in Experiment 4 where edge thickness was increased to 2.0 cm. Each transparent face of the cube contained 25 randomly placed dots, which stabilized appearance on ambiguous trials as a single rigid rotating body. Cubes rotated about a vertical axis at a rate of 45 degrees s⁻¹, from a starting orientation at stimulus onset such that their front and back edges were vertical and coincident at the center of the image (45 degrees of yaw). The roll and pitch angles, which determine whether the cube appeared to be viewed from above or below at stimulus onset, were either both +25 or both –25 degrees. (Note that through the course of an entire rotation, both above and below viewpoints would be seen.) This parameter was balanced across both training and test trials because “above” and “below” configurations are associated with different motions of the cube edges at stimulus onset; such a difference between stimuli presented the “top” and “bottom” locations could otherwise be a confounding cue. All stimuli were viewed through red–green glasses, in order to present disparity information and to control the eye of presentation.

Five established cues to depth were used, in different combinations across the experiments, to disambiguate the direction of rotation of the training cubes (Figure 1): Disparity cues to depth were created by the use of red– green anaglyphic images. Cubes were presented with geometrically correct disparity at the simulated distance of 1.0 m. Cubes had sides of length 20.0 cm, therefore the maximum disparity during rotation between the nearest and farthest points of the cube was 1.0 degree of visual angle. An occlusion cue to depth consisted of a central column around which the cube rotated. The column was a 2-dimensional vertical strip of 4.0-cm width, extending from the top to the bottom of the screen area, presented stereoscopically at the screen distance. Far portions of the cube were occluded as they moved around the back of the column, whereas closer portions of the cube were visible in front of the column. Experiment 4, where cube frame edges had discernable 3D structure due to their increased width and breadth, used a directional light source to add depth-from-shading and internal occlusion cues to the rotating cube. The impact of the light source on different faces of the cube edges was further manipulated by use of the Vizard “fog” function, which simulates the effect of haze or “aerial perspective” whereby contrast decreases with distance. In our stimuli, fog reduced the contrast linearly from full contrast at the closest edges of the cube to zero contrast at the most distant edges.

Figure 1.

Screen shots (cropped) showing the various depth cues used. (a) Training cube disambiguated by disparity and occlusion cues. (b) Corresponding test cube, presented monocularly. (c) Training cube disambiguated by occlusion, depth-from-shading, internal occlusion, and haze. (d) corresponding test cube. Cubes a and b are representative of those used in Experiments 1, 2, and 3. Cubes c and d are representative of those used in Experiment 4. Screen shots also show the location of the fixation square and path of the comparison dot for a “top” cube in Experiments 1 and 4. The background was black in the experiments.

Test cubes were presented monocularly to observers’ right eyes (i.e., only the green image), hence had no disparity information, and also contained no other monocular cues to depth. All training and test cubes were presented using orthographic projection so that perspective cues did not indicate front and back of the cube in the test case. Some observers reported perceiving associated distortions of the training cube, as if edges varied in length during rotation, but none failed to perform correctly in training trials due to this perceptual effect.

Trial sequence

In all experiments, a 2.0 cm × 2.0 cm square-outline fixation marker was presented binocularly, at the screen depth. The fixation marker remained on the screen at all times. Subjects were instructed to achieve fixation of the marker prior to initiating each trial with a key press, and to then maintain fixation rather than look directly at the rotating cube stimulus. On initiation of each trial, a rotating cube appeared adjacent to the fixation marker (with the spatial relationship between the marker and the cube, and the location of the marker on the screen, varying by experiment as detailed below). Simultaneously, a comparison dot repeated cycles of horizontal motion through the fixation marker. The subjects’ task was to indicate whether the direction of motion of the comparison dot (leftward or rightward) was the same as the direction of motion of the front of the cube or the back of the cube (key press “2” and “8,” respectively). These keys are spatially correspondent with “front” and “back” on a horizontally held numeric keypad. Subjects were instructed that we were interested in the accuracy, not the speed, of their responses; on debriefing, subjects reported perceptual reversals on less than 1% of ambiguous trials overall. Dot speed was 15.7 cm s⁻¹ and dot direction was randomized, with equal probability for leftward and rightward motions. The dot was presented at fixation depth on training trials and monocularly on test trials. The cube and comparison dot remained on the screen for a minimum of 1.5 s and a maximum of 6.0 s; the subject’s response terminated the presentation at any time after 1.5 s.

Due to random assignment of dot direction, the measure of interest, perceived direction of rotation, was not correlated with the key-press response. None of the subjects who took part in the following experiments reported being aware that cube rotation was dependent on cube location even though location correlated perfectly with rotation direction on training trials. Instead, subjects typically reported upon debriefing that they had noticed strings of “front” (“2”) or “back” (“8”) responses, the occurrence of which is entirely expected in random binary strings.

Subjects

Subjects were adult members of the public who were recruited from the local area (New York City) via Craig’s List and were paid for their time. Subjects’ vision was normal or corrected-to-normal with non-bifocal lenses. Stereoscopic vision was assessed using the TNO Stereo-acuity test; subjects were required to have a minimum stereoacuity of 240 s of arc. However, stereoacuity in static images is not always an indicator of stereoacuity in dynamic images such as the rotating cube stimulus used here (Rouse, Tittle, & Braunstein, 1989). Accordingly, our critical measure of subjects’ suitability for the experiment, in terms of both stereoacuity and task comprehension, was their performance on training trials. In the course of the following four experiments, a total of five subjects, who met other criteria but did not reach performance levels on training trials of 95% or over, were excluded from the study. A total of 56 subjects completed experiments in the study.

Experiment 1—New environments

This experiment was designed to assess whether the location cue learned in one venue generalizes to other venues. Although this question is rarely asked within the context of visual perception, other forms of learning show only partial transfer from the training environment to new environments (e.g., Siegel, 1972). Hence, it is important to determine whether the visual system assumes (implicitly, as revealed by utilization of the new cue) that a newly learned cue will have ecological validity in a location other than where it was learned. Additionally, this experiment allowed us to simultaneously replicate the previously documented effect (Backus, 2009; Backus & Haijiang, 2007; Haijiang et al., 2006) for comparison with later experiments.

The venues, Room 1 and Room 2 (Supplementary Figures 1 and 2), each had its own in situ projection system as described in the General methods section. A fixation marker was presented centrally on the projection screen, and cubes were centered 12 deg above or below the fixation marker, termed “Top” and “Bottom,” respectively (Figure 2a). Subjects were trained under either “top-clockwise (CW), bottom-counterclockwise (CCW)” contingency, or vice versa. The rotation direction of “training” stimuli was specified by disparity and occlusion cues to depth. “Test” stimuli contained no cues to depth (other than the new cue of location), as detailed in the General methods section, and hence their direction of rotation was ambiguous.

Figure 2.

Spatial arrangement of fixation marker and cube locations. Gray horizontal line indicates vertical midpoint of display. All stimuli were presented centrally but are shown here side by side for comparison: (a) Experiments 1 and 4; cubes appeared either above or below a central fixation marker. (b) Experiment 2a; cubes appeared either above or below the center of the display, together with a fixation marker that was always above the presented cube. (c) Experiment 2b; cubes appeared above or below the center of the display (note increase in cube spatial separation), together with a fixation marker at one of 16 locations circularly arranged around each of the two possible cube locations. (d) Experiment 3; cubes appeared directly above or below the fixation marker, which was at one of 16 possible cube locations circularly arranged around the center of the display.

On Day 1, all subjects completed one session consisting of 240 training trials and 240 test trials. Training and test trials were randomly interleaved, within balanced sets of 8 trials that contained all combinations of training/test trial type, top/bottom cube location, and above/below cube viewpoint (as detailed in the General methods section). The first 8 trials were constrained to present the 8 possible trials in a train–test sequence, alternating between locations and cube viewpoints. The order of presentation of these first 8 trials was counterbalanced across subjects. On Day 2, trials were organized as before, but subjects viewed the opposite location–rotation contingency to that viewed on Day 1.

A total of 16 subjects took part in Experiment 1. A crossover design was used: 8 subjects were tested in Room 1 on Day 1, and 8 were tested in Room 2. Half of each group switched rooms on Day 2, and half remained in the same room, giving 4 subjects in each of 4 groups.

Results

The percentage of test trials seen as CW at the top and bottom cube locations is presented in Figure 3. Each panel shows data for one group of 4 subjects. Within each panel, the two lines of a given color show top and bottom data, respectively, for one subject. If training on Day 1 was ineffective, then there would be no difference in the perceived rotation of test cubes at the two locations. However, for all four groups the top and bottom locations were perceived as rotating in opposite directions, showing that the top/bottom location cue was learned on Day 1. Similarly, if the location cue was not retained on Day 2 when subjects were trained with the opposite location– rotation contingency, then the data should show a symmetrical “X” pattern, with cue recruitment on Day 2 being exactly opposite to that on Day 1. Again, this is not what is seen: In all four groups, the location–rotation contingency trained on Day 1 exerted an influence on Day 2, despite the equal and opposite training received on Day 2.

Figure 3.

Percent of test trials seen as clockwise, for four groups of subjects, tested in different combinations of Room 1 and Room 2 on Days 1 and 2. Perceptual outcome at the top stimulus location is denoted with filled symbols, perceptual outcome at the bottom location with outline symbols. Colors represent subjects, with four subjects in each group.

In order to compare the efficacy of training at the two cube locations, the data plotted in Figure 3 as “percent seen as clockwise” were transformed to “percent seen as trained”, so that equivalent deviations of performance from chance, in the opposite directions trained at the two cube locations, would result in identical performance measures. These percentages were then transformed to z-scores, a measure of the likelihood of the observations given normally distributed noise in a decision variable, which allows independent effects to be modeled as additive (which is useful for analysis of variance). For the purpose of analysis, saturated values (0% or 100%) were replaced with the z-score representing one non-conforming value within the group of observations. In the present case, if a subject perceived 120 out of 120 test trials at the top cube location as rotating in the trained direction, a z-score of 2.394 was used in later analysis, which corresponds to 119 out of 120. Similarly, a perceptual outcome of 0 out of 120 was represented by a z-score of −2.394.

A 2-way ANOVA on the z-scores for Day 1, with cube location (top or bottom) as a within-subjects factor, and Day 1 venue (Room 1 or Room 2) as a between-subjects factor, found no significant difference in the extent of cue recruitment between either the two venues or the two cube locations, and no interaction between these two variables. A 3-way ANOVA on the z-scores for Day 2, adding Day 2 venue as a third between-subjects factor in addition to the previous factors, also found no significant main effects of cube location or venue on Day 1 or Day 2, and no interactions between them.

As there was no difference in the efficacy of training at the top and bottom cube locations, from this point onward we quantify cue recruitment by summing the two “seen as trained” z-scores, which gives a measure (−4.788 to 4.788) of the propensity to see the cube rotation differently at the top and bottom locations. This “difference of z-scores” (zDiff) is analogous to a d-prime measure in signal detection theory (Backus, 2009). A zDiff of zero means that the same proportion of trials was seen as clockwise at the two cube locations (no cue recruitment). A positive zDiff indicates cue recruitment as trained, whereas a negative zDiff indicates that perceived rotation was opposite to training. This new measure was used in all further testing for differences in perceived rotation.

The extent to which the location–rotation contingency trained on Day 1 was retained on Day 2 was assessed by comparing perceptual outcome on Day 2 test trials to the contingency trained on Day 1. Complete retention of the cue recruited on Day 1 would result in the zDiff score for the 2 days being the same. Conversely, if the cues trained on Day 1 no longer exerted any influence, the new location–rotation contingency trained on Day 2 would be recruited to the same extent as the opposite contingency was recruited on Day 1. Hence, Day 2 zDiff scores would be of equal magnitude but opposite sign to that measured on Day 1. Due to the already-demonstrated non-significant main effect of venue, we collapsed our data into two groups, room “same” and room “swap”, to provide a stronger test of the generality of cue recruitment from one venue to another (Figure 4).

Figure 4.

zDiff scores for two groups of 8 subjects on Day 1 and Day 2. “Same” subjects were tested in the same rooms on Days 1 and 2; “swap” subjects were tested in different rooms.

A 2-way ANOVA with testing session (Day 1 or Day 2) as a within-subjects factor and testing group (same or swap) as a between-subjects factor found no difference between the “same” and “swap” groups overall and confirmed a strong difference in the effect of training on Days 1 and 2. The influence of Day 1 training was greater on Day 1 than on Day 2, as reflected in significantly lower zDiff scores on Day 2 (Day 1 mean = 3.64, s.d. = 0.97; Day 2 mean = 1.27, s.d. = 1.79; F(1,14) = 32.45; p < 0.001). This could be expected, as subjects received reversed training on Day 2. However, the zDiff score on Day 2, despite being decreased, was still positive (t = 2.84, p = 0.013). This shows that perceptual experience on Day 2 agreed more with the original training from Day 1 than with the opposite training on Day 2. Thus, cue recruitment from Day 1 was retained into Day 2. Most importantly for our motivating question, there was no interaction between the day of testing and same/swap group. That is, we found no evidence that cue retention on Day 2 depended on whether the venue remained the same or changed.

Our final analysis looked at the time course of cue recruitment on Day 1, and the “unlearning” on Day 2. Trials were split into 6 “superblocks”, each containing 10 blocks of 8 balanced trials, giving a total of 40 test trials (interleaved with 40 training trials) in each superblock. zDiff scores for each observer were calculated as before. Note that z-scores are now constrained to a maximum and minimum of z(39/40) and z(1/40), respectively (±1.960), so the maximum zDiff score is lower (3.920) than when considering results over an entire testing session. Results are plotted in Figure 5.

Figure 5.

zDiff scores averaged over all 16 observers, over the course of Day 1 and Day 2 testing sessions. Error bars are standard errors of the mean across subjects, computed separately for each superblock.

Cue recruitment plateaued at close-to-ceiling levels very rapidly on Day 1, dropped before or during the first block of Day 2, and appeared to be slightly unlearned during the course of Day 2 in accord with the reverse contingency training on that day. While we can say with confidence that the level of cue recruitment at the start of Day 2 is due to long-term learning, and that the strength of this long-term learning can be assessed by resistance to reverse contingency training, our experiment was not designed to distinguish between two possible causes of time-dependent changes during the course of each session. The rapid acquisition of the retinal location cue at the beginning of Day 1 is predicted from short-term priming by immediately preceding training trials (Fuller, Backus, van Dam, & Ernst, 2009); however, the growing influence of long-term learning on perceptual outcome within Day 1 cannot be assessed. Similarly, some part of the unlearning that is evident at the start of Day 2 could be attributed to short-term priming by the reverse contingency training trials; gradual decrement of the learned long-term bias during Day 2 cannot be directly assessed within our methodology. Separation of these two influences would require presentation of probe test stimuli, followed by immediate termination of the experiment, as the probe stimuli themselves could affect both priming and learning.

Experiment 1—Discussion

The results replicate previous findings (Backus & Haijiang, 2007; Haijiang et al., 2006), showing that location can be trained as a cue to apparent rotation direction in ambiguous Necker cubes. Cue recruitment was rapid and was retained through a second day of training with opposite location– rotation contingency. Reliance on the cue was no greater in the local environment (testing room) where training occurred; instead use of the new cue generalized to a new environment.

The data from this experiment are not easily interpreted within an incremental learning paradigm because the change on Day 1 (utilization of the new cue) reached plateau levels very quickly. However, the data reflect a combined influence of both short-term and long-term changes in appearance, and it seems likely that the long-term influence grows over time. Whether that growth is incremental with each trial or abrupt after some number of trials (Gallistel, Fairhurst, & Balsam, 2004; Rescoria & Wagner, 1972) remains to be determined.

Experiment 2—Training of world location cue only

Next we turned to the issue of whether the recruited location cue is mediated by retinal location, world location (position on the projection screen), or perhaps both. The spatial arrangement of the fixation marker and cube locations used in Experiment 1 does not distinguish between the two, as the fixation marker was always presented centrally, and cubes were at fixed screen locations above or below the fixation marker. Hence, both retinal location and world location were potentially being trained. Backus and Haijiang (2007) also tested whether cue recruitment was mediated by retinal position or position in the environment. While cue recruitment occurred quickly for training with consistent retinal coordinates, they did not conclusively rule out a “world location” component to the recruited location cue. In Experiment 2, we trained subjects with consistent world location cues but inconsistent retinal location cues.

Experiment 2a

Cubes were presented at the same two locations as in Experiment 1, 12 degrees above or below the center of the screen. However, in Experiment 2 each cube was presented with its own fixation marker, 12 degrees above its center (Figure 2b). Hence “top” and “bottom” cubes, as defined by their positions on the screen, were presented at the same retinal location. This is the simplest possible manipulation to dissociate retinal and world locations of the cube stimuli and to therefore allow us to test whether world location can be recruited as a cue in its own right.

Trial structure was identical in all respects to Experiment 1, with a total of 480 trials presented in blocks of 8 randomly drawn trials, balanced for cube location (top or bottom), cube viewpoint (above or below), and trial type (train or test). As before, each trial commenced with presentation of a fixation marker, which could now appear at either of the two locations according to the randomized trial sequence. Subjects were instructed to achieve fixation prior to initiating stimulus presentation with a key press and to maintain fixation throughout stimulus presentation.

Results

The average zDiff score, for eight new subjects, is shown in Figure 6 together with performance in Experiment 1 and in later experiments for comparison. Retention of the world location cue on Day 2 was not assessed because it was not recruited on Day 1.

Figure 6.

zDiff scores averaged over subjects, for all experiments. Error bars are standard errors of the mean. Day 1 results are depicted in red, and Day 2 results are depicted in blue. Significant differences of mean values from zero are indicated.

Clearly, screen location was not effectively recruited as a cue. This was true not only for the aggregate mean but also for all subjects individually. In general, the zero-value zDiff scores resulted from subjects seeing the same rotation direction at both locations. In theory, a zDiff score of zero could result from any mix of clockwise and counterclockwise percepts, as long as the proportion of clockwise percepts was the same for the two cube locations. However, subjects either saw all cubes as clockwise or all cubes as counterclockwise, throughout the experiment, with few exceptions (Figure 7).

Figure 7.

Data from eight subjects, showing percent of cubes seen as rotating clockwise, averaged over both cube locations. Error bars for these binomial data are omitted for clarity of presentation.

Experiment 2b

The simplest possible manipulation to provide consistent ecological validity to screen location as a new cue, without retinal location or other stimulus attributes as a confounding cue, did not result in recruitment of the screen location cue in Experiment 2a. However, it was still possible that screen location might be recruited if the number of training trials were greater, if the difference in screen location of the two cubes were greater, or if retinal location were randomized rather than being fixed (which might, for example, prevent strong biases from developing at a single retinal location as was observed in Experiment 2a).

In Experiment 2b, we increased the amount of training, increased cube separation, and randomized retinal location. The trial sequence began with a block of 96 training trials only. After the initial training block, training and test trials were interleaved as before, but at a higher training ratio of 2:1. In addition, the spatial difference between the two cube locations was increased, with the cube centers now 18 degrees instead of 12 degrees above or below the center of the screen. Cubes were presented with one of 16 fixation markers, unique to each cube, equally spaced at a radius of 12 degrees from the center of the cube (Figure 2c). As in Experiment 2a, the fixation marker for the next trial appeared once a response had been collected from the previous trial; subjects indicated fixation and readiness for the next trial with a key press. As the cube was sometimes horizontally offset from the fixation marker, it was necessary to shorten the path of the comparison dot through the marker to 5 degrees, so it did not impinge on the cube itself.

Trials were drawn in random order from balanced blocks of 32 (initial training only) or 96 (training and test interleaved) possible trials, with each block containing all permutations of cube location, location of the fixation marker, and training or test trial type (at the 2:1 ratio). Cube viewpoint was fixed for each fixation marker location, and alternated between “above” and “below” for spatially adjacent markers.

Results

The average zDiff score, for eight new subjects, is shown in Figure 6. The Day 1 zDiff score was significantly different from zero (mean = 0.20, one-sample t-test t = 2.41, p = 0.046). However, it was very small in magnitude compared to the previously observed effect (Experiment 1), and as such it provides, at best, weak support for recruitment of the screen location cue. Hence, cue retention on Day 2 was not assessed.

A very small zDiff indicates that subjects perceived the same proportion of trials as clockwise at both cube locations: Again, we analyzed the time course of subjects’ perceived rotation. Responses were split into 4 super-blocks of 32 test trials; this was the smallest unit containing a fully balanced trial set. This time, subjects’ perceived clockwise rotation spanned the full range from 0 to 100% (Figure 8). In addition, some subjects showed variability across the session in the total percent of test stimuli seen as rotating clockwise. Despite the greater overall variability in perceived clockwise rotation across and within subjects (as compared to the two-fixation paradigm of Experiment 2a), this heterogeneity did not facilitate recruitment of the screen location cue.

Figure 8.

Data from eight subjects, showing percent of cubes seen as rotating clockwise, averaged over both cube locations.

Experiment 2—Discussion

The results of Experiment 2 stand in contrast to those of Experiment 1: The screen location cue was not recruited, but the retinal location cue had been strongly recruited, as evidenced by its persistence on a consecutive day of reverse contingency training. Clearly, if we had found recruitment of a non-retinal location cue in Experiment 2, we would have suspected that the location cue recruited in Experiment 1 had a non-retinal component. However, we could not have concluded this unequivocally, as it might have been that subjects simply maintained central fixation against our instructions and therefore, not surprisingly, recruited a retinal location cue. In any case, this is not what we found, and hence we are confident that the learning takes place in retinotopic coordinates.

While subjects reported that they did on occasion look directly at the rotating Necker cube, this apparently did not prevent learning in Experiment 1. It is possible that the retinal location of the stimulus at onset is sufficient to determine perceived rotation direction during brief presentations, even when central fixation wavers. Alternatively, learning could be robust to small shifts in eye position, or eye movements may simply have been sufficiently infrequent that they did not disrupt the strong recruitment of retinal location cues. Note that although eye posture and eye movements may be linked to reversals in prolonged presentations of static Necker cubes (Einhauser, Martin, & Konig, 2004; Toppino, 2003; van Dam & van Ee, 2006), eye movements toward the stimulus could not have driven the perceptual outcome at onset in Experiment 1 (or later Experiments 3 and 4) as observers were not cued as to the location of the impending stimulus. Additionally, eye pursuit movements during presentation of rotating Necker cubes have been shown to have no effect on perceptual outcome (Brouwer & van Ee, 2007).

Finally, in our experience, the perceived direction of rotation of the Necker cube at stimulus onset is robust to volitional control. Even if subjects had volitional control of their percept, it is unlikely that this affected our results, as subjects were always surprised on debriefing to discover that there had been a correlation between the stimulus location and rotation direction. It would be difficult to explain how subjects could volitionally cause percepts that conformed to a contingency they were unaware of. Hence we believe that our results represent a genuine alternation of perceptual appearance that is unconfounded by eye movements or eye position and is the result of processes that are specific to retinal location.

Experiment 3—Training of retinal location in the absence of world location cues

In our final attempt to observe recruitment of a world location cue, we adopted a reverse tactic: Instead of training valid screen locations, we trained valid retinal locations as in Experiment 1 but rendered invalid the previously present world location cue. Hence, if screen location had been recruited to any extent in Experiment 1, training without valid screen location cues would reduce the magnitude of the measured effect.

In order to present valid retinal location cues, cubes were always located directly above or below the fixation marker, as in Experiment 1. However, so that screen location was not correlated with the rotation percept as before, the fixation marker could appear at one of 16 locations on the screen (Figure 2d). The 16 locations were equally spaced at a radius of 12 degrees from the center of the screen. Due to having 16 possible fixation locations, trials were now drawn randomly from balanced blocks of 64 trials (16 fixation locations, 2 trial types, 2 cube viewpoints). However, as in Experiment 1, there was no initial training-only block of trials, and test and training stimuli were interleaved at a 1:1 ratio from the outset. The first 8 trials were constrained to a repeating train–test sequence similar to that used in Experiment 1; four fixation locations were drawn at random, and alternating cube locations (“top” and “bottom”, relative to fixation) and cube viewpoints (above and below) were presented.

To anticipate our results, the retinal location cue was effectively recruited on Day 1, so subjects were trained on the following day with the reverse location–rotation contingency. Eight new subjects completed 480 trials on both days.

Results

Average zDiff scores are shown in Figure 6. A one-way ANOVA showed that zDiff scores were lower but not significantly so on Day 2 compared to Day 1 (F(1,7) = 1.86, p = 0.215), and both were significantly greater than zero (one-sample t-test; Day 1 mean = 3.82, t = 9.01, p < 0.001; Day 2 mean = 3.05, t = 4.27, p = 0.004). Hence cue recruitment was largely retained on Day 2, and was not significantly unlearned during the reversed location– rotation contingency training.

Comparison of results from Experiment 1 (valid retinal and screen location cues) and Experiment 3 (valid retinal location cues only) showed that the two did not give significantly different results overall (F(1,22) = 3.17, p = 0.089), and that the main factor of testing day was significant (F(1,22) = 20.36, p < 0.001). However, the interaction of Experiment and Testing Day was also significant (F(1,22) = 5.28, p = 0.032), due to the lower zDiff for Experiment 1 on Day 2.

Experiment 3—Discussion

Removal of the valid screen location cue did not lessen the extent of recruitment of the location cue in Experiment 3 compared to Experiment 1, suggesting that the recruited cue in Experiment 1 was retinal location only. However, cue retention on Day 2 was greater in Experiment 3, hence we conjecture that removing the valid screen location cue in Experiment 3 actually caused stronger learning of the retinal location cue. Removal of the valid screen location cue might have caused an increase in the salience of the retinal location cue per se. Alternatively, as a result of the change in fixation marker location from trial to trial, there could have been greater fixation accuracy on the part of observers, or greater allocation of visual attention to the stimulus (Fuller & Backus, 2010). Any or all of the above might have occurred to increase the learning.

Experiment 4—Ocular transfer

Our final experiment considered whether the recruited location cue was robust to ocular transfer. Experiments 1–3 provided strong evidence that the location cue was retinotopic, suggesting that the learning of this cue was realized by modification of neural activity relatively early in the visual processing stream. To further restrict possible cortical loci for the learning effect, we measured the extent of ocular transfer for the recruited cue.

Subjects were trained using monocular stimuli. The direction of rotation of training stimuli was specified using occlusion, shape-from-shading and internal occlusion, and haze cues (Figure 1c), as detailed in the General methods section. Test trials used monocular versions of the training stimuli, with all cues to depth removed (Figure 1d). All other parameters including trial sequence and spatial arrangement of stimuli were as for Experiment 1. As for Experiment 1, subjects were trained with a fixed location– rotation contingency on Day 1 and with the opposite location–rotation contingency on Day 2.

Sixteen new subjects took part in Experiment 4. Within an experimental session, all training and test stimuli were presented to the same eye. Eight subjects were presented with stimuli to the left eye on Day 1, and eight subjects were presented with stimuli to the right eye. On Day 2, with reversed contingency training, half of each group of subjects was presented with stimuli in the opposite eye to that used on Day 1, and the other half was presented with stimuli in the same eye. All stimuli were green; the eye of presentation was controlled by reversal of the red–green glasses.

Results

A 3-way ANOVA (with Testing Day as a within-subjects factor, and Day 1 Eye and Day 2 Eye as between-subjects factors) showed that zDiff scores were significantly higher on Day 1 than on Day 2 (Day 1 mean = 3.64, Day 2 mean = 0.04; F(1,12) = 59.57, p < 0.001). There was no significant interaction between the eye tested on Day 1 and zDiff scores on Day 1 or Day 2, or between the eye tested on Day 2 and zDiff scores on Day 1 (self-evident) or Day 2, i.e., the eyes were equivalent. Hence zDiff scores are presented below in two groups (Figure 9); “same” eye used on both days, and “swap” of eyes between the days.

Figure 9.

zDiff scores on Day 1 and Day 2 for subjects who were tested with the same eye on both days or who swapped eyes between the 2 days.

Subjects who used the same eye on both days showed only slightly greater retention of the location cue, as reflected in their Day 2 zDiff score, as compared to subjects who swapped eyes. This was mirrored in the interaction of Testing Day, Day 1 Eye, and Day 2 Eye in the above 3-way ANOVA (F(1,12) = 6.383, p = 0.080), which was suggestive but did not reach significance. While it is possible that ocular transfer of the recruited retinal location cue is incomplete, without doubt the bulk of the learning occurs within binocular neural mechanisms.

Overall results from Experiment 4 are presented alongside results from Experiments 1 to 3 in Figure 6. While our monocular training stimuli led to effective recruitment of retinal location on Day 1, we observed less cue retention on Day 2 than had been observed with binocular training stimuli. However, a complete absence of cue retention would have resulted in zDiff scores of equal magnitude but opposite sign on Day 2; this was not the case. Instead, significant cue retention was observed.

General discussion

Location can be recruited as a cue to appearance, using a bistable Necker cube stimulus, in an associative learning paradigm (Backus, 2009; Backus & Haijiang, 2007; Haijiang et al., 2006). The experiments presented here show that this location cue is dependent on the retinal position of the stimulus only. World location, which might have corresponded to testing room, location within the room, location relative to the body, or location relative to the head, was not recruited as a cue in the short training regimes used here. Lastly, the internal representation of the recruited location cue is largely binocular, as shown by substantial ocular transfer. We surmise that learning of the cue is instantiated in retinotopically selective areas beyond layer 4 of V1, which is believed to be the last cortical site where information from the two eyes is completely segregated (Hubel & Wiesel, 1977).

Implications for neural basis of learning

These findings have implications for the cortical locus of learning of the location cue, and therefore also for what exactly about the stimulus is learned. The strong and pervasive influence of stimulus’ retinal location, combined with our lack of success in training other types of location as cues to appearance, suggest that holistic object rotation may not be the perceptual attribute that becomes contingent on the new cue. Whole-object representations are generally considered to be due to activity in cortical areas that are far less selective for retinal location, such as lateral occipital cortex (Grill-Spector et al., 1998; Lerner, Hendler, Ben-Bashat, Harel, & Malach, 2001). Our results are more compatible with the known tuning properties of macaque MT, where the velocity and disparity of edge components and surface dots in our stimuli could be encoded (DeAngelis, Cumming, & Newsome, 1998; DeAngelis & Uka, 2003; Maunsell & Van Essen, 1983a, 1983b). Indeed, MT responses in both macaque (Bradley, Chang, & Andersen, 1998; Dodd, Krug, Cumming, & Parker, 2001; Grunewald, Bradley, & Andersen, 2002) and humans (Brouwer & van Ee, 2007) have been shown to correlate with perceptual outcome for ambiguous structure-from-motion stimuli similar to those used here.

Is it plausible that MT neurons could support a learned bias to perceive leftward motion as near and rightward motion as far, or vice versa? MT neurons have retinotopic receptive fields, so this bias could be trained specific to retinal location. MT also holds traces of ocular dominance (Kiorpes, Walton, O’Keefe, Movshon, & Lisberger, 1996; ; Zeki, 1978), which would be consistent with greater cue retention on Day 2 in the same eye than in the opposite eye than had been trained on Day 1. Our results in Experiment 4 were suggestive, but non-significant, in that respect. In sum, our results are consistent with a clockwise or counterclockwise cube rotation percept that is built from the combination of localized motion components with disparity biases.

Relationship to studies of perceptual stabilization

To what extent is recruitment of retinal location cues compatible with multiple percept stabilizations in different parts of the visual field? While the neural dynamics underlying percept stabilization are not our primary concern here, the similarities between our Day 1 results and previous studies that have investigated the perceptual outcome of sequences of bistable stimuli should be noted.

The perceptual outcome for strings of ambiguous stimuli randomly interleaved with unambiguous stimuli as used here, and its variation with stimulus duration and inter-stimulus interval, has not to our knowledge been systematically studied. However, intermittent presentation of ambiguous rotating Necker cubes only, at a single location, may result in either repetition of the same percept, or alternation between the two possible percepts (Leopold, Wilke, Maier, & Logothetis, 2002). The perceptual outcome depends on presentation timing (Klink et al., 2008; Noest, van Ee, Nijs, & van Wezel, 2007), with blank inter-stimulus intervals of 2 s or more reliably resulting in perceptual stabilization, independent of the stimulus presentation time. This result was found using strings of ambiguous presentations; the perceptual outcome of a single ambiguous presentation following an unambiguous presentation likewise depends on the inter-stimulus interval (Long & Moran, 2007).

The ISI between stimuli was typically just under 2 s in our self-paced trials (and due to the randomized sequence of presentations, was often much longer for stimuli presented at the same location); this is a duration that would be predicted to lead to perceptual stabilization in strings containing ambiguous stimuli only. Hence as regards presentation timing, the short-term recruitment of retinal location cues on Day 1 is compatible with existing studies of percept stabilization. Note, however, that the persistence of recruited cues on Day 2, in the face of training with reversed location–rotation contingency, is not predicted by the existence of such short-term effects. While we would agree that a history-driven bias drives the perceptual outcome at stimulus onset (Brascamp et al., 2008), existing models would need to extend to longer timescales and account for the persistence of a trained bias in the face of equal but opposite training, in order to account for our long-term effects.

The retinal location specificity of the long-term learning is also compatible with previous work investigating the spatial extent of within-session percept stabilization. For example, percept stabilization of a bistable rotating cylinder is specific to retinal location (Chen & He, 2004). In some models, negative adaptation aftereffects and positive priming aftereffects are time-dependent outcomes of the same underlying neural dynamics (Klink et al., 2008; Noest et al., 2007); in that case previous demonstrations of the retinotopic nature of negative adaptation effects for rotating Necker cube stimuli (Long, Toppino, & Kostenbauder, 1983; Toppino & Long, 1987) also lead to an expectation of retinotopic percept stabilization. However, others have suggested that stabilization has a more global spatial extent than adaptation (Long & Moran, 2007; Orbach, Ehrlich, & Heath, 1963). Again, note that these studies do not address the long-term learning we have observed; they merely provide a basis for expectation (or otherwise) of cue recruitment on Day 1. Furthermore, previous demonstrations that the visual system could recruit a retinal location cue did not preclude the possible recruitment of other types of location as cues to appearance.

Long-term biases specific to retinal location have also been shown for binocular rivalry (Carter & Cavanagh, 2007). In this case, the biases were not “learned” but assumed to be preexisting. Analogous to the persistence of our long-term learning effect in the face of reverse contingency training, the long-term biases demonstrated by Carter and Cavanagh (2007) were shown to recover after short-term adaptation effects decayed.

Retinal location as a cue

To what extent should retinal location be considered a “cue”? To recapitulate, our data suggest that two separate populations of neurons, selective for two different parts of visual field, were trained to have different long-lasting biases. Formally speaking, this learning constitutes cue recruitment: retinal location came to be utilized by the visual system for disambiguating otherwise ambiguous stimuli. The direction of the learning was in agreement with long-trusted cues that were correlated with retinal location during training. However, the original tests of cue recruitment (Haijiang et al., 2006) were devised with a different sort of “cue” in mind. It was hypothesized that a high level representation of location such as “near the top of the display screen” would be recruited, but that was not found to be the case here. Of course, the visual system must use signals that it can measure, and retinal position is easily measured, so one might have predicted that the visual system would learn to utilize retinal position. Yet “near the top of the display screen” can also be reliably determined by the visual system, and this cue was not recruited.

To explain this difference in recruitment as a functional difference, one must appeal to notions of optimality. Within a Bayesian formulation, for example, one would have to posit that the visual system believes, a priori, that retinal location is a more likely indicator of 3D rotation than is location within the environment. We cannot see why that would be the case. Accordingly, to us the most satisfying explanation of these effects is not functional (with high learning rates for the most plausible cues) but rather mechanistic (with stimulation history causing changes to the pattern of activity in relevant cell assemblies). We can only speculate that some cues are more readily recruited than others as a result of idiosyncratic characteristics of plasticity in specific neural machinery.

Monocular training, MT dynamics, and feedback

Successful training of the retinal location cue using stimuli disambiguated by monocular cues to depth (Experiment 4) is of special interest because monocularly disambiguated training trials had the same effect as stimuli disambiguated by disparity signals (Experiment 1). We suggest that the percept of rotation on training trials, whether evoked by monocular or binocular depth cues, led to motion-related activity in a common population of neurons, which we propose to reside in MT. While we cannot rule out the possibility that cue recruitment has different cortical loci under monocular and binocular training regimes, our suggestion is more parsimonious with existing reports: After the association of shape cues with motion, shape cues come to activate direction-selective neurons in macaque MT (Schlack & Albright, 2007), and pictorially recalled motion can lead to activation of MT neurons (Kourtzi & Kanwisher, 2000). Similarly, it is likely that the monocularly provided experience of motion and depth, via top-down pathways, resulted in activation of a common population of MT neurons here.

Conclusion

The previously demonstrated recruitment of location as a cue to appearance in ambiguous rotating Necker cube stimuli is mediated solely by retinal location in the short training sessions used here. Both binocularly and monocularly disambiguated cubes are effective in the associative learning paradigm. Overall, our results are consistent with learning about the depth of moving components of the 2D stimulus, and we do not have to appeal to learning about the 3D rotation of the holistic cube. This learning could be instantiated by biases in the dynamics of neurons in retinotopic visual areas that are jointly tuned to motion and disparity, such as area MT.