Fixating picture boundaries does not eliminate boundary extension: Implications for scene representation

Abstract

Observers frequently remember seeing more of a scene than was shown (boundary extension). Does this reflect a lack of eye fixations to the boundary region? Single-object photographs were presented for 14–15 s each. Main objects were either whole or slightly cropped by one boundary, creating a salient marker of boundary placement. All participants expected a memory test, but only half were informed that boundary memory would be tested. Participants in both conditions made multiple fixations to the boundary region and the cropped region during study. Demonstrating the importance of these regions, test-informed participants fixated them sooner, longer, and more frequently. Boundary ratings (Experiment 1) and border adjustment tasks (Experiments 2–4) revealed boundary extension in both conditions. The error was reduced, but not eliminated, in the test-informed condition. Surprisingly, test knowledge and multiple fixations to the salient cropped region, during study and at test, were insufficient to overcome boundary extension on the cropped side. Results are discussed within a traditional visual-centric framework versus a multisource model of scene perception.

Boundary extension is an error in scene representation in which memory overflows the boundaries of a view (Intraub & Richardson, 1989; see Hubbard, Hutchison, & Courtney, 2010; Intraub, 2010; Michod & Intraub, 2009). Why do we remember seeing what we clearly never saw? This is a mystery only if scene perception and memory are conceptualized in terms of the visual input alone. Intraub and Dickinson (2008) and Intraub (2010, 2012) proposed that in addition to the visual sensory input, scene representation includes multiple top-down sources of information, such as amodal perception of surfaces and objects cropped by a boundary (Kanizsa, 1979; Kellman, Yin, & Shipley, 1998), scene classification (Greene & Oliva, 2009; Potter, 1976, Thorpe, Fize, & Marlot, 1996), and object-to-context associations (Bar, 2004). Once the sensory input is gone (even for an interval as brief as a saccade; Dickinson & Intraub, 2008), top-down information just beyond the original boundaries is misattributed to vision, thus causing boundary extension. What is not known, however, is whether this misattribution occurs because observers fail to fixate near the boundary and thus do not obtain a high-quality visual representation of that region, or whether they do fixate the boundary region, but high-quality visual input is simply not sufficient to overcome boundary extension.

In his review of oculomotor behaviour and cognition, Rayner (2009) argued that in contrast to research on reading, eye tracking has been underutilized in the study of scene perception and memory. We think this point is well taken in the case of boundary extension. Although eye tracking has played a role in research exploring the early time course of boundary extension (Dickinson & Intraub, 2008, 2009; Intraub & Dickinson, 2008; Intraub, Hoffman, Wetherhold, & Stoehs, 2006), questions about the pattern of fixations in more extended viewing tasks have not been pursued, perhaps because it has seemed so obvious that participants could see the boundaries because they fall within the field of view. Falling within the field of view, however, does not necessarily mean that the boundary region was fixated or attended to.

Since the earliest research on boundary extension (Intraub & Richardson, 1989), the visual angles of the pictures have been small enough for the view boundaries to fall within the field of view,¹ requiring no head movements to be seen. If participants had simply maintained central fixation, the boundaries would fall either in the parafoveal region of the eye (within 5° of fixation; see Rayner, 2009) or in the near periphery, depending on where they were seated in the room. However, with picture duration set at 15 s, and participants instructed to attend not only to the details of the main objects but to the background as well (the standard instruction), participants could make up to 45 eye fixations per picture. These fixations to objects in the scene would probably bring the boundaries of the picture into the parafoveal region, even if they were not directly fixated. Under these conditions, there was no question that observers could see the boundaries of the view; however, we have no information about whether they fixated the boundary region, gaining high-acuity visual information near the edges of the picture.

In an effort to draw participants’ attention to the boundaries of the view, Intraub and Bodamer (1993) added two types of information to the standard instruction described earlier. Unlike in the control condition, in which the type of memory test was left unspecified during study, participants in the experimental conditions were informed prior to study: (a) that memory for the expanse of the view would be tested or (b) about the phenomenon of boundary extension and were challenged to prevent it. Surprisingly, neither type of additional information eliminated boundary extension. However, both instructions led to a reduction in boundary extension as compared to the standard instruction alone. The cause of this reduction and its relation to oculomotor activity remains an open question. If we are to fully understand why people remember seeing more of the world than was shown, it is important to determine whether people do indeed fixate the boundaries of a picture and whether instructions to remember boundary placement impacts oculomotor behaviour both when the observer first studies the stimulus and then at test when the observer is making decisions about remembered boundary placement.

Although one would assume that instructions that explicitly reference the boundaries, such as those provided in Intraub and Bodamer’s (1993) experimental conditions, would lead to multiple fixations to the boundary region, there is evidence in the eye-tracking literature that raises questions about this assumption. First, it has been reported that observers demonstrate a strong bias to fixate the objects rather than the background of a display (Neider & Zelinsky, 2006; Vincent, Baddeley, Correani, Troscianko, & Leonards, 2009). Second, Tatler (2007) reported a bias to fixate the centre of a picture, even when the scene was composed in a way that would lead one to expect that it would draw the eyes toward one of the picture’s edges (such as an off-centre view of a sink in which all the main objects are near the bottom edge of the picture).

These findings raise the possibility that a failure to fixate the boundary region is in large part responsible for boundary extension. In contrast, if participants do fixate the boundary region, but nonetheless remember having seen more of the world than was shown, this would suggest that some distinctions be made about the role of fixations in scene perception and memory. High-acuity information obtained from fixations clearly benefits memory for details of objects within a view (Henderson & Hollingworth, 1999; Hollingworth & Henderson, 2002). Does this type of information also help the observer remember the spatial expanse of a view; specifically, where the view ended? Put another way, the boundary of a picture may simply be another detail of the photograph that participants can remember given sufficient visual information, or, in contrast, it may be a spurious terminus that is readily observable when the sensory information is present, but that loses clarity in the context of a scene representation that is graded—decreasing in detail from information gained through fixations, to information in the periphery, to information from amodal perception and world knowledge.

This issue motivated us to study oculomotor activity near the view-boundaries of pictures, both at study (when first examining the picture) and at test (when making a decision about boundary placement) in the context of the standard viewing instruction and in the context of a test-informed instruction. In addition to replicating Intraub and Bodamer (1993) while tracking movements of the eyes, we introduced two new variables to allow us to better understand the role of high-acuity visual information in memory for boundary placement. The first variable was whether or not the main object was cropped by a view-boundary. We reasoned that the point of contact between a boundary and the object would create a salient local marker of boundary placement. This would provide a potential fixation target that might attract fixations, particularly in the test-informed conditions in which participants would try to remember boundary placement. The presence of a local marker would allow more traction in determining the relationship of fixation activity to memory for the boundaries of a view. Unlike prior studies of occlusion and boundary extension, in the current research, occlusion was systematically studied by presenting the same scenes either with or without the object being cropped.

The second variable is a new dependent variable that would provide a quantitative assessment of remembered boundary placement, particularly relevant to assessing memory for the placement of a boundary that had cropped the main object. Prior recognition memory tests showed that boundary extension occurred whether or not an object was cropped by a boundary (Bertamini, Jones, Spooner, & Hecht 2005; Intraub, Bender, & Mangels, 1992; Intraub & Bodamer, 1993). However, recognition memory was assessed via a boundary rating scale in which observers rated the view as a whole (as being the same, more close up, or more wide angled; Intraub & Bodamer, 1993; Intraub & Richardson, 1989). Thus we do not know whether observers remembered the cropped relation; we know just that they remembered having seen a more wide-angle view, in spite of the cropped relation. For this reason, after replicating Intraub and Bodamer (1993) using the boundary rating task in combination with eye tracking in Experiment 1, in Experiments 2–4 we used a border adjustment task (Intraub et al., 2006) in which the participant could adjust each boundary (top, bottom, left, and right) independently to recreate the remembered view. This allowed us to determine not only whether boundary extension occurred (i.e., if people remembered seeing a greater expanse of the scene than was shown), but also whether or not participants remembered border placement when a salient marker of boundary placement was provided.

In the following experiments, we addressed three questions about oculomotor behaviour and long-term memory for the expanse of a view. Do participants fixate the boundary region during study and at test? Does forewarning that boundary memory will be tested affect oculomotor behaviour? Does a salient marker of boundary placement (cropping the main object with a picture boundary) serve as a saccade target, and, if so, will this eliminate boundary extension on the cropped side?

EXPERIMENT 1

In Experiment 1, we sought to determine where participants look when trying to remember pictures in a boundary extension experiment similar to that of Intraub and Bodamer (1993). Both groups were told to attend to the background as well as to the main object, but the test-informed group also was forewarned that boundary memory would be tested. In half of the 12 pictures, the main object was cropped by one of the picture’s boundaries to provide a salient marker of boundary placement. As in the earlier study, pictures were presented for 15 s each, allowing participants to make multiple eye fixations. Areas of interest for the eye-tracking analysis included a region around the boundary and, in the case of the cropped-object pictures, the area within that region where the boundary cropped the object. Eye movements were also monitored at test to determine whether participants checked the boundary and cropped regions when rating the scope of the views.

Method

Participants

Participants were 72 University of Delaware undergraduates (47 females) fulfilling a requirement for an introductory psychology course.

Stimuli

Stimuli were 12 digital photographs. Each was a close-up view of a single main object on a natural background, such as a basketball on a gym floor in which the main object filled, on average, 34% of the picture space. Two versions of each picture were created by cropping a wider angle view using Adobe Photoshop. There was a whole-object version in which the entire object was visible and was surrounded on all sides by the background, and a cropped-object version in which the cropping box was shifted slightly so that the main object was cropped by a single picture-boundary (whole-object versions of each picture are shown in Appendix A; descriptions indicating which side was cropped in the cropped-object version are listed in Appendix B). Cropping was obvious, but very slight; on average 1.2% of each object was cropped by the boundary. Stimuli were cropped equally often by the top, bottom, left, and right boundaries; three pictures were cropped at the top, three at the bottom, three on the left, and three on the right. Figure 1 provides an example of the whole-object version and cropped-object version for two scenes. All stimuli were 547 × 365 pixels, subtending approximately 15° × 10° of visual angle. They were presented in the centre of a grey background (1,024 × 768 pixels).

Figure 1.

Whole-object version (A & C) and cropped-object version (B & D) of two stimuli.

Apparatus

Stimuli were presented on a 21-in. flat-screen CRT monitor in 32-bit colour at a resolution of 1,024 × 768 pixels. The refresh rate was 120 Hz (the video card contained 128 Mb of video memory). Stimulus presentation was controlled by a Dell Dimension DIMXPS (P4/2.8 GHz) running Microsoft Windows XP. The software was based on a template program supplied by SR Research Ltd. written in the C programming language with Microsoft Visual C++ 6.0 that used Simple DirectMedia Layer (SDL) v.1.2.9. and the EyeLink Windows API Version 2.0 (©) 1997–2002 by SR Research Ltd. Eye data were recorded with an EyeLink II head-mounted video eye-tracking system (SR Research Ltd., Mississauga, Ontario, Canada). Eye position was sampled at 500 Hz, the system’s spatial resolution was estimated to be less than 0.4°, and head position and viewing distance were fixed with a chin rest.

Design and procedure

Participants were randomly assigned to the test-naïve (n = 36) or test-informed conditions (n = 36).

All participants were instructed to remember each picture in as much detail as possible and to pay equal attention to the background and to the main object. In addition, test-informed participants were told that the scenes would be shown again at test and that they would judge whether the view was the same as before, or if the camera was “closer up” or “farther away”. They were shown an example of closer and wider views of a bicycle against a fence (the same sample as that used by Intraub & Bodamer, 1993). The purpose of the sample views was to illustrate that the structure of the scene (i.e., the size of the object and amount of background visible) changes as a function of the camera’s position. They were told that when the camera is closer up, the object fills more of the picture space, and less of the background is shown, whereas when the camera is farther away, the object fills less of the picture, and more of the background is shown. While studying the pictures, they were encouraged to pay attention to the relationship between the object and the background—for example, to remember how much of the background the object covered and the size of the object in relation to the background.

The 12 scenes were always presented in the same order but picture version (whole-object vs. cropped-object) was counterbalanced; the six scenes shown as cropped-object pictures to half the participants were shown as whole-object pictures to the other half in each instruction condition. In addition to these counterbalancing procedures, the pictures were mirror-reversed for half the participants. Versions (cropped vs. whole-object) were intermixed with the constraint that no more than two of a given type could appear in a row.

The experimental sequence was presented as follows. Participants fixated a yellow fixation cross at the centre of the screen and pressed the space bar. This initiated a 500-ms delay, followed by the first picture (15 s) and a 300-ms mask. The fixation cross followed for 1 s (the cross was yellow for the first 300 ms and then turned red for the remaining 700 ms as a warning that the next picture would soon appear), followed by the next picture (15 s) and a 300-ms mask. This continued until all 12 pictures had been shown. Thus the stimulus onset asynchrony (SOA) was 16.3 s. Eye movements were recorded throughout the sequence. Drift correction was performed at the start of the sequence.

After presentation, the test instruction was read to all participants along with a sample trial; these instructions took approximately 2 min. Although test-informed participants already knew the instruction, in this way we ensured that the retention interval was the same across instruction conditions. At test, the identical views of each picture were presented (participants were told that the view could be the same, closer up, or farther away). Picture order was changed, but again no more than two of a given picture type (cropped vs. whole-object) was presented consecutively. Participants rated each picture on a 5-point scale to indicate whether they thought it was taken by a camera that was much closer up (−2), a little closer up (−1), the same distance (0), a little farther away (1), or much farther away (2) than before. These choices appeared under the picture, and the participant used the mouse to indicate their choice.

These were replaced by a choice of confidence ratings: sure (3), pretty sure (2), not sure (1), and an alternative option of “don’t remember picture” (DRP) as in previous boundary extension experiments. All DRP responses were removed from analyses. After both responses were entered, the next test picture appeared. Eye movements were recorded during each test picture presentation. Drift correction was performed at the start of each test trial.

Defining regions of interest

Figure 2 shows examples of the two regions of interest for two stimuli. The boundary region was defined as a rectangular region that included the boundary and extended inward for 1.5° of visual angle and outward beyond the boundary (in the grey area outside the picture) for 0.5° of visual angle. The inward extent allowed us to capture cases in which the participant was obtaining high-acuity information from the picture near the boundary; the outward extent was included to minimize loss of fixation information due to calibration error. We defined the cropped region as a rectangle falling within the boundary region; its extent was determined by drawing two parallel lines, perpendicular to the boundary, that encompassed the edges of the cropped area plus some additional space (0.25° or 0.5°), depending on the shape of the object. The cropped regions subtended a visual angle in this dimension that ranged from 2.54° to 11.01° depending on the shape of the main object and how it was cropped.

Figure 2.

Examples of the two regions of interest for two stimuli. The boundary region was the same for all stimuli; examples are shown in Panels A and C. The cropped region was tailored to each picture as described in the text; examples are shown in Panels B and D.

Results and discussion

Oculomotor behaviour at study

We assessed fixation activity in the boundary and cropped regions in four ways: (a) the mean percentage of time spent fixating the region, (b) the mean percentage of all fixations that were made to that region, (c) the mean index of the first fixation to that region (for example, if the index is 10, the first fixation in the region was the 10th fixation on the picture), and (d) the mean latency of the first fixation to the region (i.e., how soon the region was fixated). Fixation activity in the boundary region for all pictures is shown in Table 1 (top). As can be seen, test-naïve participants did fixate the boundary region. Test-informed participants, however, fixated it sooner, longer, and more frequently than did test-naïve participants. For example, test-naïve observers spent 15.4% of their time fixating the boundary region, whereas 26% of the test-informed observers’ time was devoted to this region. Next we examined fixation activity in the boundary region for whole-object and cropped-object pictures. As shown in Table 1 (middle), the same patterns were observed, with the exception of latency to the first fixation: The difference approached but did not reach significance (p = .06).

Table 1.

Oculomotor behavior in the boundary region during study (Experiment 1, N = 72)

Oculomotor behavior in boundary region	Test-naïve	Test-informed	t(70)	Sig.	p
All pictures
Percent of time fixating	15.4	26.0	4.5	***	<.001
Percent of fixations	16.2	25.4	4.6	***	<.001
Index of first fixation	11.6	9.1	3.5	***	<.001
Latency of first fixation (ms)	4135	3167	3.0	**	.004
Whole-object pictures
Percent of time fixating	15.7	24.4	3.4	**	.001
Percent of fixations	16.4	24.1	3.5	***	<.001
Index of first fixation	11.7	9.6	2.1	*	.017
Latency of first fixation (ms)	4117	3371	1.9	n.s.	.059
Cropped-object pictures
Percent of time fixating	15.1	27.6	5.0	***	<.001
Percent of fixations	16.1	26.7	4.9	***	<.001
Index of first fixation	11.6	8.7	3.2	**	.002
Latency of first fixation (ms)	4157	2966	3.0	**	.003

Note: Sig. = Significance; n.s. = not significant.

^*p<.05;

^**p<.01;

^***p<.001.

The cropped region itself drew attention. Table 2 shows oculomotor behaviour during study in the cropped region of the cropped-object pictures. As shown, both groups fixated the cropped region but, as with the boundary region, the test-informed group fixated it sooner, longer, and more frequently, indicating they deemed this region important for success on the upcoming memory test. The boundary region included the cropped region; thus to determine whether this increase was limited to activity in the cropped portion only (or instead reflected increased activity to other parts of the boundary region) we analysed fixation activity to the boundary region after subtracting the cropped region. As shown in Table 2 (bottom), the increase in fixation activity to the boundary region was not limited to the cropped region alone.

Table 2.

Oculomotor behavior in the cropped region during study (Experiment 1, N =72)

Oculomotor behavior in cropped region	Test-naïve	Test-informed	t(70)	Sig.	p
Cropped-object pictures
Percent of time fixating	4.6	11.1	4.5	***	<.001
Percent of fixations	4.5	9.7	5.1	***	<.001
Index of first fixation	15.9	11.0	4.1	***	<.001
Latency of first fixation (ms)	5796	3758	3.8	***	<.001
Boundary region minus cropped region
Percent of time fixating	10.6	16.5	3.5	***	<.001
Percent of fixations	11.6	17.0	3.4	**	.001
Index of first fixation	14.1	13.4	0.7	n.s.	.474
Latency of first fixation (ms)	5029	4858	0.4	n.s.	.665

Note: Sig. = Significance; n.s. = not significant.

^*p<.05;

^**p<.01;

^***p<.001.

Finally, to determine whether increased fixation activity to the cropped region was due to the cropped relation between the object and border, and not to some other salient factor about that part of the object, we compared fixation activity in the cropped region to the identical space in the whole-object versions (where there was no cropping). These analyses are shown in Table 3. As shown in the table, no differences were observed for test-naïve observers. However, the region was the focus of greater fixation activity for test-informed observers, suggesting that test-informed observers recognized the importance of the cropped region as a marker of boundary placement.

Table 3.

Comparisons of oculomotor behavior for “cropped region” in cropped-object and whole-object pictures (Experiment 1, N= 72)

Oculomotor behavior in “cropped region”	Whole-object pictures	Cropped-object pictures	t(70)	Sig.	p
Test-naïve
Percent of time fixating	4.6	4.6	.04	n.s.	.97
Percent of fixations	4.6	4.4	.02	n.s.	.84
Index of first fixation	16.8	15.9	.81	n.s.	.42
Latency of first fixation (ms)	5993	5797	.26	n.s.	.78
Test-informed
Percent of time fixating	6.6	11.1	3.2	**	.003
Percent of fixations	6.5	9.7	3.1	**	.004
Index of first fixation	13.9	11.0	3.4	**	.002
Latency of first fixation (ms)	4842	3758	3.3	**	.002

Note: Sig. = Significance; n.s. = not significant.

^*p<.05;

^**p<.01;

^***p<.001.

Taken together, the oculomotor data show that participants fixated the boundary region and cropped region in response to the standard boundary extension instruction, and that prior test-knowledge increased fixation activity to those areas. Did these well-placed fixations result in the elimination of boundary extension? Surprisingly they did not.

Boundary ratings

Figure 3 shows mean boundary ratings as a function of instruction and picture type. As shown in the figure, significant boundary extension occurred in each case. A 2 × 2 Test Instruction (test-naïve, test-informed) × Picture Type (whole-object, cropped-object) mixed measures analysis of variance (ANOVA) on the mean boundary ratings showed that prior test knowledge reduced boundary extension, F(1, 70) = 8.13, p = .006, but that, surprisingly, the presence of a cropped object had no effect on boundary memory, F < 1. There was no interaction, F < 1. Although participants in both conditions had fixated the cropped regions during presentation, this did not yield accurate memory on the boundary rating task. As in Intraub and Bodamer (1993), confidence ratings indicated that participants were “pretty sure”; mean confidence ratings ranged from 2.1 to 2.17 across the four conditions (2.0 = “pretty sure”).

Figure 3.

Mean boundary rating for whole-object pictures and cropped-object pictures in the test-naïve and test-informed conditions; error bars show the 95% confidence interval for each mean (Experiment 1).

Oculomotor behaviour at test

Oculomotor behaviour at test was assessed in the same way as during presentation. Fixation activity in the boundary region and in the cropped region is shown in Tables 4 and 5, respectively. At test, participants in both conditions clearly fixated the critical regions of interest. The mean amount of time that participants spent exploring the pictures before responding was 6,184 ms in the test-informed condition and 5,061 ms in the test-naïve condition; this trend approached but did not reach significance, t(70) = 2.0, p = .055.

Table 4.

Oculomotor behavior in the boundary region during test (Experiment 1, N = 72)

Oculomotor behavior in boundary region	Test-naïve	Test-informed	t(70)	Sig.	p
All pictures
Percent of time fixating	21.1	30.3	4.0	***	<.001
Percent of fixations	21.0	28.0	3.5	***	<.001
Index of first fixation	7.7	6.8	1.8	n.s.	.081
Latency of first fixation (ms)	2073	1737	1.9	n.s.	.061
Whole-object pictures
Percent of time fixating	20.1	28.8	3.3	**	.001
Percent of fixations	20.0	26.7	2.8	**	.006
Index of first fixation	8.3	7.2	1.7	n.s.	.092
Latency of first fixation (ms)	2253	1876	1.7	n.s.	.088
Cropped-object pictures
Percent of time fixating	22.0	31.5	3.8	***	<.001
Percent of fixations	21.6	29.1	3.5	***	<.001
Index of first fixation	7.2	6.5	1.1	n.s.	.267
Latency of first fixation (ms)	1891	1630	1.3	n.s.	.199
Boundary region minus cropped region
Percent of time fixating	9.5	13.9	2.8	**	.007
Percent of fixations	10.0	13.5	2.3	*	.023
Index of first fixation	11.2	11.4	0.1	n.s.	.890
Latency of first fixation (ms)	3131	3205	0.2	n.s.	.831

Note: Sig. = Significance; n.s. = not significant.

^*p<.05;

^**p<.01;

^***p<.001.

Table 5.

Oculomotor behavior in the cropped region during test (Experiment 1, N =72)

Oculomotor behavior in cropped region	Test-naïve	Test-informed	t(70)	Sig.	p
Cropped-object pictures
Percent of time fixating	12.5	17.6	2.7	**	.008
Percent of fixations	11.6	15.7	2.5	*	.015
Index of first fixation	8.5	7.6	1.1	n.s.	.269
Latency of first fixation (ms)	2192	1984	0.8	n.s.	.399

Note: Sig. = Significance; n.s. = not significant.

^*p<.05;

^**p<.01;

^***p<.001.

The most obvious difference in eye activity between study and test can be observed in the case of the test-naive participants. Now that they knew the nature of the test, they increased their focus on the critical regions. For example, the percentage of time spent fixating the cropped region increased from 4.6% at study to 12.5% at test. Pairwise comparisons shown in Tables 4 and 5 reveal that test-informed participants persisted in fixating the boundary and cropped regions more than did the test-naïve participants. Differences in the latency of first fixation were no longer significant; this is probably the case because test-naïve observers were now aware of the type of memory test.

In sum, both test-naïve and test-informed participants fixated the critical regions during study and test. Test-informed participants recognized the importance of these regions, increasing activity to the boundary and cropped regions. Yet, in spite of this seemingly efficacious scanning strategy, though boundary extension was attenuated, it persisted in the test-informed condition, even for cropped-object pictures. But was the cropped relation remembered? This question was addressed in Experiment 2.

EXPERIMENT 2

Although participants fixated the cropped region, there was no reduction in boundary extension for cropped-object pictures. This may indicate that the cropped relation was not remembered. On the other hand, it might be the case that the cropped side was remembered as such (like an anchor), and that extension was increased on the other three sides, so that they yielded more boundary extension than the comparable sides in the whole-object condition, as illustrated in Figure 4. Boundary ratings can provide no purchase on this question. For this reason, in Experiment 2, we tested boundary memory for the same pictures using a border adjustment task (Dickinson & Intraub, 2009; Gagnier et al., 2011; Intraub et al., 2006). Because test instructions were the same as those in Experiment 1, and our interest was in boundary placement, this research was run without the eye tracker.

Figure 4.

The stimulus view of the “light bulb on grass” (top panel) and two alternative ways in which boundary extension may have occurred: boundary extension that includes a failure to remember that the object was cropped (middle panel), and boundary extension in which the observer remembers seeing more of the scene, but not on the cropped border (lower panel).

Method

Participants

Participants were 72 (49 female) University of Delaware undergraduates fulfilling a requirement for an introductory psychology course (originally n = 72, but one outlier was eliminated from the test-informed condition because the mean percentage increase in area was greater than 3 standard deviations from the test-informed mean).

Stimuli

All but one picture were the same as those in Experiment 1. The “roll of paper” picture was replaced with a new picture of a roll of paper on a tile floor because we did not have a wide enough view of the original scene to be used in a border adjustment task.

Apparatus

Pictures were presented by a program that was based on a template program supplied by SR Research Ltd. written in the C programming language with Microsoft Visual C⁺⁺ 6.0 that used Simple DirectMedia Layer (SDL) v.1.2.9. and the EyeLink Windows API version 2.0 (©) 1997–2002 by SR Research Ltd. They were shown on a 21-in. Dell monitor with the screen resolution set to 1024 × 768 × 32 bits of colour, which was run by a Dell Dimension DIMXPS (P4/2.8 GHz) running Windows XP. The viewing distance was approximately 80 cm.

Design and procedure

Instructions and counterbalancing were the same as those in Experiment 1. The presentation sequence was similar, with the following exceptions. The sequence began with a yellow fixation cross that remained on the screen until participants pressed the space bar. The first picture was presented for 14 s followed by a 1-s masked interval, which was immediately followed by the next picture; the sequence continued like this until all 12 pictures had been shown.

Test procedure

The same pictures were presented in the same order at test, and the position of the borders was the same as that during study. Participants were instructed to decide whether the boundaries were in the same position as before or differed. If they felt they were the same, they could simply accept it by pressing the space bar without making any adjustments. If they differed, they were asked to adjust the borders to recreate the remembered view. Each border could be moved inward or outward using the mouse. The instructions took approximately two minutes. Participants were allowed to practise moving the borders on one of the pictures shown in the practice sequence (this was not one of the 12 stimuli). The test sequence followed. On each trial, participants rated their confidence by using the mouse to click on “sure” (3), “pretty sure” (2), or “not sure” (1). As in Experiment 1, if they did not remember seeing the picture at all, they were asked to click on DRP (don’t remember picture).

Measurement

To determine the percentage change in area, the number of pixels in the reconstructed view was divided by the number of pixels in the stimulus view and multiplied by 100. To assess changes in border position, the distance between the centre of the picture and each border was measured (in pixels) in the original and in each participant’s reconstruction. On each border, the difference between the two was divided by the original distance and multiplied by 100 to obtain the percentage change at each border. It was decided a priori that any participant whose mean area remembered was 3 standard deviations or greater from the group mean would be excluded from analysis.

Results and discussion

One test-informed participant’s data were excluded from the analysis. Participants selected “do not remember picture” on 0.5% of the trials in each instruction condition. Mean confidence ratings were 2.0 (SD = 0.4) in the test-naïve condition and 2.2 (SD = 0.4) in the test-informed condition.

Test views were identical to the stimulus views, but participants frequently “corrected” them by moving the borders outward. Figure 5 (top) shows the mean percentage change in area in each condition for both types of pictures. Both whole-object and cropped-object pictures were remembered with extended boundaries. A 2 × 2 Test Instruction (test-naïve, test-informed) × Picture Version (whole-object, cropped-object) mixed measures ANOVA on mean percentage increase in area showed that, like in Experiment 1, prior test knowledge attenuated boundary extension, F(1, 69) = 17.43 p<. 001, the presence of a cropped object did not affect overall boundary memory, F < 1, and there was no interaction, F < 1.

Figure 5.

Results of Experiment 2. Top: Mean percentage change in area for the whole-object pictures and cropped-object pictures in both conditions. Bottom: Mean percentage change in border position from the centre of the picture to the cropped border, the border opposite the cropping, the border adjacent to the cropping moving clockwise (adjacent C), and the border adjacent to the cropping moving counterclockwise (adjacent CC) for cropped-object pictures in both conditions. Error bars show the 95% confidence interval for each mean.

Our main question was whether observers remembered the cropped side but extended the others. The answer is no. The change in border position relative to the centre of the picture is shown in Figure 5 (bottom) for the cropped border, the opposite border, the adjacent-clockwise border (adjacent C, moving clockwise from the cropped border), and the adjacent-counterclockwise border (adjacent CC, moving counterclockwise from the cropped border) in each condition. In the test-naïve condition, the cropped and adjacent C borders were reliably shifted outward. In the test-informed condition, the cropped border was the only boundary that was reliably moved outward. A 4 × 2 Border (cropped, opposite, adjacent C, adjacent CC) × Test Instruction (test-naïve, test-informed) mixed measures ANOVA revealed an effect of border, F(3, 207) = 96.69, p < .001, an effect of instructions, F(1, 69) = 13.41, p < .001, and an interaction between the pattern of border adjustments and test instructions, F(3, 207) = 19.44, p < .001. Inspection of Figure 5 (bottom) suggests the interaction was probably driven by the cropped border being the only border to yield boundary extension in the test-informed condition.

Did the cropped border yield the greatest amount of extension because participants moved it out very far on only a small number of trials? No. Participants correctly remembered that the object was cropped on only 25% of the trials in the test-naïve condition and 62.5% of the trials in the test-informed condition. Extension of the cropped border cannot be attributed simply to object completion. Participants moved the border out to complete the object on 75% of trials in the test-naïve condition and 37.5% of trials in the test-informed condition. However, on 98.7% of these trials in the test-naïve condition and 92% of these trials in the test-informed condition, participants moved the border out to show background beyond the object.

In sum, participants were poor at remembering the cropped relation of the object and boundary in both conditions. Boundary extension (increased area) was limited to one or two sides in this task. Surprisingly, for the test-informed participants, this was always the cropped side, although eye tracking in Experiment 1 showed that test-informed participants recognize the importance of the cropped relation for remembering boundary placement.

EXPERIMENT 3

The purpose of Experiment 3 was to replicate Experiment 2 under conditions in which the position of the test boundaries was clearly incorrect, forcing participants to move all four borders to reconstruct the remembered view. On half the test trials, the borders were set very close together (revealing very little of the scene; small-aperture test trials), and on half the trials they were set very far apart (revealing much more of the scene than had been shown before; large-aperture test trials). Figure 6 shows a stimulus view in the top panel and an example of the two possible test views for that picture: (a) small-aperture test view (middle picture), and (b) large-aperture test view (bottom picture). By requiring reconstruction of all the boundaries, we thought we might get a better understanding of the remembered space on all sides of the view.

Figure 6.

An example of a stimulus view (top), a “small-aperture” test view (middle), and a “large-aperture” test view (bottom) for the “bananas on rocks” scene.

Method

Participants

Participants were 72 (44 female) University of Delaware undergraduates, fulfilling a requirement for an introductory psychology course (n = 36 in each condition).

Stimuli and apparatus

The stimuli and apparatus were the same as those in Experiment 2.

Design and procedure

The presentation sequence was identical to that in Experiment 2. Only the test differed, and test-informed instructions were changed accordingly. They were informed that the views at test would be different from those during presentation because the borders would be moved in to show less of the scene or moved out to show more of the scene (samples of each were shown). Thus prior to seeing the presentation sequence they knew that at test their task would be to recreate the original view by moving the borders. On half the trials, the pictures’ borders were set very close together, revealing little of the scene (a 273 × 182-pixel region in the centre of the picture; small-aperture trials), and on half they were set far apart, revealing more of the scene than before (a 820 × 547-pixel region of the picture; large-aperture trials). Trial types were randomly mixed with the constraint that no more than two of a kind could appear consecutively. Figure 6 shows a stimulus view (top panel), a small-aperture test trial (middle panel), and a large-aperture test trial (bottom panel). Test stimuli were presented in the same order as that during presentation.

On each trial, following border adjustment, participants indicated their confidence using the same rating scale as that in the previous experiments.

Measurement

The measurement was the same as that in Experiment 2. As in Experiment 2, it was decided a priori that any participant whose mean area remembered was 3 standard deviations or greater from the group mean would be excluded from analysis.

Results and discussion

There were no participants excluded from the analysis. Participants reported that they didn’t remember a picture (DRP) on 0.7% of the trials in the test-naïve condition and 0.6% of the trials in the test-informed condition. Mean confidence ratings were 2.1 (SD = 0.4) in the test-naïve condition and 2.3 (SD = 0.4) in the test-informed condition.

When participants were faced with boundaries at test that were clearly misplaced (inward or outward) they reconstructed the views, increasing the area (boundary extension) and doing so by shifting all four boundaries outward in both conditions. The mean percentage change in area for both types of pictures in each condition is shown in Figure 7 (top).

Figure 7.

Results of Experiment 3. Top: Mean percentage change in area for the whole-object pictures and cropped-object pictures in both conditions. Bottom: Mean percentage change in border position from the centre of the picture to the cropped border, the border opposite the cropping, the border adjacent to the cropping moving clockwise (adjacent C), and the border adjacent to the cropping moving counterclockwise (adjacent CC) for cropped-object pictures in both conditions. Error bars show the 95% confidence interval for each mean.

A 2 × 2 Test Instruction (test-naïve, test-informed) × Picture Version (whole-object, cropped-object) mixed measures ANOVA revealed that prior test knowledge attenuated boundary extension, F(1, 70) = 7.87, p = .006, cropped-object pictures yielded 4.6% less boundary extension than whole-object pictures, F(1, 70) = 6.45, p = .013, and there was no interaction, F < 1.

Figure 7 (bottom) shows the mean percentage change in border placement for the cropped border and the other three borders in both conditions. In all cases, all borders were reliably shifted outward. A 4 × 2 Border (cropped, opposite, adjacent C, adjacent CC) × Test Instruction (test-naïve, test-informed) mixed measures ANOVA revealed an effect of border, F(3, 210) = 19.22, p < .001, an effect of instructions, F(1, 70) = 7.34, p = .008, and an interaction between the two, F(3, 210) = 4.50, p = .004. Inspection of the figure suggests that the large displacement outward of the cropped border in the test-naïve condition drove the interaction.

As in Experiment 2, object completion alone cannot account for the cropped border being moved outward. Participants remembered a complete object on 87.4% of the trials in the test-naïve condition and 50.9% of the trials in the test-informed condition. On 96% of these trials in each condition, participants did not just complete the object, but instead moved the border out to show more background beyond the completed object. We analysed this separately for both small-aperture and large-aperture trials, and similar results were obtained across both types of trials in both the test-naïve and test-informed conditions.

Border adjustment and test views

Similar to other studies that used the border adjustment procedure (Dickinson & Intraub, 2009; Gagnier et al., 2011; Intraub et al., 2006), boundary extension was affected by test view type. On large-aperture trials, the mean area increase was 58.7% (SD = 33.1) and 33.7% (SD = 32.5) in the test-naïve and test-informed conditions, respectively; in both cases the increase was significant: t(35) = 10.65, p < .001, and t(35) = 6.24, p < .001, respectively. On small-aperture=trials, the mean area increase was 35.5% (SD = 29.0) and 25.0% (SD = 30.2) in the test-naïve condition and test-informed conditions, respectively; increase was significant in both cases: t(35) = 7.55, p < .001, and t(35) = 4.98, p < .001, respectively. A 2 × 2 Aperture Size (small, large) × Test Instruction (test naïve, test informed) mixed measures ANOVA showed that large-aperture trials yielded greater boundary extension than small-aperture trials, F(1, 70) = 20.44, p < .001.

In sum, when participants were forced to move all the borders to recreate the remembered view, boundary extension occurred in both conditions for both types of pictures. Cropped-object pictures yielded less boundary extension, but the decrease was small in magnitude (4.6%).

Unlike Experiment 2, participants shifted all four borders outward in all conditions.

EXPERIMENT 4

The purpose of Experiment 4 was to replicate Experiment 2 with eye tracking to address one remaining question. Is there a relationship between how much time participants spend fixating the cropped region and whether or not the cropped relation was remembered? If high-acuity visual information is important for remembering the cropped relation, then we should find a positive relationship between fixation activity and memory. The key difference between this eye tracking experiment (with border adjustment) and Experiment 1 (rating task) is simply the type of test. An important similarity is that in both cases the test views were identical to the stimulus views.

Method

Participants

Participants were 112 (72 female) University of Delaware undergraduates fulfilling a requirement for an introductory psychology course (n = 56 in each condition).

Stimuli

The stimuli were the same as those in Experiments 2 and 3.

Apparatus

The apparatus was the same as that in Experiment 1.

Design and procedure

The presentation sequence was the same as that in Experiment 1, and the instructions and test procedure were the same as those in Experiment 2 (test views were identical to stimulus views).

Defining regions of interest

The regions of interest were the same as those in Experiment 1.

Measurement

The measurement was the same as that in Experiments 2 and 3. As in Experiments 2 and 3, it was decided a priori that any participant whose mean area remembered was 3 standard deviations or greater from the group mean would be excluded from analysis.

Results and discussion

Border adjustment data

There were no participants excluded from the analysis. Participants reported that they didn’t remember a picture (DRP) on 0.1% of the trials in the test-naïve condition and 0.5% of the trials in the test-informed condition. Mean confidence ratings were 2.1 (SD = 0.3) in the test-naïve condition and 2.0 (SD = 0.3) in the test-informed condition.

The mean percentage change in area in each condition for both types of pictures is shown in Figure 8 (top). As shown in the figure, boundary extension occurred in all cases. A 2 × 2 Test Instruction (test-naïve, test-informed) × Picture Version (whole-object, cropped-object) × mixed measures ANOVA revealed that prior test knowledge attenuated boundary extension, F(1, 110) = 36.67, p < .001. As in Experiments 1 and 2, memory was not better for cropped-object pictures, F(1, 110) = 1.66, p = .20. There was a significant Test Instruction × Picture Version interaction, F(1, 110) = 4.69, p = .033. The interaction may be driven by a tendency toward greater boundary extension for the cropped-object pictures in the test-naïve condition.

Figure 8.

Results of Experiment 4. Top: Mean percentage change in for area for the whole-object pictures and cropped-object pictures in both conditions. Bottom: Mean percentage change in border position from the centre of the picture to the cropped border, the border opposite the cropping, the border adjacent to the cropping moving clockwise (adjacent C), and the border adjacent to the cropping moving counterclockwise (adjacent CC) for cropped-object pictures in both conditions. Error bars show the 95% confidence interval for each mean.

Figure 8 (bottom) shows the mean percentage change in border placement for the cropped border and the other three borders in both conditions. As can be seen in the figure, the cropped side generated the greatest amount of boundary extension in both conditions and was the only border to elicit boundary extension in the test-informed condition. A 4 × 2 Border (cropped, opposite, adjacent C, adjacent CC) × Test Instruction (test-naïve, test-informed) mixed measures ANOVA revealed an effect of border, F(3, 330) = 121.55, p < .001, a decrease in boundary extension in the test-informed condition, F(1, 110) = 28.16, p < .001, and a significant interaction, F(3, 330) = 15.67, p < .001. As in Experiment 2, the fact that the cropped border was the only border to elicit boundary extension in the test-informed condition probably drove the interaction.

As in Experiments 2 and 3, object completion alone cannot account for participants moving the cropped border outward. Test-naïve participants remembered a complete object on 70.5% of the trials, and test-informed observers remembered a complete object on 39.9% of the trials. When test-naïve participants completed the object, they moved the border out to include some background beyond the object’s edge on 98.5% of these trials; test-informed participants did this on 92.2% of these trials.

Oculomotor behaviour during study

We conducted the same analyses as those in Experiment 1. Fixation activity in the boundary region for all pictures is shown in Table 6 (top). As shown in the table, test-naïve participants fixated the boundary region, but test-informed participants fixated it sooner, longer, and more frequently. For example, test-naïve observers spent 18.1% of their time fixating the boundary region, whereas test-informed observers spent 25.5% of their time in this region. Next we examined fixation activity in the boundary region for whole-object and cropped-object pictures separately. As shown in Table 6 (middle), the same patterns were observed in these analyses.

Table 6.

Oculomotor behavior in the boundary region during study (Experiment 4, N = 112)

Oculomotor behavior in boundary region	Test-naïve	Test-informed	t(110)	Sig.	p
All pictures
Percent of time fixating	18.1	25.5	4.4	***	<.001
Percent of fixations	18.2	25.6	4.7	***	<.001
Index of first fixation	10.9	8.4	4.9	***	<.001
Latency of first fixation (ms)	3932	2896	4.1	***	<.001
Whole-object pictures
Percent of time fixating	17.6	25.2	4.0	***	<.001
Percent of fixations	17.8	25.8	4.5	***	<.001
Index of first fixation	10.4	8.5	3.1	**	.002
Latency of first fixation (ms)	3696	2971	2.6	*	.011
Cropped-object pictures
Percent of time fixating	18.5	25.9	4.2	***	<.001
Percent of fixations	18.7	25.5	4.2	***	<.001
Index of first fixation	11.2	8.2	4.9	***	<.001
Latency of first fixation (ms)	4116	2791	4.4	***	<.001
Boundary region minus cropped region
Percent of time fixating	13.3	15.6	1.7	n.s.	.084
Percent of fixations	13.9	15.8	1.5	n.s.	.129
Index of first fixation	13.2	13.1	0.2	n.s.	.884
Latency of first fixation (ms)	4902	4696	0.6	n.s.	.527

Note: Sig. = Significance; n.s. = not significant.

^*p<.05;

^**p<.01;

^***p<.001.

Because the boundary region includes the cropped region, we wanted to determine whether the increase in activity in this region was limited to activity in the cropped portion only or reflected increased activity to other parts of the boundary region. To answer this, we analysed fixation activity to the boundary region after subtracting the cropped region. As shown in Table 6 (bottom), fixation activity no longer differed across instruction groups. This shows that test-informed observers’ increased fixations to the boundary region were driven solely by fixations to the location where the boundary cropped the object.

Next, we report fixations to the cropped region. Table 7 shows oculomotor behaviour during study in the cropped region for the cropped-object pictures. As shown in the table, both groups fixated the cropped region, but the test-informed group fixated it sooner, longer, and more frequently. We also contrasted the fixation activity in the cropped region with that in the same region in the whole-object pictures. As shown in Table 8, there were no differences in fixation activity in the test-naive condition. However, test-informed observers spent longer and devoted a greater percentage of fixations to this region of the cropped-object pictures than to the whole-object pictures, suggesting that they deemed this region important to study to succeed on the upcoming test.

Table 7.

Oculomotor behavior in the boundary region during study (Experiment 4, N = 112)

Oculomotor behavior in cropped region	Test-naïve	Test-informed	t(110)	Sig.	p
Cropped-object pictures
Percent of time fixating	5.2	10.3	5.6	***	<.001
Percent of fixations	4.8	9.7	6.5	***	<.001
Index of first fixation	14.9	11.8	2.8	**	.006
Latency of first fixation (ms)	5329	4167	2.6	*	.011

Note: Sig. = Significance.

^*p<.05;

^**p<.01;

^***p<.001.

Table 8.

Comparisons of oculomotor behavior for the “cropped region” in cropped-object and whole- object pictures (Experiment 4, N =112)

Oculomotor behavior in cropped region	Whole-object pictures	Cropped-object pictures	t(110)	Sig.	p
Test-naïve
Percent of time fixating	4.6	5.2	.98	n.s.	.33
Percent of fixations	4.4	4.8	.81	n.s.	.42
Index of first fixation	16.0	14.9	.76	n.s.	.45
Latency of first fixation (ms)	5701	5329	.75	n.s.	.46
Test-informed
Percent of time fixating	7.1	10.3	4.4	***	<.001
Percent of fixations	7.0	9.7	4.3	***	<.001
Index of first fixation	13.4	11.8	1.8	n.s.	.08
Latency of first fixation (ms)	4683	4167	1.5	n.s.	.14

Note: Sig. = Significance; n.s. = not significant.

^*p<.05;

^**p<.01;

^***p<.001.

Finally, we wanted to determine whether fixation activity was greater when participants correctly remembered that the object was cropped (irrespective of whether they remembered the exact location of the boundary) than when they did not. Participants’ data were included if they (a) made at least one fixation on the cropped area and (b) had at least two pictures for which they remembered the cropping and two pictures for which they moved the borders to complete the object. As shown in Table 9, no differences were obtained for either test-naïve or test-informed participants. Because pictures have idiosyncratic characteristics, we decided to ask the same question, but this time within the context of a picture analysis. Consistent with the previous analysis, for test-naïve participants we found that for only one picture of the 12 (the car), participants who remembered the cropping made a significantly greater percentage of fixations in the cropped region than participants who did not remember it (7.9% vs. 4.3%), t(15) = 2.2, p = .04. For test-informed participants, this difference reached significance for only 4 of the 12 pictures: the tyre (14.8% vs. 9.2%), t(24) = 2.2, p = .04; the car (11.5% vs. 4.3%), t(26) = 2.7, p = .01; the bowl of cereal (10.9% vs. 4.3%), t(17) = 2.2, p = .04; and the cone (17.9% vs. 7.4%), t(24) = 3.4, p = .002.

Table 9.

Oculomotor behavior in the cropped region during study (Experiment 4, N =112)

Test-naïve	Remembered cropping	Did not remember cropping	t(37)	Sig.	p
Percent of time fixating	8.8	9.4	0.3	n.s.	0.8
Percent of fixations	7.6	8.2	0.4	n.s.	0.7
Index of first fixation	10.2	11.8	1.0	n.s.	0.3
Latency of first fixation (ms)	3266	4196	1.4	n.s.	0.2
Test-informed	Remembered cropping	Did not remember cropping	t(43)	Sig.	p
Percent of time fixating	12.1	10.4	1.0	n.s.	0.3
Percent of fixations	11.9	9.6	1.6	n.s.	0.1
Index of first fixation	11.6	12.3	0.4	n.s.	0.7
Latency of first fixation (ms)	4221	4627	0.5	n.s.	0.6

Note: Sig. = Significance; n.s. = not significant.

^*p<.05;

^**p<.01;

^***p<.001.

Oculomotor behaviour during test

Fixation activity during test is shown in Table 10. As shown in the table, all participants fixated the critical scene regions at test, and there were no differences in eye movements across the two conditions. This is probably due to the fact that the border adjustment test forced participants to fixate the boundary and cropped regions and thus probably led to both groups using similar strategies at test.

Table 10.

Oculomotor behavior in boundary and cropped regions during test (Experiment 4, N =112)

Oculomotor behavior in boundary region	Test-naïve	Test-informed	t(110)	Sig.	p
Whole-object pictures
Percent of time fixating	.41.1	43.1	0.9	n.s.	.373
Percent of fixations	36.9	37.7	0.4	n.s.	.663
Index of first fixation	5.3	5.2	0.4	n.s.	.657
Latency of first fixation (ms)	1441	1359	0.8	n.s.	.451
Cropped-object pictures
Percent of time fixating	45.6	45.6	0.0	n.s.	.978
Percent of fixations	39.3	38.4	0.6	n.s.	.575
Index of first fixation	4.6	4.7	0.2	n.s.	.813
Latency of first fixation (ms)	1144	1136	0.1	n.s.	.937
Boundary region minus cropped region (cropped-object pictures)
Percent of time fixating	25.3	25.6	0.2	n.s.	.808
Percent of fixations	22.5	23.0	0.4	n.s.	.713
Index of first fixation	10.4	10.3	0.3	n.s.	.782
Latency of first fixation (ms)	3491	3346	0.5	n.s.	.602
Oculomotor behavior in cropped region	Test-naïve	Test-informed	t(110)	Sig.	p
Percent of time fixating	20.3	19.8	0.3	n.s.	.768
Percent of fixations	16.8	15.4	1.3	n.s.	.186
Index of first fixation	5.8	5.7	0.1	n.s.	.928
Latency of first fixation (ms)	1536	1533	0.0	n.s.	.989

Note: Sig. = Significance; n.s. = not significant.

^*p<.05;

^**p<.01;

^***p<.001.

In sum, as in Experiment 2, the cropped border generated the greatest amount of boundary extension in the test-naive condition and was the only border to elicit boundary extension in the test-informed condition. This occurred in spite of the fact that participants fixated the cropped region and that test-informed participants considered it to be important (showing increased fixation activity to that region). Surprisingly, the ability to recognize that an object was cropped by a boundary was not associated with greater fixation activity. The persistence of boundary extension on the cropped side in the test-informed condition and the failure to find a compelling relationship between fixation activity and memory for the cropping (irrespective of boundary placement) raise interesting questions about the extent to which the high-acuity content gleaned from eye fixations informs memory for the spatial expanse of the view shown in a photograph. We discuss the implications of these observations in the next section.

GENERAL DISCUSSION

We set out to determine whether boundary extension reflects a failure to fixate near the edges of a picture. What we found is that boundary extension occurred in spite of multiple fixations to the boundary region, including the place where the boundary cropped the main object in the scene. Whether participants did or did not know in advance that boundary memory would be tested, they fixated these regions. Test-informed participants made use of their knowledge and fixated these regions sooner, longer, and more frequently. Top-down goals clearly impacted their oculomotor behaviour (Buswell, 1935; Yarbus, 1967; see Tatler, 2009, Eye Guidance in Natural Scenes. A Special Issue of Visual Cognition, for a review), but surprisingly, this was not sufficient to allow them to remember a clear marker of boundary placement. They frequently failed to recognize the identical cropped view that they had recently studied during presentation. Boundary ratings (Experiment 1) and border adjustment (Experiments 2–4) showed that in spite of participants’ attempts to remember the boundaries, they tended to fail and instead remembered having seen more of the scene than was shown.

Prior test information did, however, affect boundary extension. It led to a reduction in the size of the error, thus replicating Intraub and Bodamer’s (1993) primary observation. This might suggest that the reduction was directly caused by the increased fixation activity that provided high-acuity visual information near the boundary. Cropped-object trials provided an opportunity to test this hypothesis, because on these trials we could assess eye movements to a specific marker of boundary placement both during study and at test. If multiple eye fixations to the place where the boundary actually touched and slightly cropped the main object led to good memory for that boundary, then we would expect to find: (a) less boundary extension for cropped-object pictures than for whole-object pictures, (b) better memory for boundary placement on the cropped side of the cropped-object pictures than for the other sides, and (c) greater fixation activity to the cropped region on those trials in which they remembered that the object was cropped than on those in which they did not.

None of these expectations were borne out. When identical views were shown at test, boundary extension (the remembered area of the view) did not differ between whole-object and cropped-object pictures when participants rated the same view at test (Experiment 1) or when they could accept the identical view or “correct” it by adjusting the borders (Experiments 2 and 4). Only in Experiment 3 did border adjustment reveal slightly less boundary extension on cropped-object trials (a 4% decrease in area). This was the only border-adjustment experiment in which the test view was not identical to the studied view; borders were shifted either outward or inward, requiring the participant to move all the boundaries. As we discuss next, we could find no evidence that this slight difference was due to better memory for the cropped relation.

Most often, participants failed to recognize (Experiments 2 and 4) or recall (Experiment 3) that there was a cropped relation between a boundary and the main object. They tended to shift the boundary outward so that not only was the object completed, but more background space was interposed between the edge of the now complete object and the boundary. It is interesting to note that on whole-object trials, on average at least 2–3 sides were reliably shifted outward, but in the case of the cropped-object pictures, the cropped side was the only side that was reliably shifted outward. One possibility is that at test, participants’ attention was rapidly drawn to the cropped edge, and while fixating that area they found that it was difficult to remember whether or not the object had been cropped. That is, they knew this was a diagnostic region, but their representation did not allow them to determine boundary placement. This may have resulted in less scrutiny of the other sides when participants adjusted them. Indeed, when test-informed participants were forced to adjust each side (Experiment 3), all four borders of the cropped-object pictures were reliably shifted outward (see Figure 7). These results do not suggest that it was the high-acuity information obtained during scrutiny of the cropped region that led to a reduction in boundary extension for test-informed participants.

One additional analysis was conducted to seek evidence linking fixation activity to memory for the cropped region. Here we compared eye data between the (relatively few) trials on which participants successfully remembered the cropped relation and those trials on which they did not. We did this for each of four dependent measures related to the cropped region: (a) percentage of the time spent fixating it, (b) percentage of fixations, (c) the index of first fixation, and (d) the latency to the first fixation to the region (i.e., how soon the region was fixated in a trial). We found no differences in oculomotor behaviour on any of these variables in either instruction condition. We also did a similar set of comparisons in the context of a picture analysis so we could hold picture characteristics constant, but we found no difference on any of the oculomotor variables for the majority of the 12 pictures (11 in the test-naïve condition, and 8 in the test-informed condition).

This is surprising in that we generally expect to see a relationship between fixations and memory. Many experiments have demonstrated that memory for object detail in multiobject scenes is improved when the object is fixated (e.g., Henderson & Hollingworth, 1999; Hollingworth, 2004; Hollingworth & Henderson, 2002; Irwin & Zelinsky, 2002; Zelinsky & Loschky, 2005). However, with respect to remembering a detail associated with the expanse of a view, the present research provided no evidence for a direct relation between number of fixations and memory. Although prior test-knowledge apparently caused participants to try to remember the location of the boundaries, including a focus on that part of the boundary that cropped the object, they had great difficulty in remembering that critical detail. Perhaps, participants’ global attention to the picture and the relation of the object to the surrounding space as a whole led to the reduction in boundary extension.

In considering why multiple fixations to the boundary region and to the cropped region would have such a limited impact on memory, we suggest that there may be two distinct goals of the systems that underlie scene perception. One goal is to obtain visual information about specific objects and their spatial relation to one another—these are characteristics of things in the world. Here, memory for fixated visual information is beneficial. For example, change detection in a scene improves when fixations are made to the to-be-changed objects (Henderson & Hollingworth, 1999; Hollingworth, 2004; Hollingworth & Henderson, 2002). The other goal is to understand the surrounding world that a given view only partially reveals. The spurious boundaries of a given view are not characteristics of “things in the world”, but accidental boundaries of the current view. To achieve the second goal it would be advantageous to ignore the spurious boundaries of successive views—fleeting boundaries that are not a part of the world.

One way to think about this second goal is to step away from a strictly visual account of scene representation. As an alternative, consider the multi-source model (Intraub, 2010, 2012; Intraub & Dickinson, 2008) of scene representation, in which a viewer-centred spatial framework serves as the scaffolding that supports multiple sources of top-down information in addition to the visual sensory information. Scenes can be conceptually identified (classified) within a single fixation (e.g., Potter, 1976; Thorpe et al., 1996), providing expectations about the surrounding context, and object-to-context associations (Bar, 2004) can refine the nature of those surroundings further. Just beyond the edges of the view, amodal continuation of objects (Kanizsa, 1979) and surfaces (Kellman et al., 1998) is highly constrained by the visual input, and, when remembered, some of this information is attributed to having been seen, even moments later (Dickinson & Intraub, 2008; Intraub & Dickinson, 2008).

This memory beyond the edges may facilitate integration of successive views into the viewer-centred framework surrounding space. Indeed, it has been shown that boundary extension can serve to prime visual perception of surrounding layout (Gottesman, 2011). Additionally, boundary extension seems to be specifically tied to the boundaries of a view, not other types of boundaries within the view (e.g., object boundaries, see Gottesman & Intraub, 2003). The results of the current experiments support the possibility that there is something “special” about view boundaries, in that increased fixation to the boundary regions (including the cropped region) was not sufficient to support good recognition memory for the expanse of the view. This failure is consistent with the idea that the multisource representation is graded in nature, without detailed visual encoding of view boundaries.

The results do not support the idea that memory for specific fixated visual details in the boundary region is what led to a reduction in boundary extension for test-informed participants. Instead this improvement may be related to how people attended to the scope of the view more globally (e.g., the spatial relationship of the size of the object to the size of the background region). By increasing attention to the background relative to the object, this may change the threshold for how much of the nonvisual (top-down) information just beyond the boundaries is later attributed to vision. Some evidence for this type of trade-off was obtained in an experiment in which attention was manipulated by having participants either ignore or take part in a demanding visual search task superimposed on briefly presented pictures (Intraub, Daniels, Horowitz, & Wolfe, 2008). In both conditions, participants knew that a boundary rating would be required (i.e., they were test-informed). When attention was divided, and memory for visual detail probably reduced, the pictures were remembered as having shown even more of the surrounding world than when attention was not divided.

Other support for a model of scene perception that is fundamentally spatial is that boundary extension occurs following haptic exploration of scenes when participants are blindfolded, or when a woman who was deaf and blind since early life was tested (Intraub, 2004; also see Intraub, 2010). Similar to participants’ difficultly in recognizing boundary placement on the cropped side of the pictures in the current research, Intraub and Delaney (2010) found that when blindfolded sighted participants who received the standard boundary extension instruction explored a scene in which a main object physically touched one of the boundaries, they persisted in reliably moving that border out at test, failing to remember a point of contact that they had felt with their hands minutes earlier.

In conclusion, the present research raises some important questions about the relationship between eye movements and scene representation. If scene representation captured only the visual information in the picture, then our memory tasks should have been trivial in the test-informed conditions. Participants in those conditions knew what would be tested, studied critical regions, fixated those regions again at test, and yet failed to recognize the identical view that they had studied minutes earlier. These results are difficult to explain in context of a visual-representation-only conceptualization of scene representation. Although scene perception is usually studied by presenting pictures as surrogates for the world, it may be important to remember that scenes in the world do not exist only in front of the observer, but also surrounding that observer. Even when looking at a picture, scene representation may well represent the likely surrounding world (i.e., the scene) that the picture only partially reveals.

APPENDIX A

Whole-object versions of stimuli

APPENDIX B

List of stimuli for all experiments

Picture name	Description	Cropped border
Backpack	Backpack on leaves	Bottom
Bananas	Bananas on rocks	Right
Basketball	Basketball on gym floor	Top
Car	Car in parking lot	Left
Cereal	Bowl of cereal on table	Left
Light bulb	Light bulb on grass	Bottom
Racquet	Racquetball racquet on concrete	Right
Tape	Scotch tape on carpet	Left
Tire	Tire against wall	Bottom
Roll of paper	Toilet paper roll on tile floor	Top
Tractor	Tractor by field	Right
Traffic cone	Traffic cone on asphalt	Top