When less is more: Line-drawings lead to greater boundary extension than color photographs

Abstract

Is boundary extension (false memory beyond the edges of the view; Intraub & Richardson, 1989) determined solely by the schematic structure of the view or does the quality of the pictorial information impact this error? To examine this color photograph or line-drawing versions of 12 multi-object scenes (Experiment 1: N=64) and 16 single-object scenes (Experiment 2: N=64) were presented for 14-s each. At test, the same pictures were each rated as being the “same”, “closer-up” or “farther away” (5-pt scale). Although the layout, the scope of the view, the distance of the main objects to the edges, the background space and the gist of the scenes were held constant, line-drawings yielded greater boundary extension than did their photographic counterparts for multi-object (Experiment 1) and single-object (Experiment 2) scenes. Results are discussed in the context of the multisource model and its implications for the study of scene perception and memory.

People tend to remember seeing beyond the boundaries of a view (boundary extension; Intraub & Richardson, 1989). What affects the size of this anticipatory representation? Numerous experiments have demonstrated that boundary extension is affected by the scope of a view, such that a close-up leads us to remember having seen more of the surrounding world than does a wider-angle view of the same scene (Bertamini, Jones, Spooner & Hecht, 2005; Intraub & Richardson, 1989; Intraub & Dickinson, 2008). Bertamini et al. (2005) argued that this difference is mediated by the distance between the main objects and the edges of the picture, rather than the scope of the background. However, Gagnier, Intraub, Oliva & Wolfe (2011) held the distance between the main objects and the edges of the picture constant but changed the vantage point of the camera so that more or less of the background surface could be seen. They found that views that showed less of the background surface led to greater boundary extension. The present research sought to gain greater insight into factors that affect boundary memory by presenting observers with either photographs or line-drawings of the same scenes.

This comparison was of interest because both the photographs and line-drawings would present the same viewpoint, relationship of objects to one another and to the boundaries, and the same general meaning (“gist”; see samples of both types of stimuli in Figure 1). The key difference between these versions is that the photographs provide a richer, more realistic visual representation that includes color, shading, high-resolution details and other spectral information. Would the absence of these factors affect boundary extension? If object layout and the relation of the object to the pictures’ boundaries (e.g., Bertamini et al., 2005) is the critical factor, then with object layout held constant, we would expect no difference in the size of the boundary error between photographs and their line-drawn counterparts. If boundary extension is affected by the availability of rich, specific details that might elicit more specific expectations about the surrounding layout, then the more realistic views provided by photographs would be expected to elicit greater boundary extension than would line drawings. Of course, a third possibility is that boundary extension might be greater for the line drawings than the photographs. If so, what characteristics of line drawings vs. photographs might support this outcome?

Figure 1.

The color photograph and outline drawing version of the “ballet class” stimulus from Experiment 1 (top row) and “man sitting” stimulus from Experiment 2 (bottom row). Note the photographs were shown in color.

Here, a recent multisource model of scene perception (Intraub & Dickinson, 2008; Intraub, 2010, 2012) raises a possibility. According to the multisource model, scene representation includes not only memory for visually presented information, but also memory for the amodal continuation of the scene’s background beyond the edges of the view (Kanizsa, 1979; Kellman, Yin & Shipley, 1998). When deciding at test how much of the remembered scene had actually been seen, the similarity between the remembered visual information from just inside the boundary and the remembered amodal information from just outside the boundary might be greater in the case of a simple line-drawing than in the case of a rich multidimensional photograph. If so, then participants may be likely to misattribute more of this amodal continuation to vision after viewing a line drawing than after viewing a photograph.

We report two experiments in which we sought to determine if boundary extension would differ between scenes that shared the same viewpoint, object layout, and gist, but differed in that the photographic version includes rich, realistic visual information whereas the line drawn version was a more pared down, schematic depiction. Would boundary extension remain the same, or would the photographs or line-drawings tend to be remembered as having presented a more expansive view of the world?

Experiment 1

Multi-object pictures were presented either as color photographs or line-drawings (a sample pair, the “ballet class” is shown in Figure 1, top row). At test, identical views were rated as being the same or as showing more or less of the world using the scale developed by Intraub & Richardson (1989). If an identical view is rated as being more close-up than before, this indicates boundary extension in memory for the studied view.

Method

Participants

Participants were 64 (36 female) undergraduates enrolled in the University of Delaware research pool (N=32 in each condition).

Stimuli

Stimuli were 12 color photographs of multiple-object scenes and line-drawings traced from the photographs. An example is shown in Figure 1, top row. All pictures were 550 × 458 pixels and were presented in the center of a 1024 × 768 gray background.

Apparatus

Pictures were presented on a 21″ Dell monitor (screen resolution: 1024 × 768 × 32 bits of color) using a program written in C (for more details see Gagnier, Intraub, Oliva & Wolfe, 2011). Participants were seated in two rows with a viewing distance of approximately 150 cm and 210 cm in the front and back rows respectively; visual angle subtended by the stimulus view was approximately 8°× 7° and 6° × 5°, respectively.

Design and Procedure

Participants were randomly assigned to photograph or line-drawing conditions and were run in groups of five or fewer. In preparation for a memory test, participants were instructed to remember each picture in as much detail as possible, paying equal attention to the objects and the background. The 12 stimuli were presented in the same order but with the images mirror reversed for half the participants in each condition so that any left-right idiosyncrasies in composition that might affect memory would be counterbalanced. Participants fixated a yellow fixation cross on a gray field. When the sequence was initiated the first picture appeared (14 s), followed by a visual noise mask (500 ms) and a repetition of the fixation field (500 ms) to signify onset of the next picture. This cycle repeated until all pictures were shown. All participants observed a 3-picture practice sequence in advance to familiarize them with the timing.

Immediately following presentation they received a 3-minute test instruction and test. The same 12 pictures were presented in the same order and participants were allotted 20 s to rate each one as being, “much closer-up” (−2), “slightly closer-up” (−1), “the same” (0), “slightly farther away” (1) or “much farther away” (2) than the studied view, and to rate their confidence as, “sure” (3), “pretty sure” (2), “not sure” (1). They indicated these responses on a scantron in a dim room with the experimenter present. An alternative option of “don’t remember picture” was also included. A visual noise mask (500 ms) and the fixation field (500 ms) appeared after each picture to indicate onset of the next.

Results and Discussion

The mean boundary rating and .95 confidence interval for each picture condition is presented in Figure 2; as may be seen in the figure, significant boundary extension occurred in both conditions¹. However, boundary extension was greater in the line-drawing condition, t(62) = 2.07, p<.05. Mean confidence rating was identical across conditions, in both case it was 2.2 (SD = .40) [2 = “pretty sure”]. Only two participants in each condition selected the “don’t remember picture (DRP)” option; the mean number of DRP selections did not differ across groups, [t(62)= .79, n.s.].

Figure 2.

Mean boundary ratings for the color photograph and outline drawing conditions in Experiment 1. Error bars indicate 95% confidence interval. Means significantly less than zero indicate boundary extension.

Clearly although object size and placement was the same, boundary extension was greater for line-drawing than photographs. However, in creating the line drawings, in some cases incidental background objects were eliminated because they were not understandable in outline form. We realized that this occasional elimination minimized background clutter. For example in Figure 1, top row, some of the reflected items in the background mirror were not included in the outline drawing. Thus, we wanted to determine if subtle differences in background content rather than picture type was responsible for the difference in boundary extension. In Experiment 2, this potential factor was eliminated.

Experiment 2

In Experiment 2 a new picture set was presented in which single objects were presented on a textured background; thus there were no incidental objects in the background that could be differentially represented across picture type. An example (“man seated in front of a brick wall”) is shown in Figure 1 (bottom row). We sought to determine if the difference observed in Experiment 1 would be replicated or eliminated.

Method

Participants

Participants were 64 (46 female) undergraduates from the same research pool as in Experiment 1 (N=32 in each condition).

Stimuli

The stimuli were 16 color photographs of single-object scenes or line drawings made by tracing the scene (from Intraub, Gottesman & Bills, 1998; an example is shown in Figure 1, bottom row). Pictures were 547 × 365 pixels, subtending a visual angle of approximately 8° × 6° and 6° × 4° for the first and second row, respectively. As in Experiment 1, each was presented in the center of a 1024 × 768 gray field.

Apparatus, Design and Procedure

The apparatus, design and procedure were the same as in Experiment 1, except that set size was 16 instead of 12 and because each scene only contained a single, central main object (so there were no major left-right differences in composition), pictures were never mirror reversed.

Results and Discussion

The mean boundary rating and .95 confidence interval for each picture condition is presented in Figure 3; as may be seen in the figure, significant boundary extension occurred in both conditions. Again the mean rating was greater for line-drawings than photographs, t(62) = 2.48, p<.02. Participants’ confidence did not differ across groups; the mean confidence rating was 2.2 (SD = 0.40) in both conditions [2 = “pretty sure”]. One participant in the photograph condition and four participants in the outline condition selected “don’t remember picture”; the mean number of selections did not differ across groups, [t(62)= .51, n.s.].

Figure 3.

Mean boundary ratings for the color photograph and outline drawing conditions in Experiment 2. Error bars indicate 95% confidence interval. Means significantly less than zero indicate boundary extension.

General Discussion

The photographs and line-drawings in these experiments shared many important characteristics: the viewpoint taken, the layout of the objects, the objects’ relation to the boundaries of the view, and the general meaning (“gist”) of the view. Although test views were always identical to the stimulus views, participants tended to rate them as “too close-up,” indicating that the original view was remembered as showing more of the scene. However, in both experiments, boundary extension was greater for line-drawings than photographs. This difference could not be attributed to the occasional loss of smaller background objects in the line drawings (as in Experiment 1) because the same outcome was obtained for single-object pictures in which the background was simply a surface (Experiment 2). Clearly object layout was not the sole determinant of boundary extension. Whether or not more realistic pictures (photographs) helped to elicit greater expectations about surrounding space, picture realism did not translate into greater boundary extension. In remembering line-drawings, participants simply remembered having seen more of the world. Why?

In considering all three possible outcomes, as discussed in the introduction, a new model of scene representation termed the multisource model (Intraub, 2010; 2012), provided a rationale for why line drawings might elicit greater boundary extension. According to this view, line drawings and photographs are both expected to recruit scene processing. Any view (photograph or line-drawing) would be understood within a viewer-centered spatial framework that would support bottom-up information (vision), as well as top-down sources of information. The top-down sources include, amodal completion of objects cropped by a boundary (Kanizsa, 1979), amodal continuation of surfaces beyond the boundaries (Kellman, Yin & Shipley, 1998), object-to-context associations (Bar, 2004) and expectations based upon rapid scene classification (within 150 ms of onset: Greene & Oliva, 2009a, 2009b; Potter, 1976; Thorpe, Fize & Marlot, 1996; Schyns & Oliva, 1994). The idea is that we rapidly grasp both the continuation and broader context of a scene. Indeed boundary extension has been shown to occur across retention intervals as brief as a saccade (e.g., 42 ms; Dickinson & Intraub, 2008; Intraub & Dickinson, 2008).

If we think of scene perception in terms of the multisource framework, then not only characteristics of the visual information, but also characteristics of the top-down contributions to scene perception can impact boundary extension. As an example, consider a recent study in which amnesic patients (bilateral hippocampal lesions) who have normal visual perception, but great difficulty imagining surrounding space, took part in a variety of boundary extension tasks (Mullally, Intraub & Maguire, 2012). Paradoxically, they showed better memory for the boundaries than the matched controls. In the absence of any change to the visual content of the views, the lack of a coherent top-down spatial representation beyond the boundaries may limit how much surrounding space was attributed to vision. In the current research, we suggest that the similarity between visual (modal) information and the amodal continuation of that information just beyond the boundaries is greater for line drawings than for multidimensional photographs. Participants then misattribute more of the surrounding space to having been seen before (a source monitoring error; Johnson, Hashtroudi & Lindsay, 1993; Lindsay, 2008).

The same idea can also explain two other observations in the boundary extension literature. In one series of experiments, attention was divided by requiring participants to take part in a demanding search task while studying briefly presented pictures; instead of causing greater random error in memory, boundary extension was more expansive when attention was divided than when it was not divided (Intraub, Daniels, Horowitz and Wolfe, 2008). Divided attention was thought to compromise visual detail, thus increasing the similarity between the modal and amodal information across the boundary. In another experiment, when pictures were presented for either 250 ms or 4 s each at the same presentation rate, boundary extension was slightly greater for the 250-ms pictures (Intraub, Gottesman, Wiley & Zuk, 1996). In terms of the similarity hypothesis, here, a much briefer presentation would compromise the retention of visual detail, decreasing the difference between the modal and amodal information on either side of the boundary. In all three cases, blurring the line between modal and amodal information in memory increases the amount of amodal information that is attributed to having been seen.

An alternative explanation is that line drawings elicit greater expectations about surrounding space than do photographs, but as yet it is not clear why this would be. This essentially restates the finding rather than explaining it. In contrast, the notion of similarity between modal (visual) and amodal aspects of a multisource scene representation can explain the current findings as well as several other reports in the literature that are unrelated to line drawings (e.g., Intraub et al., 2008; Intraub et al, 1996; Mullally et al., 2012). We suggest that this framework provides a worthwhile alternative for thinking about scene perception. By focusing on space, it motivates questions about spatial memory across modalities (e.g., haptic boundary extension; Intraub, 2004) and by focusing the multiple sources of information that are rapidly available during scene perception, it motivates questions about other potential top-down factors, in addition to amodal perception, that might affect boundary extension, such as scene typicality (see Ehinger, Xiao, Torralba, & Oliva 2011). In conclusion, prior research demonstrated that various attributes of layout affect boundary extension; the current research shows that object layout alone does not determine the size of this anticipatory error.