Surface Information Loss in Comprehension

Abstract

Shortly after a sentence has been comprehended, information about its exact surface form (e.g., its word order) becomes less available. The present research demonstrated this phenomenon during the comprehension of nonverbal stimuli (picture stories). In Experiment 1, significantly more surface (left/right orientation) information was lost after comprehending several picture stories than just one; in Experiment 2, more was lost after comprehending an entire picture story than half of one. In Experiment 3, subjects segmented the picture stories into their constituents; in Experiment 4, significantly more surface information was lost after crossing these constituents’ boundaries than before. The present research also investigated why surface information is lost. Four explanations were considered: Surface information loss is the result of performing grammatical transformations (the linguistic hypothesis), exceeding short-term memory limitations (the memory limitations hypothesis), integrating information into gist (the integration hypothesis), shifting from building one substructure to initiating another (the processing shift hypothesis). The linguistic and memory limitations hypotheses were considered inadequate; the integration and the processing shift hypotheses were tested in the last set of experiments. In Experiment 5 (using nonverbal stimuli), the predictions made by the processing shift hypothesis were confirmed; in Experiment 6 (using verbal stimuli), these results were replicated. Other implications of the processing shift hypothesis concerning surface information loss are discussed.

A well-known phenomenon involved in language comprehension is this: Shortly after a passage is comprehended, information about the exact surface form of its sentences (e.g., their word order) becomes less available. By far the most cited demonstration of this phenomenon is Sachs’ (1967). Her subjects listened to a narrative story that included a sentence such as

1. He sent a letter about it to Galileo, the great Italian scientist.

After comprehending this sentence, subjects decided whether it or Sentence 2 was the sentence they just heard.

• 2A letter about it was sent to Galileo, the great Italian scientist.

If subjects were tested immediately after hearing the target sentence, their ability to discriminate between it and its reversed form was around 90%. However, if they were tested after comprehending only 80 additional syllables, performance fell to just above chance.

This phenomenon is well known partly because we experience it in our everyday lives and because it has been empirically demonstrated in numerous psychological laboratories.¹ The present research also examined this phenomenon but using narratives “told” completely without words; the stories were composed of professionally drawn pictures, like the sequence shown in Fig. 1.

Fig. 1.

Example sequence from an experimental story (From One Frog Too Many, copyright © 1975 by Mercer and Marianna Mayer. A Dial Books for Young Readers book. Reproduced by permission of E. P. Dutton, a division of New American Library.)

These picture stories provided a fertile ground for exploration. Scenic pictures have been a favorite stimulus for a variety of different information-processing paradigms, for example: tachistoscopic perception (Inntraub, 1979; Palmer, 1975), visual search (Biederman, Glass, & Stacy, 1973; Potter, 1975), eye movements (Friedman, 1979; G. Loftus, 1972), priming (Bruner, 1957; McKoon, 1981), long-term recognition (Nickerson, 1968), long-term recall (Goodman, 1980), long-term reconstruction (Mandler & Parker, 1976), cross-modal retention (E. Loftus & Palmer, 1974), sentence verification (Slobin, 1966), and categorization (Tversky & Hemenway, 1983). Yet, little is known about how these stimuli are comprehended when they compose a narrative sequence.

Two different studies by Baggett, however, have provided a spring-board for such investigations. In one (Baggett, 1975), subjects viewed sequences of four simple line drawings depicting everyday events (e.g., getting a haircut). When answering subsequent questions, subjects were facile at making needed inferences—much like “reading between the lines” (see also Jenkins, Wald, & Pittenger, 1977). In another study (Baggett, 1979), subjects recalled a story after either viewing it as a movie without dialogue or reading it as a text. Recall protocols from the two conditions were organized very similarly. These studies suggest that despite the difference in modality, processing picture stories may be like processing their language-based counterparts. One general goal of the present research was to further uncover such parallels.

The more specific goals of this research were first, demonstrating that surface information loss is not unique to language-based comprehension and second, investigating why surface information is lost. That we quickly forget the exact wording of an utterance is usually accepted as a matter of record. Over the years, the phenomenon has motivated considerably more demonstrations than explanations. Moreover, the few explanations submitted to empirical scrutiny have been chiefly those derived from linguistic theories. However, if the phenomenon could be demonstrated within a nonverbal domain, one could then look outside language-based hypotheses for its explication.

DEMONSTRATING THE PHENOMENON OF SURFACE INFORMATION LOSS

The stimuli used in the first set of experiments were four picture stories. Each comprised 24 pictures and each successive story was a sequel. These stories (Mayer, 1967, 1969; Mayer & Mayer, 1971, 1977) were perhaps intended as children's literature, but my college-level subjects also found them entertaining. Given their nonverbal nature, memory for their surface form could not be tested by rearranging word order. Instead, the type of surface information tested was each picture's original left/right orientation (cf. Bartlett, 1932; Bartlett, Gernsbacher, & Till, 1984; L. Cohen, 1977; Gernsbacher, 1980; Madigan & Rouse, 1974; Standing, Conezio, & Haber, 1970).

Left/right orientation seemed the ideal type of surface information to test because it is analogous to linguistic surface form in a very important way: Memory for both appears to be unrelated to memory for content or meaning. Just as it has been shown that memory for a sentence's word order is unrelated to memory for its meaning (Begg, 1971; Sachs, 1967, 1974), it has also been shown that memory for a picture's left/right orientation is unrelated to memory for its meaning (Kraft & Jenkins, 1977; Nickerson & Adams, 1979). Only in the rarest of situations does a picture's left/right orientation affect its content or meaning. Such is the problem faced by many preliterate children when discriminating the letter b from d; before they learn the arbitrary association, one orientation is just as meaningful as the reverse (Stein & Mandler, 1974, 1975). For adults, most novel pictures—that, of course, do not contain script—convey virtually the same message when displayed in one orientation or their mirror image. This is demonstrated in Fig. 2. The picture on top is semantically identical to the one on bottom; the orientation has simply been reversed.

Fig. 2.

Example picture from an experimental story displayed in one orientation (top row) and the reverse (bottom row). (Reprinted, with permission, from Mayer & Mayer, 1971.)

However, memory for the original orientation of a picture within a story is also analogous to memory for the linguistic form of a sentence in a way that is less ideal for the present (or any) study: Measuring memory for both can be confounded with response bias. With sentences, subjects can sometimes respond correctly on a subsequent test by reconstructing the original context and guessing what could or should have been said at that point in the discourse (Bates, Kintsch, Fletcher, & Guiliani, 1980; Bates, Masling, & Kintsch, 1978; Keenan, MacWhinney, & Mayhew, 1977; Kemper, 1980; Kintsch & Bates, 1977; Offir, 1973). Jenkins et al. (1977; see also Kraft & Jenkins, 1977) showed that a similar type of response bias could occur during picture orientation judgments when the orientation of pictures composing an event sequence was arranged by some contextual constraint (for instance, the main character was always on the left). To avoid this problem and heed the Bates et al. (1978, 1980) caution against interpreting such “memory-as-reconstruction” as the more rudimentary “memory-as-retrieval,” each picture's original orientation was assigned somewhat randomly (see Fig. 1).

Finally, left/right orientation was chosen as the type of surface information tested because all evidence has suggested that it is not encoded verbally (Bartlett, Till, Gernsbacher, & Gorman, 1983; Bartlett, Till, & Levy, 1980; Bartlett et al., 1984; Gernsbacher, 1980). In these experiments, the following encoding tasks were manipulated: Each picture was accompanied by a verbal description that made special reference to its orientation (“small village with mountain range on the left”); subjects verbally generated such a description; subjects verbally identified an object that should cue each picture's orientation; subjects verbally identified the orientation of this salient object; or subjects verbally rehearsed some or all of this information prior to the test. However, none of these verbal activities improved subjects’ memory for left/right orientation over that of their respective control groups. (Performance on other recognition tasks not requiring orientation judgments was enhanced by these verbal activities; so it was not the case that the manipulations were simply ineffective.)

To summarize, left/right orientation seemed the ideal type of surface information to test in the present study. Because it is analogous to linguistic surface information along one dimension (both are unrelated to content or meaning), it provided the opportunity to demonstrate the same phenomenon with a comparable stimulus. Because it is orthogonal to linguistic surface information along another dimension (it is not encoded verbally), it provided the opportunity to demonstrate the phenomenon within a nonverbal domain.

EXPERIMENT 1

In Experiment 1, subjects viewed the four picture stories in order to comprehend them. To ensure appropriate comprehension, subjects were required to write a summary of each story after viewing it. For half the pictures in each story, memory for their original left/right orientation was tested immediately after the story was viewed and subjects had written its summary. The memory test for the other half was delayed until all four stories and a filler story (a fifth sequel) had been viewed and summarized. Thus, there were two Test-Interval conditions: Immediate and Delayed. The prediction was that more surface information would be lost after comprehending several stories (the Delayed interval) than after comprehending just one (the Immediate interval).

Method

Subjects

Forty-eight undergraduate students at the University of Texas at Austin participated as one option for fulfilling a course requirement. No subject participated in more than one experiment.

Materials and design

All pictures from the original story books were photographed twice and reproduced as two 35-mm Ektachrome slides. Each story was edited so that it comprised 24 pictures. Based on each picture's orientation in the original book, a quasi-random half of the slides were reversed with the constraint that no more than two consecutive slides remained in their original orientation or were reversed. This orientation became Orientation A; its mirror image became Orientation B. Half the subjects viewed the stories in Orientation A; the other half viewed Orientation B. This defines two input-orientation conditions.

Each picture in every story was tested and the order of the pictures during the test sequence was the same as during the input sequence (see Bekerian & Bowers, 1983, for empirical support of this format). Two test orders were manipulated (Order 1 and Order 2). Half the slides in each story, a total of 48, were randomly selected. In Test-Order 1, these 48 slides were tested in the Immediate condition; in Test-Order 2, they were tested in the Delayed condition. The remaining 48 slides were tested in the Delayed condition in Test-Order 1 and in the Immediate condition in Test-Order 2. Half the subjects were tested with Test-Order 1, the other half with Test-Order 2.

Crossed with the Input-Orientation and Test-Order variables was a Test-Orientation variable (Orientation C and Orientation D). At each test interval, half the slides were tested in the same orientation in which they were originally viewed, while half were tested in the reversed (different) orientation. The experiment, therefore, was a 2 (Test-Interval: Immediate vs Delayed) × 2 (Input-Orientation: Orientation A vs B) × 2 (Test-Order: Order 1 vs 2) × 2 (Test-Orientation: Orientation C vs D) factorial design, with the first variable manipulated within subjects and the others between subjects.

Procedure

Subjects were tested in groups of six, each group corresponding to one of the eight different between-subjects conditions. Upon entering a small amphitheater, subjects were seated in three progressively elevated rows of desks, approximately 10 to 12 ft (3 to 4 m) from a standard size projection screen. Each subject was given a response booklet containing pages for the summary task and the orientation tests.

Subjects were told they were going to see five related picture stories, each of which they should try to understand. A sample story, unrelated to the other five, served as an illustration. Subjects were told that after viewing each story, they would write a few statements summarizing it. For illustration, the experimenter orally summarized the sample story. Subjects were told that in addition to measuring their understanding of the stories, the experimenter would also be testing their memory for the pictures within the stories. A slide and its mirror image from the sample story were used to illustrate the nature of the orientation test.

At the beginning of the first story, the word Ready appeared on the screen. After all 24 slides of the story had been shown, the phrase The End—Please Write Your Summary Statements appeared on the screen. Subjects then had 2 min to do so. At the end of this period, the word Test appeared on the screen. At this point, subjects turned to the first blank answer sheet and prepared to respond to the first test slide. To indicate their answers, subjects circled one of the phrases, “Sure Same,” “Guess Same,” “Guess Different,” or “Sure Different.” After 12 slides had been tested, the word Ready reappeared on the screen. Subjects then prepared to see the next story. This entire cycle repeated four times. At the end of the summary-writing period for the fifth story (the filler story), the word Test appeared on the screen. Subjects were then informed that this last test series would cover pictures viewed throughout the experiment, that is, pictures from any of the previously viewed stories.

All stimuli were projected by a Kodak Carousel projector. Slides in the input stories were shown at the rate of one every 2.5 s with the only interstimulus interval being the time required for the projector to change slides. Slides in the test series were shown at the rate of one every 5 s with the same interstimulus interval. The only light in the testing room was that provided by the projector. All slides filled the area of a standard size projection screen.

Results and Conclusions

To evaluate how well subjects comprehended the stories, their summaries were examined. Previously, two judges had viewed each story at a leisurely pace and formulated 12 major idea units per story. These two judges then scored the subjects’ summaries. On the average, the subjects included 10.6 (SD = 1.1) major idea units per story (interjudge reliability was .97). Thus, there is strong evidence that subjects were successful at comprehending these stories.

For the orientation tests, three related performance measures were computed for each of the 48 subjects and 96 pictures. The first was simple percentage correct; the second was a “corrected confidence measure”: A score of 4 for correct answers made with high confidence (“Sure Same” or “Sure Different”), 3 for correct answers made with low confidence (“Guess Same” or “Guess Different”), 2 for incorrect, low-confidence answers, and 1 for incorrect, high-confidence answers. The third measure was A′, a nonparametric index of sensitivity and bias (Grier, 1971). A′ scores range from 0.0 to 1.0: A perfect hit rate and false alarm rate yields an A′ of 1.0; an equal hit rate and false alarm rate yields an A′ of 0.50. Subjects’ means of all three performance measures are shown in Table 1.

TABLE 1.

Subjects’ Mean Percentage Correct, Corrected Confidence, and A′ in Experiments 1, 2, 4, 5, and 6

Experiment	Manipulation	Percentage correct	Corrected confidence	A ′
1	After comprehending one vs several picture stories	66	2.942	.752
		57	2.694	.634
2	After comprehending half vs an entire picture story	74	3.168	.835
		62	2.852	.705
4	Before a constituent boundary vs after a constituent boundary	79	3.305	.872
		70	3.081	.795
5	After comprehending a normal vs a scrambled picture story	68	3.012	.782
		62	2.841	.700
6	After comprehending a normal vs a scrambled written story	70	3.037	.787
		62	2.859	.700

All three measures were analyzed in sets of 2 (Test-Interval) × 2 (Input-Orientation) × 2 (Test-Order) × 2 (Test-Orientation) analyses of variance (ANOVAs). One set of ANOVAs analyzed the measures computed for each subject, and the variable “subjects” was treated as a random factor; the second set analyzed these measures computed for each picture, and the variable “pictures” was treated as a random factor (Clark, 1973). The results reported here are based on minF′ tests.

For all three measures, the only significant effect was a main effect of Test Interval: More surface information was lost after comprehending several picture stories than after comprehending only one, minF′(1, 123) = 21.69, minF′(1, 115) = 23.46, and minF′(1, 120) = 22.96, for percentage correct, corrected confidence, and A′, respectively; all p's < .001. These results parallel those repeatedly revealed with language stimuli. Note, however, that the present results were not due to subjects merely being less confident about their answers on the delayed versus immediate tests. Neither were the results due to subjects employing response biases or adopting guessing strategies. Although these criticisms have been aimed toward some of the language-based studies (Clifton, Kurcz, & Jenkins, 1965; Hayes-Roth & Hayes-Roth, 1977; James, Hillinger, & Murphy, 1977; Soli & Balch, 1976), none of them are warranted here.

The only somewhat surprising result was how quickly the surface information became lost. Even on the immediate tests, average performance was unexpectedly low, around 66% correct. This was after viewing and summarizing only 24 pictures, an interval that corresponded in real time to less than 3.5 min. It would be informative to find out how much surface information is lost at an earlier point. That was the purpose of Experiment 2.

EXPERIMENT 2

As in Experiment 1, subjects in Experiment 2 viewed each picture story to comprehend it and then wrote a summary of it. However, each story was interrupted at its midpoint, and half the pictures presented up to that point were immediately tested. After the last picture in each story had been viewed, half the pictures presented in the second half of the story were also immediately tested. Testing at these two points constituted the Immediate test interval. A Delayed test occurred for each story after subjects had taken both immediate tests and had written their summaries. The prediction was that more surface information would be lost after comprehending an entire story (the Delayed interval) than half a story (the Immediate interval).

Method

Subjects

Forty-eight undergraduate students at the University of Texas at Austin participated as one option for fulfilling a course requirement.

Materials and design

The materials and design of Experiment 2 were the same as those of Experiment 1, with one exception. The test series were reconstructed so that an equal number of slides tested at each interval were from the first versus second half of each story.

Procedure

The procedure used in Experiment 2 was also identical to that in Experiment 1 with the following exception. Subjects were told that during the middle of each story, they would be tested over pictures they had just viewed, and at the very end of each story, they would be tested over pictures they had just viewed (i.e., since the first test). Subjects were also told that a final test would occur after they had written their summaries, and that this last test would cover pictures viewed throughout the entire story.

After the first 12 slides in each story were shown, the word Test appeared on the screen. Subjects then prepared to respond to the next slide, it being the first test slide. After 6 slides were tested, the phrase, The Story Continues . . . appeared on the screen, and the next slide resumed the ongoing story. After all 24 slides of the story had been shown, the word Test again appeared, and subjects again prepared to respond to the next slide. After 6 slides were tested, the phrase, The End—Please Write Your Summary Statements appeared. Two minutes later, the word Test reappeared, preceding 12 test slides. At the end of this last test series, the word Ready appeared on the screen and subjects prepared to view the beginning of the next story. This entire cycle repeated three times.

Results and Conclusions

Subjects’ summaries included, on the average, 10.3 (SD = 1.3) of the 12 major idea units per story (interjudge reliability was .98). For the orientation tests, a percentage correct, a corrected confidence score, and an A′ were computed for each subject and picture. Subjects’ means of all three measures are shown in Table 1. Analyses of each measure showed only a significant main effect of Test Interval: More surface information was lost after comprehending an entire picture story and summarizing it than after comprehending half of it, minF′(1, 100) = 47.10, minF′(1, 104) = 60.23, minF′(1, 101) = 56.43; all p's < .0001. Again, these results parallel those repeatedly revealed with language stimuli.

SURFACE INFORMATION LOSS AND CONSTITUENT STRUCTURE

How else might comprehending these nonverbal stories resemble comprehending the more traditional verbal ones? Practically all organized narratives have these basic components: a setting in which the major action of the narrative takes place, a main character or group of characters around whom the narrative revolves, and a plot (Perrine, 1970). Clearly these nonverbal narratives meet these requirements.

Furthermore, many have suggested that the plot, the event sequence of a narrative, is usually characterized by another structural specification: The events of a story can be logically partitioned into cohesive “subepisodes” or constituents (Black & Bower, 1979; Kintsch, 1977; Haberlandt, 1980, 1984; Mandler & Johnson, 1977; Meyer, 1975; Rumelhart, 1977; Thorndyke, 1977).² Note, however, that not all theorists have endorsed this proposal (e.g., Black & Wilensky, 1979). For some, narrative sequences are best described as linear chains of events. However, here the working hypothesis was the claim that all narratives do comply with this general principle; that is, even these picture stories could be decomposed into their constituent structures.

The motivation for exploring this possibility was the opportunity to demonstrate a further parallel between surface information loss following comprehension of verbal versus nonverbal stimuli. Several language-based studies have elucidated a rather intriguing aspect of the basic phenomenon: Apart from the passage of time and the comprehension of subsequent material, the constituent structure of the material greatly affects memory for its surface form. More specifically, information about original surface form becomes markedly less available just after comprehension has crossed the boundary of one constituent into another.

In these language-based studies, a constituent approximates a linguistic clause or phrase, and the type of surface information tested usually has been memory for exact wording of these clauses or phrases. For example, a series of studies by Jarvella demonstrated that words in the most recently processed constituent had the highest probability of being remembered verbatim; such exact memory declined dramatically for words prior to the last constituent boundary (Jarvella, 1970, 1971, 1973, 1979; Jarvella & Herman, 1972; Jarvella, Snodgrass, & Adler, 1978; see also Marslen-Wilson & Tyler, 1976). Caplan (1972) and Chang (1980) demonstrated that a previously seen or heard word was verified more rapidly when the constituent structure of its sentence placed it in its final, as opposed to next-to-final clause. Clark and Sengul (1979) demonstrated that identifying the referent of a pronoun (she) or definite noun phrase (the woman) occurred more smoothly when the implied referent was explicitly mentioned in the previous clause; if mentioned even two clauses earlier, momentary processing difficulties were experienced. And finally, Levelt and Kelter (1982) demonstrated that the normal tendency for the structure of a response to mimic that of its eliciting question [“(At) What time do you close?” elicits “(At) Six o'clock?”] was substantially reduced with the additional intervention of one clause following the form-eliciting question.

With the picture stories used here, one could similarly examine the effect of constituent structure on surface information loss. Memory for a picture's left/right orientation could be tested either before or after the boundary of its respective constituent had been crossed. If the temporal loss of surface information is indeed an amodal phenomenon, and crossing a constituent boundary affects this loss, then such an experiment should demonstrate results parallel to those in language-based studies. Of course, a prerequisite for this demonstration was knowledge of the constituent structure of each story. One method of charting constituent structure that has provided reasonably reliable results is simply asking subjects to make subjective judgments about it (Baggett, 1979; Bower, Black, & Turner, 1979; Grosjean, Grosjean, & Lane, 1979; Levelt, 1970; Martin, 1970; Mandler, 1980; Pollard-Gott, McCloskey, & Todres, 1979; but see Mandler & Murphy, 1983). That was the purpose of Experiment 3.

EXPERIMENT 3

Subjects in Experiment 3 first viewed each story in its normal sequential manner. Then, all the pictures from that story were displayed simultaneously and subjects marked off sequences of pictures they judged to belong to the same subepisode or constituent. By computing the frequency of subjects marking different locations, the points where a majority of the subjects agreed one constituent ended and the next began were identified.

One particular aspect of Experiment 3 needs further explanation. The strongest language-based demonstrations of greater surface information loss after crossing constituent boundaries have been those where crossing boundaries was not confounded with other measures of the test interval. The stimuli in these studies were constructed so that the same number of items (words, clauses, or sentences) intervened between a target's original occurrence and its subsequent test whether or not a boundary was crossed (Caplan, 1972, Experiment 4; Chang, 1980, Experiment 1). To provide an equally strong nonverbal demonstration, the present stimuli were also constructed this way.

When the four picture stories were edited for Experiments 1 and 2, several less consequential pictures were left out of each. In the constituent boundary experiment (Experiment 4), these previously omitted pictures served as fillers: They were inserted into a constituent when testing before its boundary was crossed and were omitted when testing after. This way, a picture was always tested after the same number of other pictures intervened.

Of course, it was also important that the filler pictures not disrupt the perceived boundary locations. So in the present experiment, half the subjects made constituent boundary assignments on the stories with their filler pictures inserted (the “extended” versions), and half made them with the filler pictures omitted (the “abbreviated” versions). If the pictures were not disruptive, the two groups’ judgments should concur.

Method

Subjects

Forty undergraduate students at the University of Texas at Austin participated as one option for fulfilling a course requirement.

Materials and design

All pictures from both the extended and abbreviated versions were photoduplicated and reduplicated onto transparency film sheets. These transparencies allowed a further photoduplication of each picture in both orientations. The pictures were then arranged according to both Input-Orientation A and Input-Orientation B (as defined in Experiment 1). The filler pictures was randomly assigned an orientation. For each story, its pictures were displayed horizontally from left to right on four strips of 2 × 14-in. paper. The four strips were pieced together creating one long strip per story (a “picture strip”).

Twenty subjects viewed and assigned constituent boundaries to the abbreviated versions; 20 did the same with the extended versions. Half of each group viewed Orientation A; the other half viewed Orientation B. The experiment, therefore, was a 2 (Story-Length) × 2 (Input-Orientation) factorial, with both variables manipulated between subjects.

Procedure

Subjects were tested in groups of 10, each group corresponding to one of the four between-subjects conditions. Testing took place in the same experimental room as used in Experiments 1 and 2. Upon entering the room, subjects were given a picture strip for each story. Each picture strip had been folded at its seams so that none of its pictures were exposed.

Subjects were told that they were going to see four related picture stories that they should try to understand. After viewing each, they would have to write a summary of it. The sample story and its summary used for illustration in Experiments 1 and 2 were also used here. Subjects were told that in addition to how well they understood the stories, the experimenter was also interested in their judgments about how each story could be divided up into its mini- or subepisodes. (To avoid confusion, the term “constituent” was never used.) The experimenter discussed what was meant by subepisodes of a story, and all subjects appeared to understand. As an illustration, the experimenter explained where she thought the divisions between subepisodes in the sample story occurred, and all subjects seemed to agree.

After viewing each story and writing its summary, subjects were instructed to unfold their picture strips and begin marking the divisions. Progressing from left to right, subjects indicated their judgments by drawing a line between adjoining pictures. The only restrictions were to make at least one mark per story and fewer than one mark per picture. Subjects were cautioned against letting the seams made in constructing the picture strips bias their judgments.

Slides in the input stories were shown at the same rate as in the previous two experiments, and again subjects were allowed 2 min to write their summaries. The boundary assignment task was not timed. The word Ready appeared on the screen, indicating the start of the next story, when all subjects were finished making their judgments.

Results and Conclusions

A first concern was whether the judged constituent structure of each story differed when presented in its abbreviated versus extended length or, less likely, when presented in Orientation A versus B. To evaluate these differences, three types of analyses were performed. First, the mean number of boundaries marked per story was compared. In the abbreviated length condition, subjects marked an average 4.58, 4.66, 5.22, and 5.67 boundaries within the first, second, third, and fourth stories, respectively. In the extended length condition, subjects marked an average 4.90, 4.85, 5.90, and 5.60 to the same. These means did not differ significantly (all t's < ± 1.00). Neither did the means differ between Orientation A versus B (all t's < ± 1.00).

Second, the frequency of subjects marking each possible boundary location (i.e., after each picture) was compared. Excluding the pictures appearing only in the extended versions, these data were analyzed by a 2 (Story-Length) × 2 (Input-Orientation) × 4 (Story 1 vs 2 vs 3 vs 4) ANOVA. No main effects or interactions were significant (all F's < 1.00).

Third, the degree of agreement across conditions was computed. The Pearson correlations between boundary frequencies assigned to stories in the abbreviated versus the extended conditions were .87, .84, .78, and .96, for the first through fourth stories, respectively (all N = 24; all p's < .001). Across all four stories, the correlation was .85 (N = 96; p < .001). The correlations between boundary frequencies assigned to stories viewed in Orientation A versus Orientation B were .83, .86, .76, and .92, for the first through fourth stories, respectively (all N = 24; all p's < .001). Across all four stories the correlation was .84 (N = 96; p < .001).

Thus, it appeared that neither the inclusion of their filler pictures nor the input orientation of their slides affected these stories’ perceived constituent structures. Equally important, about these structures subjects displayed relatively high agreement.

A final concern was where these generally agreed upon boundaries were. Boundaries were operationalized by two criteria. First, they were locations marked by more than 50% of the subjects. Second, they were locations whose frequencies were significantly greater than any other (nonboundary) location in that story. This resulted in three boundaries per story. In the first story, the first, second, and third boundaries were marked, respectively, by 55, 55, and 63% of the subjects across all conditions. The three boundaries in the second story were marked by 63, 53, and 60%, respectively. The boundaries in the third story were marked by 55, 68, and 70%, respectively, and in the fourth story by 90, 75, and 73%, respectively. For each story, these frequencies did not differ when computed within the two Story-Length conditions, or within the two Input-Orientation conditions (all z's < 0.50). However, as just mentioned, they were all significantly greater than the other frequencies in each story (all z's > 2.10; all p's < .01).

EXPERIMENT 4

With the constituent boundaries of each story assigned, a goal higher on the agenda was returned to: measuring surface information loss before versus after crossing these boundaries. Experiment 4, though similar to Experiments 1 and 2 in its fundamental purpose, differed slightly in its approach. The chief difference between those experiments and the present one was the test intervals. The two test intervals manipulated in Experiment 4 were Before versus After a constituent boundary. In the Before-Boundary condition, a picture was tested before the onset of the next constituent. This meant that a picture was tested immediately following the last picture of its constituent. In the After-Boundary condition, a picture was tested after the onset of the next constituent. This meant that a picture was tested immediately following the first picture of the next constituent.

In addition to their perceived constituent structure, another structural property of these picture stories suited this manipulation. Because the four stories were sequels, they could be combined to form one long story. By presenting the pictures as a composite story, a picture from each story's final constituent could be tested. For example, when a picture from the last constituent of the first story was tested in the After-Boundary condition, the test occurred after the beginning of the second story, that is, after its first picture had been viewed. Thus, a second difference between the previous experiments and Experiment 4 was that subjects viewed all the experimental pictures as if they were one story. Subjects were again required to write a summary, but it was for the composite story.

And the final difference between this experiment and the others was that a continuous recognition paradigm was used (cf. Amnion, 1968; Begg, 1971). At various points, the ongoing story was interrupted and subjects were tested over one of the pictures recently viewed. Immediately after this picture was tested, the story resumed. So, unlike the other experiments, every picture was not tested, rather one picture from each constituent was. In half the trials, this occurred before a boundary was crossed, and in half this occurred after. The prediction was that more surface information would be lost at the After-Boundary than the Before-Boundary interval.

Method

Subjects

Forty-eight undergraduate students at the University of Texas at Austin participated as one option for fulfilling a course requirement.

Materials and design

The stimuli were the 96 pictures used in Experiments 1 and 2, the 16 filler pictures used in Experiment 3, and the fifth sequel. (The fifth sequel was used so that a picture from the last constituent of the fourth story could be tested.) One picture from each of the four constituents of the four experimental stories was selected. These 16 pictures were the test slides. An equal number of test slides occurred three, four, five, and six slides prior to the end of their respective constituent.

Two test orders were manipulated (Order 1 and Order 2). In Test-Order 1, half the 16 test slides were tested in the Before-Boundary condition. Of these 8 slides, an equal number were from each of the original four stories, and an equal number occurred at each distance from the end of their constituents. When tested in this condition, a filler slide from each of their constituents was inserted into the input series. In Test-Order 2, these 8 slides were tested in the After-Boundary condition. In this case, their filler slides were omitted. The remaining 8 test slides were tested in the Before-Boundary condition in Test-Order 1 (with their filler slides inserted), and the After-Boundary condition in Test-Order 2 (with their filler slides omitted). Figure 3 illustrates how a constituent and its test slide appeared in the Before- versus After-Boundary conditions. In addition, four slides occurring at random points within the fifth sequel (the filler story) were tested; however, these slides were neither manipulated nor scored. Finally, as in Experiments 1 and 2, the variables of Input-Orientation, Test-Order, and Test-Orientation were manipulated.

Fig. 3.

Example constituent sequence and its test slides as they appeared in the Before- versus After-Boundary conditions. (From One Frog Too Many, copyright © 1975 by Mercer and Marianna Mayer. A Dial Books for Young Readers book. Reproduced by permission of E. P. Dutton, a division of New American Library.)

Procedure

The procedure was very similar to that of Experiment 2 with the following exceptions. First, subjects were never told that there were four picture stories, only that there was one. Second, the slide appearing after the word TEST was the only slide tested at that point and was always followed by the words, The Story Continues. When the test slide was tested in the Across-Boundary condition, i.e., the test occurred after the first slide of the next constituent, this first slide was re-presented when the story resumed. Finally, because the subjects wrote a summary covering the composite story, they were allowed 10 min to do so. Hence, after all slides (including the fifth sequel) had been viewed, the words, Please Write Your Summary appeared. All other timing was the same as in Experiments 1 and 2.

Results and Conclusions

A percentage correct, a corrected confidence, and an A′ score were computed for each of the 48 subjects and 16 pictures. Subjects’ means of all three measures are shown in Table 1. For each measure, only a main effect of Test Interval was significant: More surface information was lost immediately after crossing a constituent boundary than immediately before, minF′(1,32) = 4.61, minF′(1,34) = 4.31, minF′(1,35) = 4.86; all p's < .05. These results again parallel those previously found with verbal stimuli. In fact, the present results provide an even stronger demonstration of the constituent boundary effect than many of its verbal counterparts. This is because some of the language-based results have been interpreted as arising from spurious factors. For example, it has been suggested that the constituent boundary effect demonstrated when subjects verbally recall sentences might be attributable to a production bias (Levelt & Kempen, 1975). A related suggestion has been that the effect demonstrated with auditorily presented sentences might be attributable to intonation cues (Marslen-Wilson & Tyler, 1976). Obviously, neither criticism is appropriate here.

In addition, the summaries subjects wrote in Experiment 4 were scored. On the average, subjects included 9.9 (SD = 1.2) major idea units for each of the four experimental stories. Comparing these summaries with those written in Experiments 1 and 2 provided an estimate of how much thematic information was lost over time. This is because in Experiments 1 and 2, subjects wrote their summaries immediately after viewing each story (and taking its orientation test), whereas in Experiment 4, subjects did not write their summaries until all four experimental stories plus the filler story had been viewed. Thus, the summary-writing interval in Experiments 1 and 2 was like an Immediate test, whereas that in Experiment 4 was like a Delayed test. However, neither the comparison between Experiment 1 versus 4, nor that between Experiment 2 versus 4 showed a reliable difference in the number of major idea units contained in subjects’ summaries (both t's < 1.0). The slight decrease observed in Experiment 4 (9.9 vs 10.6 or 10.3), may have been because subjects were to write longer summaries and, therefore, they felt they could be less thorough. In any case, the analyses suggested that virtually no thematic information was lost over time, which again replicated experiments done with verbal stimuli (Sachs, 1967, 1974). This is in contrast to surface information which quickly becomes lost during comprehension. The focus of the second part of this research was investigating why surface information is lost.

INVESTIGATING THE CAUSE OF SURFACE INFORMATION LOSS

Four explanations were considered. Two were rejected because they were inadequate; a third and fourth were examined in the last set of experiments. These four explanations were the linguistic hypothesis, the memory limitations hypothesis, the integration hypothesis, and the processing shift hypothesis.

The Linquistic Hypothesis

An early explanation of this phenomenon was based on Chomsky's then popular transformational grammar (e.g., Mehler, 1963; Miller, 1962): To comprehend a sentence, one must syntactically detransform it back to its simple, declarative, active, affirmative base. Though the detransformation process provides linguistic tags which can be used for recovering original surface form, the tags are often lost or irretrievable.

One major problem with this explanation was that it required a set of syntactic rules specifying the necessary transformations used during comprehension—in other words, a psychologically “real” grammar. Specifying such a grammar for English sentences has not proved to be an easy task (Bresnan & Kaplan, 1982). And though there have been novel attempts to specify grammars for nonverbal media, e.g., cinematic films (Carroll, 1980) and musical symphonies (Bernstein, 1976), the possibility of specifying a grammar to describe the pictures used here seemed remote. Another major problem with the linguistic hypothesis was that over two decades of experiments using verbal stimuli alone, this explanation has steadily lost support (Fodor, Bever, & Garrett, 1974; Garnham, 1983; Gough, 1971). Thus, it seemed appropriate to discard this explanation because it was inadequate and to search for an explanation outside the language domain. This approach was not atypical; when other phenomena originally believed to be unique to language processing were demonstrated outside that domain (for instance, categorical perception and selective adaptation), amodal explanations were sought for them too (Diehl, 1981). Hence, the other three explanations considered could account for surface information loss regardless of modality.

The Memory Limitations Hypothesis

According to this explanation, surface information is lost because the limitations of a short-term memory are exceeded. Historically, this was the second popular explanation offered for the phenomenon. Obviously, information can be held in a short-term memory only for a short term, and though occasionally information can remain past its normally short term (via rehearsal or reactivation), the opportunities for doing so are constrained by a second limitation: Short-term memory's capacity is limited (e.g. Miller, 1956).

However, Experiment 4 and its verbal precursors demonstrated an aspect of surface information loss for which these memory limitations cannot account. These experiments demonstrated that apart from the passage of time or the subsequent comprehension of additional material, the structure of the information being comprehended affected the loss of its surface information. In other words, it was not only the quantity of information contained in short-term memory or the duration it might have been held there that predicted its probability of loss. For example, in Experiment 4, the test interval (in terms of the number of pictures and the interval of time) was the same in the Before- and After-Boundary conditions. Yet performance in these two conditions differed, and the short-term memory limitations presumed to cause surface information loss cannot account for that finding.³

To account for such findings, a corollary assumption has often been made: Surface information is held until a meaningful unit has been comprehended; then it is lost (Clark & Clark, 1977; Foss & Hakes, 1978; Jarvella, 1979; Sanford & Garrod, 1981; Slobin, 1979). But this assumption undermines the original explanation. As demonstrated in Experiment 3 (and repeatedly with language), all constituents are not the same size; ergo, they would not all consume the same amount of space or be held for the same period of time. If while waiting for a constituent to end, short-term memory can hold a variable amount of information for a variable period of time, then why is the information ever lost? Perhaps the system is so “smart” that in anticipation of a time or space limitation it chooses to expunge at a structurally appropriate interval. But one is now left without an a priori specification of how long or how much information can be held and no causal link. Therefore, the memory limitations hypothesis was also rejected because it was inadequate.

The Integration Hypothesis

According to this explanation, surface information is lost because the goal of comprehension is to integrate incoming information into a more meaningful representation, usually referred to as gist (Bransford & Franks, 1971, 1972). At the outset, original surface information is vital for comprehension even to occur; however, as the process continues, information becomes more and more integrated. Hence at the final stages, surface information may no longer be available—at least not directly in its original form—because the normal process of comprehension has transformed it into gist.

Consider the analogy of baking a cake. As the cake bakes, several raw ingredients (salt, flour, butter, sugar) become increasingly integrated. After the process is well underway, it is difficult to extract any of these ingredients in their original raw forms. Now consider Bransford and Franks’ (1971) seminal experiment. Subjects comprehended a series of thematically cohesive sentences and on a later recognition test demonstrated serious loss of surface information. In particular, subjects were poor at remembering structural information about sentence boundaries. Less well known is a later experiment by Peterson and McIntyre (1973). In one condition, they perfectly replicated the Bransford and Franks (1971) effect. In a second condition, their input sentences were not thematically cohesive, and for these sentences, subjects demonstrated significantly better retention of surface information. One explanation is that with Bransford and Franks’ paradigm, the input sentences could easily be integrated into gist; conversely, in Peterson and McIntyre's second condition, they could not and thus remained in their relatively raw form.

Other studies have converged on this explanation. J. R. Anderson and Bower (1973, p. 224) found that memory for the original (active vs passive) voice of a sentence was significantly worse when the input sentences formed a cohesive story than when the sentences were semantically unrelated. De Villiers (1974) found that more synonym substitutions occurred when recalling sentences originally processed as a thematic story than when the sentences seemed completely independent (see also Luftig, 1981; Pompi & Lachman, 1967). Similarly, bilinguals’ memory for the language in which a word was originally spoken was considerably poorer when the words composed a unified sentence rather than an unrelated list (Saegert, Hamayan, & Ahmar, 1975; see also Rose, Rose, King, & Perez, 1975).

In all these studies, integrating the input into a more meaningful representation (gist) apparently caused surface information loss. Thus, these studies presumably provided evidence that the process of integration is responsible. However, the data that have best supported the integration hypothesis were collected in experimental situations that have least represented typical comprehension. That is, these data were elicited from conditions in which the to-be-comprehended stimuli were semantically unrelated and void of thematic integrity (or at least it appeared that way to subjects). It is difficult to draw conclusions about a presumed outcome of the comprehension process from situations where comprehension (in its usual sense) cannot even occur (for comparable arguments see Moeser, 1976, and Perfetti & Goldman, 1974). A more valid test of this hypothesis would involve creating two experimental conditions; in both, comprehension could occur but integration would be less likely in one than the other. That was one purpose of Experiment 5. A second purpose was to test another explanation, the last of the four considered.

The Processing Shift Hypothesis

This last explanation is being described for the first time here. It is based on the following framework. During the comprehension process, memory “cells” are activated by incoming information. (Memory cells contain previously stored mental representations or traces.) Once activated, these memory cells immediately transmit processing information (enhancement and suppression) to their connecting cells.

According to this framework, the goal of comprehension is to build up a coherent mental representation or “structure.” Initial activation of memory cells and their transmission lays a foundation. Once laid, congruent (similar or related) information simply adds on to the developing structure. This is because the more overlapping the incoming information is with that previously received, the more likely it is to activate the same or connected cells. However, the less congruent the incoming information is, the less likely it is to activate the same or connected cells and the less readily it can be added onto the structure currently being developed. In this case, a different set of cells is activated. Because this second set of cells has not been recently activated, a relatively new foundation begins to be laid. This shift from actively building one structure, really a substructure, to initiating another is called a processing shift.

As the name implies, processing shifts are presumably the cause of surface information loss during comprehension. This is because information represented in a particular substructure is most available during the active processing of that substructure. Once a processing shift has occurred, information represented in the previous substructure becomes less available.

The processing shift hypothesis explains the effect of crossing a constituent boundary much as it does the effect of comprehending additional material. Constituent boundary effects are the direct result of processing shifts. That is, if subjects perceive a constituent boundary between two adjacent stimuli, it is unlikely that those two stimuli are congruent. And the less congruent two adjacent stimuli are, the greater the probability that the second stimulus leads to a processing shift. The effects of time or amount of information presented result from cumulative probabilities of processing shifts. That is, the longer the delay between a stimulus’ initial presentation and its test, or the more subsequent stimuli comprehended during this interval, the less likely it is that the subsequent stimuli are congruent with the initial stimulus or others in the sequence. And the less congruent any of those stimuli are, the greater the probability of one or more processing shifts occurring.

Despite this explanation's novelty, several of its basic assumptions have already received empirical support. First, data from many different sources have supported the assumption that the initial stage of comprehension involves laying a foundation. For example, in virtually all experiments measuring the reading time for each sentence in a paragraph, initial sentences took longer to read than subsequent ones (Cirilo, 1981; Cirilo & Foss, 1980; Glanzer, Fischer, & Dorfman, 1984; Graesser, 1975; Haberlandt, 1980, 1984; Haberlandt, Berian, & Sandson, 1980; Haberlandt & Bingham, 1978; Olson, Duffy, & Mack, 1984). This effect maintained regardless of where the paragraph's topic sentences occurred (Greeno & Noreen, 1974; Kieras, 1978, 1981). In addition, the first sentence of a story's subepisode (or constituent) took longer to read than other sentences in the constituent (Haberlandt, 1980, 1984; Haberlandt et al., 1980; Mandler & Goodman, 1982). Similarly, in experiments measuring the reading time for each word within a sentence, initial words took longer to read than subsequent words (Aaronson & Ferres, 1983; Aaronson & Scarborough, 1976; Chang, 1980). Moreover, the same word was read more slowly when it occurred at the beginning of a phrase than at the end (Aaronson & Scarborough, 1976). With auditory comprehension, latencies to monitor for a target phoneme or word were longer when the target occurred during the beginning of a sentence or phrase than later (Cairns & Kamerman, 1975; Cutler & Foss, 1977; Foss, 1969; Hakes, 1971; Marslen-Wilson, Tyler, & Seidenberg, 1978; Shields, McHugh, & Martin, 1974).

Both the reading time and monitoring data displayed the pattern expected if comprehenders used initial stimuli (sentences or words) to lay a foundation for their mental representations of paragraphs, story constituents, sentences, or phrases. Moreover, neither type data displayed this pattern when the stimuli did not lend themselves to coherent mental structures—for example, when the sentences or paragraphs were self embedded or extensively right branching (Foss & Lynch, 1969; Greeno & Noreen, 1974; Hakes & Foss, 1970; Kieras, 1978, 1981).

Furthermore, recall was better when the first sentence of a story constituent was the recall cue (Mandler & Goodman, 1982). This too suggests that the initial stimuli served as a foundation onto which subsequent information was added. Indeed, initial stimuli play such a fundamental role in organizing mental structures that when asked to recall the main idea of a paragraph, subjects were most likely to select the initial sentence even when the actual theme was a sentence occurring later (Kieras, 1980, 1981).

The processing shift hypothesis assumes that when incoming information is congruent, it is mapped onto the developing substructure. In language comprehension, several mechanisms are assumed to signal congruity (cf. Carpenter & Just, 1978; Halliday, 1967). One of the most common is repetition, ranging from literal to anaphoric. Sentences employing repetition were read more rapidly, suggesting that they were easier to map onto a developing structure (Garnham, 1981, 1984; Garrod & Sanford, 1977; Haviland & Clark, 1974; Kintsch, Kozminsky, Streby, McKoon, & Keenan, 1975; Mannelis & Yekovich, 1976; Sanford & Garrod, 1980; Yekovich & Walker, 1978). Data from memory tasks (cued recall, free recall, and priming) suggest that propositions co-referenced by repetition were more likely to be represented near one another, perhaps in the same substructure (Hayes-Roth & Thorndyke, 1979; Kintsch et al., 1975; McKoon & Ratcliff, 1980a, 1980b). Another mechanism that signals congruity is causality; the more causally related a target sentence was to its preceding context, the more rapidly it was read (Keenan, Baillet, & Brown, 1984) and the more likely it was to be recalled when cued by its preceding sentence (Black & Bern, 1981).

The processing shift hypothesis also assumes that when less congruent information is received, a shift from building one substructure to initiating another—a processing shift—will occur. This assumption has also already received empirical support. In reading-time experiments, sentences and words that changed the ongoing topic, point of view, or setting took substantially longer to comprehend than those that continued it (A. Anderson, Garrod, & Sanford, 1983; Black, Turner, & Bower, 1979; Daneman & Carpenter, 1983; Dee-Lucas, Just, Carpenter, & Daneman, 1982; Gernsbacher, 1984a; Haberlandt et al., 1980; Lesgold, Roth, & Curtis, 1979; Mandler & Goodman, 1982; Olson, Duffy, & Mack, 1980). This is the pattern expected if upon encountering these changes, subjects had difficulty mapping the incoming information onto the structure they were developing and, hence, broke off building one substructure and began another.

Finally, the processing shift hypothesis assumes that after a processing shift occurs, information represented in the previous substructure becomes more difficult to access. This assumption has also received empirical support. In a recent experiment (Gernsbacher, 1984a), subjects read paragraph-long passages, and either immediately before or after an episode shift occurred, they answered yes/no questions. The questions always probed information presented in the first part of the passage (preepisode shift information). If comprehending an episode shift necessitated a processing shift, then answering these questions should have been more difficult after an episode shift than before. And indeed, this is what was found (see also A. Anderson et al., 1983).

The component of the processing shift hypothesis that had not been empirically examined was that processing shifts lead to surface information loss. Thus, what was needed was an experiment to test this explanation. In addition, a more valid test of the remaining alternative explanation, the integration hypothesis, was also needed. That was the purpose of Experiment 5.

EXPERIMENT 5

As in the first four experiments, subjects in Experiment 5 viewed the four nonverbal stories in order to comprehend them. Immediately after viewing each, memory for its surface information was tested. However, for two of the stories, their pictures did not occur in their natural order but merely in a random sequence. Thus, there were two Story-Type conditions: Normal and Scrambled.

This scrambling manipulation served three purposes. First, it provided a more valid test of the integration hypothesis because unlike lists of isolated or seemingly unrelated sentences, stories composed of scrambled sentences possess a theme. With appropriate instructions, subjects do attempt to obtain the gist of scrambled stories, and they meet with some success—though less than with unscrambled (normal) ones (Bower et al., 1979; Kieras, 1978, 1981; Kintsch, Mandel, & Kozminsky, 1977; Mandler, 1978; Schwartz & Flammer, 1981; Stein & Glenn, 1979; Thorndyke, 1977). Indeed, the probability of integration is lowered simply by separating what should ordinarily be consecutive units (Frase, 1975; Hayes-Roth & Thorndyke, 1979; Moeser, 1977, 1979; Walker & Meyer, 1980).

Second, the scrambling manipulation provided an empirical test of the processing shift hypothesis because stimuli presented in a scrambled order, by definition, are relatively incongruent. Thus, building a coherent mental representation of a scrambled story should involve several processing shifts. Third, this manipulation pitted the two explanations against one another because the predictions derived from each were in opposition. According to the integration hypothesis, surface information is lost because it becomes transformed into gist: the lower the probability of integration, the lower the probability of surface information loss. Because comprehending scrambled stories leads to a lower probability of integration, the prediction derived from this explanation was that less surface information would be lost in the Scrambled than the Normal condition. Conversely, according to the processing shift hypothesis, surface information is lost because of shifting from building one substructure to another: the higher the probability of a processing shift, the higher the probability of surface information loss. Because comprehending scrambled stories leads to a higher probability of processing shifts, the prediction derived from this explanation was that more surface information would be lost in the Scrambled than the Normal condition.

Method

Subjects

Forty-eight undergraduate students at the University of Texas at Austin participated as one option for fulfilling a course requirement.

Materials and design

The four stories in their 24-picture lengths were used. When manipulated in the Normal condition, the slides were presented in the same sequence as in Experiments 1 and 2. A story was scrambled by arranging its slides according to a random number sequence.

Two test orders were manipulated. In Test-Order 1, the first and third stories were normal and the second and fourth were scrambled; in Test-Order 2, the first and third stories were scrambled and the second and fourth were normal. Again, each picture was tested and the order of the slides during the test sequences were the same as during the input sequences. This meant that for the scrambled stories, their slides were viewed in the same random sequence during input and test. Finally, as in Experiments 1 and 2, the Input-Orientation and Test-Orientation variables were manipulated.

Procedure

The procedure used in Experiment 5 was similar to that in Experiment 1 except that subjects were told that some of the stories would appear in a scrambled order; nevertheless, subjects were encouraged to comprehend each story as well as possible. At the beginning of the first story, the word Ready appeared on the screen. After all 24 of its slides had been viewed, the word Test appeared. After these 24 slides had been tested, the phrase Now Please Write Your Summary Statements appeared. Subjects then had 2 min to do so, after which the word Ready appeared, signaling the start of the second story. This cycle repeated three times.

Results and Conclusions

Subjects’ summaries were scored, and the average number of main idea units included in summaries of the normal stories was 10.5 (SD = 1.1), and for the scrambled stories, 8.2 (SD = 1.9). These means differed significantly, t(1,46) = 2.05, p < .05. Thus, subjects were better able to comprehend the normal than the scrambled stories. However, given that subjects’ summaries of the scrambled stories still included approximately 75% of the major idea units, it appeared that subjects did perceive those stories as having thematic integrity.

For the orientation tests, a percentage correct, a corrected confidence, and an A′ were computed for each subject and picture. Subjects’ means of all three measures are shown in Table 1. For each measure, only a main effect of Story Type was significant: More surface information was lost after comprehending scrambled than normal stories, minF′(1,131) = 8.14; minF′(1,130) = 7.47; minF′(1,130) = 7.96; all p's < .01. This was the prediction derived from the processing shift hypothesis. Thus, the shift from building one mental structure to initiating another appears to be the more likely explanation of surface information loss. However, before concluding that a truly amodal explanation of the well-known phenomenon had been found, it was important to replicate this experiment using the more traditional comprehension medium, verbal narratives. That was the purpose of Experiment 6.

EXPERIMENT 6

Experiment 6 was identical to Experiment 5 except that subjects did not comprehend the stories by viewing pictures; instead, they read sentences. Also, the type of surface information tested was not each picture's original left/right orientation; instead, it was each sentence's original word order. Despite this difference in modality, the prediction for Experiment 6 was the same as for Experiment 5: More surface information would be lost in the Scrambled condition than the Normal.

Method

Subjects

Thirty-two undergraduate students at the University of Oregon participated to earn extra credit in a course.

Materials and design

For each picture, a pair of sentences was constructed.⁴ Each pair comprised the same words, but the order of the words differed. For example, the pair of sentences constructed for the first picture in Fig. 1 was “In a box arrived a present for the little boy. / A present arrived in a box for the little boy.” For the second picture, the pair was “The turtle, frog, and dog watched as the boy opened it. / As the boy opened it the turtle, frog, and dog watched.” All sentences were 13 syllables long. Considerable care was taken during their construction so that rearranging their word order would not affect their meaning. Each sentence was type on white paper using an IBM Selectric Orator typeball. They were then photographed and reproduced as 35-mm slides.

One sentence of each pair was randomly assigned to be Word-Order A (comparable to Orientation A in Experiment 5), and its mate became Word-Order B (comparable to Orientation B). These two Word-Order conditions were manipulated in the same way as the two Input-Orientation conditions of Experiment 5; similarly, two Test-Word-Order conditions were manipulated in the same way as the two Test-Orientation conditions of Experiment 5. The Story-Type and Test-Order variables were also manipulated.

Procedure

The procedure followed in Experiment 6 was identical to Experiment 5.

Results and Conclusions

A percentage correct, a corrected confidence score, and an A′ were computed for each subject and sentence. Subjects’ means of all three measures are shown in Table 1. For each measure, only a main effect of Story Type was significant: More surface information was lost after comprehending scrambled than normal stories, minF′(1,100) = 10.72, minF′(1,121) = 8.59, minF′(1,162) = 9.78, respectively; all p's < .01. These results replicate those of Experiment 5. Indeed, when the two sets of data were combined and analyzed with ANOVAs that included the variable “experiment” (Experiment 5 vs 6), no main effects or interactions were revealed for this variable in any of the three response measures’ analyses (all minF's < 1.00).

So, together the results of these two experiments imply that the processing shift hypothesis is an adequate amodal explanation of surface information loss in comprehension. Indeed, this explanation should fare well in accounting for the phenomenon with a wide array of stimuli: musical notation (Bean, 1937; Sloboda, 1974, 1976; Weaver, 1943; Wolf, 1976), technical drawings (Egan & Schwartz, 1979), physics computations (Larkin, 1981; Larkin, McDermott, Simon, & Simon, 1980), or computer programs (McKeithern, Reitman, Reuter, & Hirtle, 1981). This is because at a general level, the processing involved in comprehending these various stimuli should be comparable (see Kintsch & van Dijk, 1978, p. 364, for a related view). In each case, the goal of comprehension is to build up a coherent mental representation of the entire stimulus.

THE PROCESSING SHIFT FRAMEWORK

The processing shift hypothesis was drawn from a framework that up to this point has been only sketchily described. In this last section, more about this framework and its implications for surface information loss are discussed.

Surface Information and Enhancement and Suppression

According to this framework, once memory cells are activated they immediately transmit processing signals (enhancement and suppression) to their connecting cells. Enhancement is a facilitory signal to sustain or increase activation, and suppression is an inhibitory signal to dampen or decrease it. Both mechanisms occur after activation and both should account for a growing body of comprehension data particular to structure building.

For instance, enhancement and suppression should account for the disambiguation of homographs. It is now known that immediately upon hearing a homograph (bug), all its meanings are activated. This occurred even when a particular meaning was specified either by the preceding semantic context (spiders, roaches, and other bugs; Swinney, 1979), or in the case of noun/verb homographs (watch), the preceding syntactic context (I bought the watch vs I will watch; Tanenhaus, Leiman, & Seidenberg, 1979). However, with both types of homographs, only the contextually relevant meaning remained activated 200 ms later.

Evidence from a recent experiment by Hudson and Tanenhaus (1984) suggested that the responsible process resembles suppression as opposed to simple activation decay. In their experiment, the preceding contexts did not bias any one meaning, and both meanings remained activated past 200 ms. If the decreased activation of contextually irrelevant meanings found in other experiments was due to automatic decay, then in the Hudson and Tanenhaus experiment both meanings should have decayed at 200 ms. Instead, it appeared that because there were no sources of suppression, neither meaning was suppressed at 200 ms. This was true even at 500 ms, further suggesting that this process was not simply decay.

Evidence for an enhancement process comes from experiments investigating “unbalanced” homographs whose meanings differ in their frequency of use (mint). Immediately after presentation of an unbalanced homograph, both the dominant meaning (a flavor or candy) and the subordinate meaning (a place where coins are made) showed equal activation (Onifer & Swinney, 1981). Then, with biasing contexts, both the inappropriate dominant and inappropriate subordinate meanings began showing suppression at 100 ms. But at 100 ms, another important pattern emerged: The appropriate dominant meanings began to show an advantage over the appropriate subordinate ones. That is, the appropriate dominant meanings were activated above their initial state (Lucas, 1983). This is the pattern expected if the dominant meanings receive postactivation enhancement.

Why Surface Information?

According to the processing shift hypothesis surface information is lost because information represented in a particular substructure is most available during the active processing of that substructure. Once a processing shift has occurred, information represented in the previous substructure becomes less available. But why is surface information less available after a processing shift than other types of information, for example, thematic information (Sachs, 1967, 1974)?

One reason is that in most situations surface information is the most rapidly changing entity. Though surface information is usually defined as information about a stimulus that does not contribute to its meaning, another definition is that the surface properties of any stimulus are those that change the most rapidly. For instance, consider a passage of text. If well composed, each sentence conveys the same thematic idea, but each sentence does not present the same word order or syntactic form. Because in normal passages word order changes more rapidly than thematic content, word order is considered surface information while thematic content is not.

Based on this definition, the mechanisms of enhancement and suppression could explain why surface information is less available after a processing shift than other types of information. Enhancement serves to sustain activation whereas suppression serves to dampen it. Because thematic content is less rapidly changing than word order, thematic information probably receives more enhancement than surface information. Similarly, because word order is more rapidly changing, it probably receives more suppression than thematic information. The action of these two processes is not mutually exclusive. Either one might occur or they might co-occur to produce the effect.

Surface Information and Hard-to-Build Structures

Understanding surface information loss via the above definition and this framework should help to answer another question: Why is surface information less available in thematically organized than seemingly unrelated sentences (Anderson & Bower, 1973; Peterson & McIntyre, 1973; de Villiers, 1974)?

One reason is that with unrelated sentences, surface information is no longer more rapidly changing than thematic information; therefore, it would be less likely to be suppressed or more likely to be enhanced. For instance, in the Anderson and Bower (1973) experiment, half the sentences were presented in the active voice and half in the passive voice. In other words, the syntactic form of every sentence—in both the related and unrelated conditions—was always one of two types. Because the sentences in the unrelated condition had no thematic continuity, their greatest common denominator was their syntactic form. On the other hand, the greatest common denominator of the sentences in the related condition was their thematic content.

Using the same logic, another pattern of results can be explained. Several studies have reported that surface information (tested by synonym substitution) was more available after comprehending abstract than concrete sentences; on the other hand, thematic information (tested by subject-object reversal) was more available after comprehending concrete than abstract sentences (Begg & Paivio, 1969; M. Johnson, Bransford, Nyberg, & Cleary, 1972; Pezdek & Royer, 1974; Moeser, 1974). Later studies demonstrated a fundamental difference between the two sets of sentences used in these studies: The abstract sentences were less “comprehensible” than the concrete, according to several different measures (Holmes & Langford, 1976; Holyoak, 1974; Klee & Eysenck, 1973; Moeser, 1974; Schwanenflugel & Shoben, 1983).

These authors implied that the abstract sentences had less thematic content than the concrete ones. So, comprehending the words of abstract sentences may have been like comprehending the sentences of unrelated groups (no thematic cohesion); on the other hand, comprehending the words of concrete sentences may have been like comprehending the sentences of related groups (thematically cohesive). Thus, performance with the abstract sentences could have resulted from less enhancement of their thematic information or less suppression of their surface information. On the other hand, performance with the concrete sentences could have resulted from greater enhancement of their thematic information or greater suppression of their surface information. Evidence already exists to support this explanation: When the abstract sentences were each embedded in their own contextual paragraph—that is, a thematic idea was supplied—the pattern disappeared (Pezdek & Royer, 1974). With the added thematic continuity, comprehending abstract sentences mimicked comprehending concrete ones.

Surface Information and Comprehension Skill

There is strong empirical evidence that individuals differ in comprehension skill (see Carr, 1981; Perfetti, 1983, for reviews). Unfortunately the focus of much of this research has been on differences in comprehending information in one modality, namely, the printed word, and on individuals who differ at one stage of skill development, namely, beginning readers. So, it is not surprising that the mechanisms previously pinpointed are those specific to reading. However, when investigating comprehension skill among adults, one can go beyond those sources, because at an adult level of proficiency, skill at comprehending written language has been shown to be highly correlated with skill at comprehending spoken language (Daneman & Carpenter, 1980, 1983; Jackson & McClelland, 1979; Perfetti & Lesgold, 1977; Sticht, 1972). Moreover, recently I (Gernsbacher, 1984b) found that skill in comprehending language stimuli (written and spoken) was highly correlated with skill in comprehending nonverbal stimuli (e.g., the picture stories used here). Thus, the mechanism(s) underlying much of adult comprehension skill must be modality independent.

Daneman and Carpenter (1980, 1983) and Perfetti and his colleagues (Perfetti & Goldman, 1976; Perfetti & Lesgold, 1977) have provided a starting point for identifying this mechanism. These researchers pinpointed a characteristic of less skilled comprehenders that appeared during both reading and listening: It was greater surface information loss. Moreover, recently I (Gernsbacher, 1984b) found that greater surface information loss also characterized less skilled comprehenders regardless of whether they were comprehending spoken, written, or nonverbal stimuli.

However, meshing this characteristic of less skilled comprehenders with the cause of surface information loss according to the processing shift hypothesis yields a rather paradoxical hypothesis: It is that less skilled comprehenders suffer from making too many processing shifts. Recently, I (Gernsbacher, 1984b) investigated this hypothesis via the scrambling manipulation used here. I found that highly skilled comprehended lost significantly more surface information in the scrambled versus normal conditions (thus, replicating Experiments 5 and 6). On the other hand, less skilled comprehenders—whose overall performance was significantly worse than the highly skilled subjects—showed virtually no difference between the scrambled and normal conditions. This suggested that for the less skilled subjects, comprehending the normal stories involved almost as many processing shifts as comprehending the scrambled ones. Thus, the hypothesis that less skilled comprehenders suffer from making too many processing shifts in ordinary comprehension was supported.

A greater tendency toward processing shifts is probably itself only a symptom. The underlying mechanism(s) could be any of the following. Less skilled comprehenders may have difficulty mapping congruent information in order to develop a coherent structure or substructure. That is, instead of continuing to build onto a developing structure, less skilled comprehenders may shift and initiate an additional substructure. This difficulty could occur at the outset of comprehension when the comprehender should be laying a foundation. Or it could occur during subsequent comprehension when the comprehender should be building onto a developing structure or substructure. In addition, another component of the present framework suggests a basis for less skilled comprehenders’ mapping problems: They may have difficulty transmitting processing signals (enhancing relevant information while suppressing irrelevant information). Current work in my laboratory is investigating these possibilities.