Benefits of Sign Language Interpreting and Text Alternatives for Deaf Students' Classroom Learning

Abstract

Four experiments examined the utility of real-time text in supporting deaf students' learning from lectures in postsecondary (Experiments 1 and 2) and secondary classrooms (Experiments 3 and 4). Experiment 1 compared the effects on learning of sign language interpreting, real-time text (C-Print), and both. Real-time text alone led to significantly higher performance by deaf students than the other two conditions, but performance by deaf students in all conditions was significantly below that of hearing peers who saw lectures without any support services. Experiment 2 compared interpreting and two forms of real-time text, C-Print and Communication Access Real-Time Translation, at immediate testing and after a 1-week delay (with study notes). No significant differences among support services were obtained at either testing. Experiment 3 also failed to reveal significant effects at immediate or delayed testing in a comparison of real-time text, direct (signed) instruction, and both. Experiment 4 found no significant differences between interpreting and interpreting plus real-time text on the learning of either new words or the content of television programs. Alternative accounts of the observed pattern of results are considered, but it is concluded that neither sign language interpreting nor real-time text have any inherent, generalized advantage over the other in supporting deaf students in secondary or postsecondary settings. Providing deaf students with both services simultaneously does not appear to provide any generalized benefit, at least for the kinds of materials utilized here.

Deaf students enrolled in general educational settings frequently require classroom support services if they are to realize their academic potential. Despite decades of new ideas coming into (and passing out of) vogue, however, a variety of evidence suggests that students with significant hearing losses continue to lag behind hearing peers in a variety of academic domains and across placement settings (e.g., Karchmer & Mitchell, 2003; Kidd, Madsen, & Lamb, 1993; Stinson & Kluwin, 2003; Traxler, 2000). Schley (2005) reported data suggesting that the magnitude of the lags observed in earlier studies may have been exaggerated by selection criteria, but significant differences remain.

Some investigators have argued that the failure to reduce the difference between deaf and hearing students' academic achievement indicates that we have not yet fully elucidated differences in what they know and how they learn, and thus we have been unable to adjust our instructional methods to match (e.g., Knoors, 2005; Marschark, Convertino, & LaRock, 2006; Marschark, Convertino, McEvoy, & Masteller, 2004). This perspective has spawned renewed research into the cognitive underpinnings of learning in students with hearing loss. Meanwhile, other lines of research have focused on particular interventions designed to facilitate communication and learning in the classroom. Perhaps most apparent among these are recent efforts to provide deaf students with alternative or multiple forms of communication in the classroom via technologies such as Communication Access Real-Time Translation (CART), C-Print, and on-demand lecture notes (Stinson & Ng., 1983; Stinson, Stuckless, Henderson, & Miller, 1988; Stinson, Elliot, Kelly, & Liu, 2006; Stuckless, 1983).

Empirical interest in learning via sign language interpreting has reemerged recently, as the diaspora of deaf students to local public school classrooms unfortunately was not accompanied by progress in understanding how students learn via interpreting (Kluwin & Stewart, 2000; Marschark, Sapere, Convertino, & Seewagen, 2005a; Stewart & Kluwin, 1996). The impact of educational interpreting on achievement is just now being explored, and initial results raise questions about both its effectiveness compared to text alternatives and how both support services mesh with student communication skills. This paper concerns the convergence of these issues in exploring the impact of real-time text versus sign language interpreting on classroom learning by deaf students.

Interpreting and Technological Alternatives in the Classroom

According to the Gallaudet Research Institute (2003) Annual Survey of Deaf and Hard-of-hearing Children and Youth, which included just over 40,000 children in the United States, only 27% of the children identified attended a special school or center for deaf children. This compares to 46% of the children who were fully mainstreamed in regular public school classrooms. The latter is likely to be an underestimate, however, as many of the children not identified by the survey would likely be those who are in general education classrooms where they are the only deaf or hard-of-hearing child. It is generally agreed by those in deaf education that more than 75% of deaf students at the K–12 level now are enrolled in regular education classrooms (Karchmer & Mitchell, 2003).

At the postsecondary level, there are over 31,000 deaf and hard-of-hearing students enrolled in “mainstream” educational institutions in the United States, more than 90% of them attending classes with hearing peers. In fact, almost 50% of all 2- and 4-year institutions in the United States have identified themselves as serving at least one deaf or hard-of-hearing student, and among larger colleges and universities this number rises to around 95% (Marschark, in press). As a result, support services such as sign language interpreting and real-time text are becoming much more common in both secondary and postsecondary classrooms than they were even a decade ago.

Educational Interpreting

Sign language interpreting is an essential support service for many deaf students, but until recently little was known about how and how well deaf students learned via interpreting (see Harrington, 2000; Lang, 2002). In what appears to have been the first examination of this issue, Jacobs (1977) demonstrated that deaf college students who depended on sign language interpreting learned significantly less from classroom instruction than hearing peers. His study involved written tests, but similar findings have been obtained when learning assessments were signed (Marschark, Sapere, Convertino, Seewagen, & Maltzan, 2004). Marschark, Sapere, et al. (2004) compared learning via (American Sign Language [ASL]) interpreting and (English) transliteration by deaf college students who varied in their ASL and English-based sign language skills. Regardless of how tests were administered, there were no effects of mode of interpreting and no interactions with student language skills. Similar results were obtained by Murphy and Fleischer (1977), Marschark, Sapere, Convertino, and Seewagen (2005b), and Marschark et al. (2005; cf. Livingston, Singer, & Abramson, 1994), also with college students. More importantly, all those studies that included comparison groups have consistently replicated Jacobs' finding that deaf students learned significantly less in interpreted settings than their hearing peers.

Findings indicating that deaf students do not comprehend as much as we (or they) think they do from interpreted lectures do not appear to be the result of any methodological or demographic confounding (see Napier & Barker, 2004, for discussion). Across all the studies by Marschark and his colleagues, for example, including a meta-analysis of those studies conducted by Fabich (2005), analyses of a variety of demographic and communication variables failed to yield any consistent predictors of learning from signed lectures. In particular, those studies have not found differences in deaf college students' learning as a function of degree of hearing loss, parental hearing status, the age at which they learn to sign, their English-based signing or ASL skills, or several academic measures. Although the search for predictors continues, it appears that the heterogeneity of deaf students—even just those attending college—is such that we are unlikely to find any simple answers (Marschark, 1993).

Demonstrations of deaf students' limited learning in interpreted settings also cannot be explained by some inherent inferiority of learning via mediated instruction relative to direct instruction. Two findings are revealing in this regard. First, Marschark et al. (2005) showed that bilingual interpreters did not differ in their performance when they learned via direct instruction (93%) or mediated instruction (90%) from a hearing instructor. They used a methodology almost identical to that used in the interpreting conditions of the present experiments: Twenty interpreters saw interpreted lectures (without audio) and then received a multiple-choice assessment of learning, created in collaboration with the lecturer. Ten others took the posttest without seeing the lecture. The interpreters' near-ceiling performance after seeing the interpretation contrasted with the performance of deaf college students who scored at an average level of 53% in an almost identical interpreted condition (but including instructor voice) involving the same lectures (Experiment 1). Interpreters who did not see the lecture scored at a level (57%) similar to that of deaf students who did see the lecture. Although it may well be that the interpreters' receptive sign language skills were superior to many of the deaf students, such a situation does not mitigate the implications of those findings for deaf students' learning through interpreting in mainstream settings.

Marschark, Convertino, Macias, et al. (2006) used an even simpler methodology that examined deaf college students' comprehension of direct communication, as in classroom interactions. Students were paired according to their primary mode of communication to play Trivial Pursuit™: “strong” users of ASL, “strong”users of spoken English, and mixed (ASL–spoken English) pairs. Comprehension of simple, one-sentence questions, in face-to-face communication, occurred only 63% of the time between signing partners, as indicated by the ability to repeat the question immediately after it was signed. This number was significantly higher than that in the case of “oral” partners, who understood each other only 44% of the time when a question was spoken. This, in turn, did not differ from the comprehension between partners who did not share a mode of communication (46%). Thus, deaf students may have less than full comprehension of “through the air” communication, even when it is simple and direct.

Among other things, such findings suggest that deaf students face academic challenges beyond limited English literacy skills, and communication in the classroom is in need of further investigation. Various investigators, in fact, have discussed the relatively language-impoverished environments of young deaf children, regardless of their preferred mode of communication, and there can be little doubt of consequent impact on cognitive and academic development. Marschark et al. (2005a), however, argued that the consistent findings from interpreting studies are the product of neither student communication skills nor interpreting per se. Rather, they suggested that all the available evidence points to differences in the way that deaf and hearing students learn and the fact that interpreted lectures largely are structured by hearing teachers for hearing students (but see Experiment 3, below). As a result, information presented in mainstream classrooms often may not match the knowledge and learning styles of deaf students, regardless of how it is presented.

Findings questioning the effectiveness of classroom interpreting are quite recent. Well before them, the rush to educate deaf students in integrated, general education classrooms and the shortage and expense of qualified educational interpreters led to a search for alternatives to interpreting. Notwithstanding the well-documented reading difficulties of deaf students (e.g., Traxler, 2000), the use of printed text, with or without the aid of technology, is rapidly becoming the support service du jour, particularly in postsecondary education.

Text-Based Alternatives to Interpreting

Over the past 15 years, speech-to-text services have emerged as an alternative to interpreting to support access to information in the classroom. Speech-to-text services typically involve an intermediary operator who is often (but not necessarily) in the classroom with the deaf students. The operator produces text as it is spoken by the teacher using a stenographic machine (CART), automatic speech recognition (C-Print), or a standard keyboard (Preminger & Levitt, 1997; Stinson et al., 1999; Viable Technologies Inc., 2002). Stinson, Meath-Lang, and MacLeod (1981) were perhaps the first to examine the utility of print relative to sign language interpreting in the classroom. They found that deaf college students recalled more information when the material was presented in print rather than interpreted, although performance was quite low, ranging from 12% to 56% in various conditions. Without information concerning the reading or sign language skills of the students in that study, it is difficult to know the locus of those results. Stinson and Ng. (1983), however, found that deaf college students with better reading skills recalled more than peers with lesser reading skills following either read or interpreted lectures. This may indicate that reading level is a proxy variable for general academic acumen or simply that the two are strongly related.

Within integrated classrooms, real-time text frequently is promoted as a less expensive alternative to interpreting that also can provide greater access to the classroom for some students. Despite common assumptions, however, there is relatively little evidence that the latter assumption is true. Students report understanding more from real-time captioning than interpreting (e.g., Stinson et al., 1988), but they also think that they understand more sign language than they actually do (e.g., Marschark, Convertino, Macias, et al., 2006; Marschark, Sapere, et al., 2004). Captioning for deaf students also would appear to present a challenge for deaf students because the speed of verbatim real-time captioning is likely to exceed their reading abilities (Baker, 1985). Even controlling for reading level, Jelinek Lewis and Jackson (2001) found that deaf 4th to 6th grade deaf students still learned less from on-screen text than hearing peers, apparently because of differences in background knowledge and information processing strategies (McEvoy, Marschark, & Nelson, 1999; Strassman, 1997).

Most of the research taken as support for the utility of speech-to-text services in deaf education has been indirect. Elliot, Stinson, McKee, Everhart, and Francis (2001) and Stinson et al. (1988) surveyed college students about their use of real-time text and interpreting. Students in both studies assigned higher comprehension ratings to real-time text than to interpreting. Students in the Stinson et al. study also rated a printed hard copy of the text produced in class more favorably than handwritten notes from a notetaker. Neither study, however, assessed the accuracy of those self-reports. Similarly, the Preminger and Levitt (1997) study widely cited in support of the use of real-time text for deaf students reported that, after training, stenographers were able to transcribe conversations with multiple speakers with relatively good accuracy, but comprehension of those transcripts was not examined. Steinfeld (1998) found that captioning improved working memory performance (relative to no captioning) for both deaf and hearing students. Hearing students' memory performance still surpassed that of deaf students, however, and comprehension was not examined, despite Steinfeld's claim that “providing real-time captions improves comprehension for students who are deaf.” Other studies indicating the utility of captioning have involved hearing students who were second language learners or who had learning disabilities (e.g., Koskinen, Wilson, Gambrell, & Jensema, 1986; Neuman & Koskinen, 1992). Koskinen, Wilson, and Jensema (1986) did examine the impact of captioning on reading by deaf students. Deaf 13- to 15-year-olds in their study saw 10 repetitions of a 30-min captioned video and also received “intensive vocabulary and reading practice.” Subsequently, however, students' sight-reading of the material was reported to increase only 10%. There was no mention of increases in comprehension or transfer to other materials.

Research comparing interpreting with CART services led Stinson and his colleagues to explore the use of a lower cost, typing-based option for speech-to-text services. Their original C-Print system employs a phonetics-based word abbreviation system used by an operator who types an instructor's speech onto a laptop as it is being spoken. Approximately 3 s later, students see the text, expanded into full words, displayed on a second laptop. The word abbreviation system and training allow operators to closely capture the essence of the lecture. The system also now can be operated via automatic speech recognition using a dictation mask linked to the operator's computer. The C-Print system is being used in many school systems and postsecondary programs in the United States and other countries, but evaluation of its effectiveness for student comprehension/learning is just beginning.

Stinson et al. (2006), for example, compared deaf secondary school and college students' comprehension and memory for a lecture supported by interpreting or C-Print. No significant differences were observed between conditions among college students, but secondary school students showed reliably greater performance with C-Print. Stinson et al. (2006) suggested that the secondary school students retained more information with C-Print due to the completeness of the information, its relative permanence on the real-time display, and the printout of the C-Print notes afterward. Deaf college students' greater experience in receiving information in a variety of formats was assumed to override any potential relative benefit of the speech-to-text support.

Because hearing students were not included in the Stinson et al. (2006) study, it remains to be determined whether C-Print and similar technologies are able to “level the playing field” for deaf students in the classroom any better than interpreting. However, there are some theoretical reasons that suggest that they may not offer a complete solution. Mousavi, Low, and Sweller (1995) found that combining visual and verbal presentation of information for hearing college students was effective only when verbal information was presented auditorally (rather than visually), thus allowing students to attend to both modes simultaneously. Mayer and Morena (1998) also found that text materials were less effective for hearing students' learning than auditory presentation of verbal materials in multimedia settings. In particular, it appeared that visual presentation of two sources of information at the same time created significant impediments to information integration, an a priori difficulty also frequently exhibited by deaf learners (Marschark et al., 2006). Mayer, Heiser, and Lonn (2001) further demonstrated that when (hearing) college students were required to split their visual attention between presented text and visual supporting materials, the visual materials were “overpowered,” resulting in reduced utilization of both sources of input.

These findings suggest the possibility that using print materials and visual displays simultaneously in integrated classrooms actually may deny deaf students access to information available to hearing peers. A similar situation may be created by classrooms which include both an interpreter and textual support, potentially forcing deaf students to focus on one or the other at different times (Marschark et al., 2005). Even without a third source of visual information (e.g., slides or computer screens), students in such situations risk losing the thread of a lecture because the different information sources will be out of synchrony and they likely will be unable to predict which source is more important at any given time.

These issues were considered in the present study via four experiments that compared learning by deaf students in situations including sign language and/or real-time text displays. Experiment 1 compared learning from college-level lectures when communication was supported by interpreting, C-Print, or both. Experiment 2 provided a replication and extension of Experiment 1, as college students were presented with lectures using CART, C-Print, or interpreting. Unlike Experiment 1, but pertinent to the divided visual attention issue and more like real classrooms, instructors in Experiment 2 used visual displays in their lectures. Students also were provided with classroom notes corresponding to each condition at a delayed testing session 1 week later. Experiment 3 examined classroom learning by students in Grades 7–9 via direct instruction (signing by a deaf teacher), C-Print, or both. Supplementary printed materials (e.g., maps) were simultaneously available. Experiment 4 was a replication of Experiment 3, but instead of direct instruction, it involved sign language interpretation, as in Experiments 1 and 2. Students attending secondary schools for the deaf watched segments of two television programs, either with captions or with captions and interpretation. Their general understanding of the programs and their knowledge of previously unknown vocabulary were assessed.

Experiment 1

This experiment compared three alternative support services for deaf students in college-level classrooms: sign language interpreting, real-time text (C-Print), or both. The latter situation is not yet a common one, due to the level of resources required. It was included here because it represents a service increasingly being requested by deaf students and potentially one that could offer them better access to classroom communication. The combined service is more likely to occur as deaf students become more frequent in integrated general education classrooms because multiple deaf students in a single class may vary considerably in their language skills.

Method

Participants

The study included 95 deaf students and a comparison group of 32 hearing students, all enrolled at Rochester Institute of Technology (RIT) and paid for their participation. RIT includes the National Technical Institute for the Deaf as one of its eight colleges, but deaf students were drawn from across the university. Demographic data were available on the deaf students from institutional records, with some missing data; the number of usable data cases therefore will be indicated whenever such data are discussed. Hearing thresholds among the (79) deaf students ranged from 35 to 120 dB in the better ear with a mean of 100 dB. The one student with a 35-dB loss in the better ear had a 120-dB threshold in the other ear. Only five other students had hearing thresholds less than 80 dB (67–77 dB) in the better ear.

Materials and procedure

Two hearing members of the RIT faculty were recruited to provide introductory-level lectures, one on soil mechanics and one on visual perception. The instructors were blind to the purpose of the study beyond knowing that it involved learning by deaf and hearing students. Each digitally recorded lecture was approximately 15 min long (more recent experiments have yielded identical results with lectures up to 60 min long; Marschark, Convertino, Sapere, Pelz, & Aslin, 2006). No supplementary visual materials were used. Following recording, digital recordings were made of interpretations; for presentation purposes, the C-Print was transferred to CD-ROM via Camtasia Studio software. Both the interpretation and the transcription were provided spontaneously, as they would be in a typical classroom.

In an attempt to optimize the potential value of the support services, interpreters and real-time text operators in this and the following two experiments were chosen on the basis of recommendations from departmental managers as representing outstanding service providers with extensive experience in the classroom. Although results might vary in other settings and with less experienced personnel, the goal was to provide a best-case scenario for research purposes.

Testing was conducted in small groups, with each student seeing a single lecture. Approximately half of the participants in each condition saw each lecture. In an effort to simulate the use of these support services in actual classrooms, deaf students saw (a) the lecture and interpreting presented via video projection (life-sized), (b) the lecture via video projection with C-Print presented on a laptop, or (c) both. The C-Print real-time display presented up to 25 lines (typically about 16 lines of text) on the computer screen. Projected materials (instructors and interpreters) were shown on the front wall of the classroom in which testing occurred; audio remained on for all groups (see Marschark et al., 2005, for evidence indicating no difference between video projection and live presentation for learning in this context). Hearing students saw and heard the instructor, via a television monitor. Testing was conducted by two senior sign language interpreters (different from the one who interpreted the lectures), with extensive research experience, who used spoken language and/or sign language as appropriate.

In order to be able to take into account the fact that deaf and hearing students might enter the classroom with different amounts of knowledge concerning the lecture topics, content-specific pretests, each containing six questions, were developed in collaboration with the instructors. Postlecture learning assessments also were developed with the assistance of the instructors and contained 15 questions each. All pre-test and learning assessment questions were multiple-choice with four alternative answers.

Results and Discussion

Unless otherwise noted, all effects reported here and in subsequent experiments were significant at the .05 level, and only those effects are reported. Preliminary analyses of pretest scores indicated no significant effects of gender, lecture, or any interactions with hearing status. Overall, however, hearing students came into the classroom with more knowledge than deaf students, as indicated by their higher pretest scores, F(1, 125) = 29.31, MS_e = 443.66 (see Table 1). As a result, analyses of the learning assessment scores were conducted using analysis of covariance (ANCOVA), controlling for pretest (prior knowledge) scores.

Table 1.

Means and standard deviations of pretest and posttest (learning assessment scores) performance (proportions) for college students: Experiment 1

	Deaf (n = 95)	Hearing (n = 32)
Pretest	0.63 (0.23)	0.86 (0.13)
Learning assessment
Overall	0.60 (0.18)	0.84 (0.14)
Interpreting	0.53 (0.20)
C-Print	0.67 (0.15)
Interpreting + C-Print	0.56 (0.15)

An overall 2 (deaf vs. hearing) × 2 (lecture: soil mechanics vs. visual perception) ANCOVA indicated no effect of lecture, but hearing students learned significantly more from the lectures than their deaf peers (over 30% more), even when prior knowledge was controlled, F(1, 122) = 24.77, MS_e = 284.64 (see Table 1). In order to examine differences among the deaf students as a function of support service, an analysis of variance (ANOVA) considered those three independent groups using a 3 (interpreting, C-Print, interpreting plus C-Print) × 2 (lecture) design. This analysis yielded both a main effect of group, F(2, 89) = 7.84, MS_e = 274.37, and a Group × Lecture interaction, F(2, 89) = 4.10. Pairwise comparisons (with Bonferonni adjustment) indicated that students receiving C-Print scored significantly higher than those receiving either interpreting alone or interpreting plus C-Print (see Table 1). With regard to the difference between the C-Print alone and the interpreting plus C-Print conditions, students in the latter condition informally reported difficulty in deciding which source of information to attend to, and attempts to utilize both apparently led to lower performance than with C-Print alone (Mayer et al., 2001).

None of the simple comparisons between the two lectures in each condition was significant. The interaction appeared to be the result of the fact that the soil mechanics lecture yielded higher mean scores in the interpreting and C-Print plus interpreting conditions, whereas the visual perception lecture yielded higher scores with C-Print alone. Looking ahead, the visual perception lecture appeared to contain more new vocabulary than the soil mechanics lecture, perhaps producing the advantage for the C-Print but leaving unclear why that advantage would not have accrued in the C-Print plus interpreting condition. Interestingly, an ANOVA involving only deaf students in the two C-Print conditions indicated no significant difference between performance by students who had experience with that support service (65%) and those who did not (61%), F(1, 57)< 1.

A correlational analysis examined relations between performance of deaf students in each condition and their reading levels, using their scores on the American College Test (ACT) reading comprehension subtest (65), the Michigan Test of English Language Proficiency (52), and the California Reading Comprehension Test (64). The only case in which there was a significant correlation of reading and learning was for students in the C-Print-only condition, where higher ACT reading comprehension scores were associated with lower learning scores, r(28) = −.38. This result may be spurious but is consistent with previous results of Stinson and his colleagues showing that reading ability does not appear to be a prerequisite to utilization of C-Print, at least for college students. Hearing loss was not significantly related to learning in any condition, although there was a trend toward a negative relation (greater hearing losses associated with poorer performance) in the C-Print plus interpreting condition (r = −.41, p = .06, for left ear pure tone hearing threshold, and r = −.43, p = .07, for right ear pure tone hearing threshold).

In summary, the primary finding in this experiment was that C-Print led to significantly greater learning than interpreting or interpreting plus C-Print, with no difference between students who did and did not have prior C-Print experience. This result is consistent with student reports of perceived comprehension of real-time text over interpreting (e.g., Stinson et al., 1988) but contrasts with actual comprehension data obtained by Stinson et al. (2006). They observed an advantage for C-Print over interpreting with secondary school students but not college students. Providing both C-Print and interpreting resulted in lower performance here, perhaps because of difficulty in attempting to utilize both sources of information simultaneously. Students may well adjust to that situation over time, although results from studies of multimedia learning by hearing students suggest that they likely will end up focusing on only one source or the other. Performance of hearing students' in the present study significantly exceeded that of deaf students in all conditions, consistent with prior results from interpreting research (Jacobs, 1977; Marschark, Sapere, et al., 2004; Marschark et al., 2005, 2005b).

Experiment 2

The conflict of the findings in Experiment 1 with those of Stinson et al. (2006) and their potential implications for deaf students receiving support services in mainstream settings suggested that a replication was in order. In fact, all three of the following experiments replicated various aspects of the first. Experiment 2 provided another comparison of C-Print and interpreting but also included two important extensions. First, in addition to a C-Print condition, a second text-based condition provided students with CART. With CART, an operator inputs spoken language using a device (such as a stenotype machine, used in court reporting) that includes 24 specially arranged keys and utilizing phonetic shorthand. Functionally, C-Print and CART differ primarily in the fact that CART provides essentially a verbatim transcript, whereas C-Print tends to be condensed, preserving the majority of the content but emphasizing information deemed to be most important.

It has been suggested that the lack of verbatim reproduction of an instructor's words might be a weakness of C-Print, but there are at least three ways in which such a disadvantage might be offset. First, because they are not required to generate verbatim text and can type out words not in their abbreviation dictionary, C-Print operators may be able to produce text with fewer errors than is typically seen with CART. Second, the transcripts (notes) produced by C-Print will not be as dense as those produced by CART. Third, C-Print operators do not require the extensive training (or pay) of CART stenographers.

Although comparisons involving CART and other support services for deaf students are limited (see Steinfeld, 2001), notetaking has been shown to benefit hearing students, at least when they take the notes themselves (e.g., Kiewra & DuBois, 1988, 1991). Because of its visual demands, taking notes for oneself is not a viable option for most deaf students, and note-taking services are an important support service for many students in general education classrooms. Elliot, Foster, and Stinson (2002) found that students and teachers thought that they would benefit from the notes produced by real-time captioning, but the actual impact on learning was not assessed.

The possibility that C-Print and CART notes might be differentially useful for deaf students led to the second extension of Experiment 2. In all conditions, students returned a week after the lecture and initial testing for a second session. They were given as much time as they wanted to study lecture notes consistent with their experimental condition prior to re-taking the learning assessment test. Ideally, students would have been allowed to take the notes home for study, but insofar as actual study time could not be ascertained, this method was adopted as a first step in evaluating the utility of notes provided by the three support services.

Method

Participants

The paid participants were 60 deaf students enrolled at RIT. They had hearing thresholds ranging from 66 to 120 dB in the better ear, with a mean of 100 dB. Only four students had pure tone hearing thresholds less than 80 dB in the better ear. Twenty students were randomly assigned to each of the three delivery mode conditions. ACT scores for reading comprehension, natural science, mathematics, and the composite score were available for most of the students.

Materials, design, and procedure

As in Experiment 1, brief, introductory-level lectures were obtained from RIT faculty members, one was on granular physics and one was on imaging science. Unlike Experiment 1 but more like real classrooms, instructors used visual displays (PowerPoint presentations) in their lectures. Students saw only one of the two lectures, which were balanced over delivery modes.

The interpreter and the C-Print and (remote) CART operators participating in this experiment again were chosen on the basis of recommendations from departmental managers as representing outstanding service providers. The RIT notetaker for the interpreted lecture also was described as “outstanding” by her supervisor and selected for that reason. The CART real-time display presented up to 10 lines on the monitor at a time; the C-Print real-time display presented up to 25 lines. The interpreter, notetaker, and captionists all were blind to the purposes of the study, and notes were used exactly as they were received: handwritten by the notetaker, edited for spell-ing/grammar by the C-Print operator, and unedited by the CART operator. The handwritten notes contained 206 words for the imaging science lecture and 271 words plus 2 drawn diagrams for the physics lecture. The C-Print notes contained 1,531 words for the imaging science lecture and 1,186 for the physics lecture. The CART notes contained 2,853 words for the imaging science lecture and 2,020 words for the physics lecture.

The procedure was essentially the same as Experiment 1, except that in addition to the immediate learning assessment, students were retested a week after the lecture. At that time, they were provided with notes corresponding to their appropriate condition: handwritten notes, C-Print transcripts, or CART transcripts. They were tested after studying the notes for as long as they wished, and study time was recorded for each student. The full design was thus 3 (interpreting, C-Print, CART) × 2 (lecture) × 2 (immediate test vs. delayed test), with the latter factor within subjects. A postexperimental questionnaire asked students about their familiarity with and preferences for the three support services under investigation (which were explained fully) as well as direct instruction by a signing teacher.

Results and Discussion

A 3 × 2 ANOVA using pretest scores as the dependent variable indicated no significant effects of condition, lecture, or their interaction, Fs < 1.7, MS_e = 234.07, suggesting that the deaf students in the various conditions came into the classroom with roughly the same level of knowledge. In contrast to Experiment 1, analysis of the complete design, using learning assessment scores as the dependent variable, yielded no significant effects of condition, lecture, or their interaction, Fs< 2.8, MS_e = 576.73. Performance was higher on the delayed test (after studying the notes) than the initial testing, F(1, 54) = 4.11, MS_e = 102.68. As can be seen in Table 2, at immediate testing, learning was highest with interpreting and lowest with CART, whereas at delayed testing, learning was highest with C-Print and lowest with CART. The small and nonsignificant differences among the conditions, however, caution against drawing any conclusions at this point, other than that the three support services were equally viable for deaf college students in science classrooms.

Table 2.

Means and standard deviations of pretest and posttest (learning assessment scores) performance (proportions) for college students: Experiment 2

	Immediate test	Delayed test	Study time (min)
C-Print (n = 20)	0.72 (0.18)	0.78 (0.19)	6.90 (3.64)
CART (n = 20)	0.65 (0.22)	0.69 (0.21)	10.95 (3.55)
Interpreting
(n = 20)	0.74 (0.17)	0.76 (0.17)	4.38 (1.20)

Analysis of the amount of time that students spent studying the three kinds of notes, using an ANOVA, revealed main effects of both condition, F(2, 54) = 25.99, and lecture, F(1, 54) = 5.22, MS_e = 5.22. Not surprisingly, because the CART, C-Print, and handwritten notes were of different lengths, students studied them for correspondingly different lengths of time (see Table 2). Nevertheless, the results with regard to learning did not change when study time was statistically controlled via a 3 × 2 ANCOVA, F(1, 56) < 1.5, MS_e = 359.47.

On the questionnaire about support services, 9 students expressed a preference for CART and 19 for C-Print. Only three of the 60 students had used CART previously, however, and only 9 had used C-Print. (All had used interpreting previously.) Neither immediate nor delayed learning scores were affected significantly by whether students had previous experience with CART or C-Print, and they were not related to stated preferences for support services. Although the numbers of students with relevant experience are too small to have great confidence in them, these results are essentially the same as those obtained in Experiment 1. With the exception of a marginal relationship, r(16) = .47, p < .06, with performance in the CART condition at immediate testing, ACT reading comprehension scores were significantly and positively related to learning assessment scores in all three conditions, rs(12–16) ≥ .55, contrasting with Experiment 1. However, reading comprehension scores were not related to pretest performance in any condition. Those analyses thus did not distinguish performance across the three groups.

In summary, learning was essentially the same when supported by C-Print, CART, or interpreting in this experiment, although Table 2 shows a trend toward CART yielding lower performance at both immediate and delayed testing. Providing students with study notes apparently increased their performance from 2% to 7% across the conditions, which did not differ reliably, F(2, 57) < 1. Alternatively, the minimal increment in scores may simply reflect hypermnesia, that is, the greater recall of verbal materials after a delay relative to immediate testing (see Experiment 3). In any case, allowing students to study the notes had little effect on their scores and did not indicate an advantage for any particular support/note service.

Experiment 2 thus leaves us with an apparent contradiction relative to Experiment 1: C-Print and interpreting were fully comparable in their support for deaf students' learning of science-related material here, whereas C-Print led to better scores in Experiment 1. This variability and the inconsistent findings from previous C-Print research (Elliot et al., 2001; Stinson & Ng., 1983; Stinson et al., 2006) may simply result from the heterogeneity typically found among deaf students. Nevertheless, Experiment 3 provided an additional replication comparing learning via sign language and real-time text, this time involving secondary school students who received C-Print, direct instruction (i.e., sign language only), or both.

Experiment 3

Experiment 3 was conducted in a bilingual school that provides educational programs for deaf children in Grades K–10. Located within the Royal Institute for Deaf and Blind Children, the school is registered as a general education program and follows a full, regular curriculum. The language of instruction is Australian Sign Language (Auslan), but the school seeks to facilitate English literacy for use in curricular contexts. An ongoing project involves use of real-time text in the classroom in order to allow students (particularly secondary school students) access to the content of instruction through both Auslan and English. Both C-Print (via keyboard) and voice recognition software are used in various contexts. This bilingual method of instruction is intended to facilitate comprehension of lesson content as well as to provide younger students with the ability to make use of real-time captioning.

Method

Participants, design, and procedure

The study included 15 deaf students aged 12–16 years, enrolled in Grades 7–9. Hearing thresholds among the students ranged from 80 to 120 dB in the better ear with a mean of 104 dB. Each student was tested initially over a 1-week period, on every other day, within regular school hours, once in each of three delivery mode conditions: (a) a deaf teacher signing in Auslan, (b) the teacher signing in Auslan with simultaneous real-time text, and (c) real-time text alone. Three groups of students were exposed to the three modes of delivery for different lectures, so that each student received all three formats across the three lessons. At any given testing session, two thirds of the class received real-time text via laptop. Up to 10 lines of captioning could appear at once. As depicted in Figure 1, half of the students receiving real-time text sat facing the signing teacher (Auslan plus text), whereas the other half sat facing away from the teacher (text only). The remaining third of the students sat at desks so that they viewed the lecture in Auslan only.

Figure 1.

Classroom layout for Auslan-only, real-time-text-only, and Auslan plus real-time text conditions of Experiment 3.

The lessons consisted of three 20-min lectures on geography. The material was new for the students, appropriate for all involved year levels, and similar in general subject, consisting of one lecture each on London, Jakarta, and Los Angeles. Similar content was addressed for each city. Each of the lessons contained approximately the same amount of new vocabulary. Unlike the previous two experiments, the lectures were previewed and rehearsed by the real-time text operator for each lecture. During the lectures, the teacher introduced some new words. In the first instance, the Auslan sign was given, followed by the finger spelling. On all subsequent occasions, only the Auslan sign was presented. The texts were set for the screen to be scrolled at a reasonable pace, and at the end of each paragraph the operator captioned students' questions and the teacher's answers before they went on to the next paragraph.

Immediately after each lesson, the students were tested on its content. As in Experiment 2, they also were retested a week later (but without notes). The questions on the learning assessment were designed to be approximately equal in difficulty across lectures and to eliminate possible effects of prior knowledge of the content. The assessment for each lesson contained 10 questions requiring children to “match the word to its meaning,” 5 that required them to “fill in the missing word,” 5 multiple-choice questions, and 5 “true or false” questions. The same 25-item questionnaires were used for the delayed test. Reading scores for all students were available from the Woodcock Reading Mastery Test (Reading Comprehension Cluster). To control for grade-related differences in reading ability, standard scores were used in analyses reported below.

Results and Discussion

Due to attrition, there were five missing observations on delayed tests, and those were replaced with cell means. Preliminary analyses indicated no main effect of gender and no interactions with it. As can be seen in Table 3, mode of delivery was unrelated to learning, as differences in scores across modes were negligible at both immediate and delayed testing. Analysis of the 3 (grade) × 3 (delivery mode) × 2 (test) design, in which the last two factors are within subjects, accordingly indicated no main effect or interactions involving mode of delivery, F(1, 12) < 1, MS_e = 0.021. There was a significant effect of grade, F(2, 12) = 9.31, MS_e = 0.06, as the 9th graders significantly outperformed the 7th and 8th graders, who did not differ. Grade did not interact reliably with any other factor, however. Performance was higher at the first testing (0.41) than the second testing (0.30), F(1, 12) = 14.90, MS_e = 0.02, but there were no interactions involving test time. This result suggests that the improvement in delayed test scores in Experiment 2 was the result of students' having lecture notes available and not a hypermnesia effect. All these results were confirmed using nonparametric statistics because of the small sample sizes. Parametric and nonparametric analyses involving question type failed to yield any significant main effects or interactions.

Table 3.

Means and standard deviations of learning assessment scores (proportions) for secondary school students in Experiment 3 (within-subjects design, n = 15)

	Immediate test	Delayed test
Auslan only	0.44 (0.20)	0.30 (0.18)
Auslan + C-Print	0.45 (0.25)	0.34 (0.16)
C-Print only	0.40 (0.19)	0.31 (0.13)

A correlational analysis between reading scores and test performance in each condition yielded significant coefficients within each delivery mode at both immediate and delayed testing, rs(12–14) ≥ .61. As in Experiment 2, these analyses did not distinguish performance across the conditions. In particular, better reading skills among secondary school students did not appear uniquely associated with greater benefits from real-time text (cf. Stinson et al., 2006). Reading level thus appears to have been simply a proxy variable for general academic or cognitive ability.

The results of Experiment 3 are consistent with some previous findings and inconsistent with others. Most notably, the advantage of C-Print alone observed in Experiment 1 with college students and by Stinson et al. (2006) with secondary school (but not college) students was not replicated here. It is noteworthy that scores across the three delivery modes were relatively low, ranging from 30% to 45%. Although no hearing comparison group was included, the comprehension scores observed here are consistent with previous results concerning deaf students' classroom learning with interpreting and real-time text (e.g., Jacobs, 1977; Livingston et al., 1994; Marschark, Sapere, et al., 2004; Murphy & Fleischer, 1979; Stinson et al., 1981). Finally, although it is commonly asserted that direct instruction is inherently superior to mediated instruction, Experiment 3 did not support this claim.

Experiment 4

The results of Experiment 3, although consistent with the majority of findings from studies involving college students, appears to be inconsistent with the study of Stinson et al. (2006), that found that secondary school students learned more via real-time text than via sign language. The Stinson et al. study, however, involved sign language interpreting, whereas Experiment 3 involved direct (signed) instruction by a deaf teacher. Experiment 4 therefore provided replication of Experiment 3, but like Experiment 1 and the Stinson et al. study, it involved interpreting rather than direct instruction. The experiment was conducted in the Netherlands, thus further extending the international generality of our findings.

Method

Participants, design, and procedure

The participants were 28 students recruited from two secondary schools for the deaf in the Netherlands. Selection criteria were a pure tone average hearing threshold of at least 90 dB, normal nonverbal intelligence, educated in a bilingual program, and a reasonable-to-good level of reading comprehension as judged by teachers of the deaf. The mean age of the students was 14.9 years; the mean pure tone average hearing threshold was 109 dB.

The deaf students watched segments of two television programs drawn from the daily News for the Youth (Jeugdjournaal), which is broadcast on one of the national Dutch television channels. This program was chosen because many young people in the Netherlands are familiar with it, and the program often is used for teaching purposes in the secondary education of deaf students. The program normally is broadcast with captions and sign language interpretation. In this experiment, additional versions of the segments involved captioning without interpreting (Sign Language of the Netherlands, NGT). Students watched one segment with captioning only and one segment with simultaneous captioning and on-screen sign language interpreting. The interpreting for Feugdjournaal is done by some of the most experienced NGT interpreters in the Netherlands. The two segments used were entitled “Weapons in Iraq” and “Speed Controls in Rotterdam.”

Looking ahead, it may be noteworthy that the superimpositioning of both captioning and interpreting on the television screen offers a somewhat different situation than any of the previous experiments, where students' gaze would have had to traverse significantly greater distances in order to shift from one source of information to the other (see Marschark et al., 2005). The previous experiments used a methodology comparable to the manner in which these support services are offered in the classroom. Experiment 4, in contrast, used a methodology comparable to that typical in television programming (e.g., Jelinek Lewis & Jackson, 2001).

Following their watching of each segment, students answered multiple-choice questions, each with four alternative answers. There were a total of 29 multiple-choice questions, 19 about word meanings and 10 about content. As a control for prior content knowledge, students' familiarity with vocabulary in the programs was assessed prior to the experiment. Students were asked to indicate whether they knew (yes/no) the meanings of “words” on a list containing 19 target words, 19 Dutch pseudowords, and 19 nonwords. The students reported recognizing an average of 12.46 words (SD = 4.66), 8.86 pseudowords (SD = 5.15), and 1.89 nonwords (SD = 3.81). On the basis of this result, it was concluded that the deaf students did not simply guess when they were asked to judge whether a word was familiar. We did not take this to mean that they really knew the meanings of the words, however. After testing, students' judgments of word familiarity were compared to their responses (correct or incorrect) on the multiple-choice test of word meanings and a ratio was calculated: the number of words indicated as familiar divided by the number of words responded to correctly. Because these ratios were significantly different for the two groups, t(26) = 2.24, they were used as a measure of prior word knowledge in subsequent ANCOVAs. The ANCOVAs also controlled for age.

Results and Discussion

As can be seen in Table 4, when both captioning and sign language interpretation were available, the deaf students learned somewhat more new word meanings and understood the content of the program somewhat better than with captioning alone. When corrected for prior content word knowledge via ANCOVAs, however, neither the difference in the number of word meanings learned, F(1, 13) < 1, MS_e = 0.00, nor the difference in content learned in the two conditions was significant, F(1, 13) = 1.75, MS_e = 0.40. These results largely replicate the findings from Experiment 3, which involved real-time text and direct (sign language) instruction, in failing to find significant differences in learning by secondary school students when supported by real-time text versus real-time text plus sign language.

Table 4.

Means and standard deviations of learning assessment scores (proportions) for secondary school students in Experiment 4

	Captions (n = 14)	Captions + sign language (n = 14)
New words	0.60 (0.20)	0.74 (0.17)
Content questions	0.73 (0.19)	0.85 (0.09)

General Discussion

Four experiments examined the effects of real-time text services and sign language interpreting in supporting deaf students' access to information in the classroom. Experiment 1, which compared sign language interpreting to C-Print and to C-Print plus interpreting, indicated a reliable advantage for C-Print alone over the other two conditions. Three other experiments, however, failed to replicate the real-time text advantage. Experiment 2 found no significant differences among interpreting, C-Print, and CART, either in immediate testing or testing 1 week later. At the delayed testing, students studied notes generated in the three classroom formats, and performance increased 2–7% over the immediate test. The third and fourth experiments also failed to demonstrate any advantage to learning when real-time text was provided, either instead of or in addition to sign language for secondary school students.

One possible explanation for the observed pattern of results—the advantage of real-time text only in Experiment 1—is that it could have resulted from the fact that those lectures appeared to have more new technical vocabulary than the other experiments. Consistent with this suggestion, the lecture in Experiment 1 that yielded higher mean scores in the C-Print alone condition appeared to have more new vocabulary than the one that yielded higher scores in the other conditions. Investigation of the effects of complexity, academic level, language abilities, and novelty in classroom content is needed, in any case, as these issues have not been examined previously with either sign language interpreting or real-time text.

An alternative possibility is that the advantage for real-time text observed in Experiment 1 was simply a product of the considerable variability found among deaf students in a variety of academic domains (Marschark, 1993). Given all the results obtained here, this possibility appears the most likely to be correct and is a reminder of the danger of accepting results from single-experiment studies involving heterogeneous populations. Anecdotal reports and a priori assumptions about the viability of various support services should not preclude a healthy skepticism toward what appears to be simple solutions to our efforts to enhance deaf students' academic achievement. Until recently, for example, we assumed that high-quality sign language interpreting “leveled the playing field” for deaf students in mainstream settings, an assumption now known to be incorrect (Marschark et al., 2005, 2005b). We have accepted students' claims that one support service or another was superior for learning, even though their metacomprehension skills make such claims suspect (Marschark, Sapere, et al., 2004; Marschark et al., 2006). In fact, the general lack of benefit gained from real-time text in these experiments stands in marked contrast to student ratings in previous studies, where they have claimed that they understood more with text than interpreting (e.g., Elliot et al., 2001; Stinson et al., 1988).

Supporters of using real-time text to aid in the development of deaf students' bilingual skills might note that deaf students in Experiments 1 and 3 did equally well when they were presented with signing only (interpretation or direct instruction) and signing plus real-time text. Optimistically, this result might be taken to suggest that, with the benefit of sustained practice with real-time text, including exposure to a dual presentation of information in sign language and captions, students may find the presentation of information in text-only format to be equally effective for accessing classroom content. It is possible, for example, that over time, students might learn to quickly locate the pertinent information in the real-time display when they are not certain of a word signed or finger spelled by an interpreter (e.g., an unfamiliar technical term).

This suggestion is consistent with the results of Experiment 4, but there is also at least one piece of evidence that cautions against this conclusion. Recall that in Experiment 2, examination of the relations between learning assessment scores and students' experience with the support services did not indicate that “practice makes perfect.” Likely because of the small sample sizes, none of the correlations were significant, but the results were not encouraging. With C-Print, learning at immediate and delayed testing was negatively correlated with experience with that service, rs(9) = −.15 and −.54; similar results were obtained with CART, rs(3) = −.33 and −.88. Only experience with interpreters was positively (although still not significantly) related to learning, rs(20) = .21 and .24, for immediate and delayed testing, respectively.

Deaf students' use of notes produced by a notetaker or with C-Print also needs further study. In Experiment 2, students did not receive the notes until just prior to the delayed testing. It thus remains unclear how their retention of information might differ depending on whether they review their own notes or those from a notetaker or real-time text services and how this would compare to hearing students. Osguthorpe, Long, and Ellsworth (1980) found that reviewing notes taken by a notetaker was much more beneficial to deaf students (who presumably were accustomed to using others' notes) than to hearing students, although the hearing group still outperformed the deaf group. A second question pertains to relations among students' choice of notetaking services, support services provided outside of the classroom, and actual learning. Experiment 2 found no performance differences among the three types of notes, but this finding may or may not generalize to everyday practice over multiple classes. Further studies should consider the effectiveness of alternative kinds of notes (e.g., matrices) for supporting deaf students' learning in different content areas and at different academic levels, as well as the extent to which variation in the quality of notes affects learning (Elliot et al., 2002).

Another issue still to be resolved is the extent to which attending to multiple sources of visual information in the classroom affects deaf students' learning. Variables such as the “density” of information in the different sources, the relative importance of information in the two or more sources to comprehension/learning, their visual quality and location in space, and possibly the distance between them all might be relevant. These issues currently are under investigation using eye-tracking technology in college classrooms. Our first such study, for example, showed that video presentation of course materials supported by sign language interpreting led to deaf students' learning just as much as with live presentation (Marschark et al., 2005). That study also showed that although deaf students frequently have to shift their visual attention from one source of information to another (instructor–interpreter–visual display), there were no apparent differences in the patterns of gaze allocation between deaf students who are skilled users of ASL and those who depend on spoken language. Both groups differed relative to hearing nonsigners, however, in the patterns of gaze allocation observed. Considerably more research is needed on such issues, however, to understand how deaf students function in multimedia classroom and determine ways to provide them with optimal access to visually presented information in such settings.

In summary, the present findings, combined with other recent studies, suggest that there is no inherent advantage or disadvantage to print materials (C-Print or CART) relative to high-quality sign language in the classroom. Importantly, all four of the present experiments were conducted under optimal conditions, whereas support services encountered in “real” classrooms may be far from ideal. Further investigation is needed to determine how the quality of support services affects learning at different age levels and with different materials.

Consistent with some prior studies (e.g., Stinson & Ng., 1983), the present experiments did not find any unique relation between reading skills and access to classroom information via print media. This result appears surprising because of the recognized challenges of print literacy for most deaf students. One might expect that sign language would provide an advantage over print materials, but it appears that the issue is one of language comprehension, not one of reading per se (Marschark et al., 2005a). Relations between reading and learning via C-Print have been observed previously with secondary school students (Stinson et al., 2006), but the settings and materials were different, suggesting the need for more comprehensive evaluation with students who have less well-developed literacy and sign language skills.

Those interested in the education of deaf students, both in K–12 settings and at the postsecondary level, have argued for the advantages inherent in various support services relative to others. If the lack of consistency favoring any particular support service in the present results is disappointing to them, it also should be a warning that the various claims that interpreting, C-Print, CART, or direct instruction offers some panacea are sorely mistaken. The relatively poor performance of deaf students across all conditions in Experiments 1–3 indicates that we have not yet arrived at the appropriate “tricky mix” (Knoors, 2006; Nelson, Loncke, & Camarata, 1993) that will provide them with full access to educational opportunities. Still, research in these areas is progressing. As we better understand the cognitive foundations of learning by deaf students and the challenges presented by various educational settings, we will be better able to match instructional methods and support services to students' strengths and needs. Only then can we provide them with full access to information in the classroom and optimal opportunities for lifelong learning.