Video Captions Benefit Everyone

Abstract

Video captions, also known as same-language subtitles, benefit everyone who watches videos (children, adolescents, college students, and adults). More than 100 empirical studies document that captioning a video improves comprehension of, attention to, and memory for the video. Captions are particularly beneficial for persons watching videos in their non-native language, for children and adults learning to read, and for persons who are D/deaf or hard of hearing. However, despite U.S. laws, which require captioning in most workplace and educational contexts, many video audiences and video creators are naïve about the legal mandate to caption, much less the empirical benefit of captions.

Introduction

Imagine a technique that can improve children’s reading skills (Linebarger, Piotrowski, & Greenwood, 2010), boost adolescents’ written and spoken vocabulary (Davey & Parkhill, 2012), increase college students’ attention to lectures (Steinfeld, 1998), enhance second-language learners’ pronunciation (Mitterer & McQueen, 2009), and raise literacy rates in developing countries (Kothari, Takeda, Joshi, & Pandey, 2002). The technique is simple: Display captions on videos.

Captions are like foreign-language subtitles; they translate a spoken language into a written language (Garza, 1991). Like foreign-language subtitles, captions appear at the bottom of the screen. Unlike foreign-language subtitles, captions translate into writing the same language that is heard in speaking, which is why captions are also called same-language subtitles. Captions also translate sound effects (“raindrops falling,” “footsteps approaching,” “horses galloping”); captions transcribe song lyrics, and captions offer other helpful clues, such as identifying conversational partners by their name and indicating off-screen voices with italics.

More than 100 empirical studies, listed in the appendix, document the benefits of captions. These studies report benefits to a wide swath of participants as measured by a wide swath of criteria: summarizing main ideas (Markham, 2000–2001), recalling facts (Brasel & Gips, 2014), drawing inferences (Linebarger et al., 2010), defining words (Griffin & Dumestre, 1992–1993), identifying emotions (Murphy-Berman & Whobrey, 1983), and of course, answering multiple-choice comprehension questions (Hinkin, Harris, & Miranda, 2014; Markham & Peter, 2002–2003; Murphy-Berman & Jorgensen, 1980).

Eye-movement studies document that captions are read easily (d’Ydewalle & de Bruycker, 2007), attended to effortlessly (d’Ydewalle, Praet, Verfaillie, & van Rensbergen, 1991), and integrated smoothly with the soundtrack of the video (d’Ydewalle & Gielen, 1992). Standard verbatim captions are as effective as more detailed or elaborated captions (Anderson-Inman, Terrazas-Arellanes, & Slabin, 2009; Murphy-Berman & Jorgensen, 1980).

The numerous empirical studies referenced in the appendix demonstrate that captions benefit everyone who watches videos, from younger children to older adults. Captions are particularly beneficial to persons watching videos in their non-native language, children and adults learning to read, and persons who are D/deaf or hard of hearing, as illustrated below.

Captions Benefit Persons Who Are D/deaf or Hard of Hearing

The early 20th century’s golden age of cinema had created a level playing field for D/deaf and hard of hearing viewers. Silent films, with their interwoven screens of captions (called intertitles), created “the one brief time that deaf and hard of hearing citizens had comparatively equal access to motion pictures” (Schuchman, 2004, p. 231). But in the late 1920s, as talkies (films with synchronized speech) pushed out silent films, the D/deaf community was shut out.

In response, the D/deaf community created captions (Downey, 2010), first by recapitulating the intertitles of the silent film era and then by reconfiguring the bottom-of-the-screen foreign-language subtitles that carried U.S. films across the world. In the late 1950s, U.S. President Eisenhower authorized a federal Captioned Films for the Deaf agency (as “part of the post-Sputnik, cold war education boom,” Downey, 2008, p. 193).

Captions began appearing on television shows in the 1970s (with their earliest appearances on ABC’s Mod Squad and PBS’s The French Chef; Withrow, 1994). In the 1980s, a handful of television shows began displaying captions in real time (e.g., the launch of the space shuttle Columbia and the acceptance speeches at the Academy Awards; Block & Okrand, 1983). By the 1990s, captions on TV shows were mandated by the U.S. law (Erath & Larkin, 2004). The Twenty-First Century Communications and Video Accessibility Act of 2010 requires that captioned TV shows also be captioned when displayed on the Internet.

It is unsurprising that captions benefit persons who are D/deaf or hard of hearing. But early experiments demonstrating that captions benefit D/deaf persons demonstrated something further: Captions also benefit hearing persons. For example, Figure 1 displays the results of a study by Nugent (1983). More than 30 D/deaf children and nearly 100 hearing children (9–14 years old) were randomly assigned to one of four conditions: watch a video with audio but without captions; read only the captions; watch the video with audio and with captions; or read and watch nothing, thereby serving as a control group.

Figure 1.

Data from Nugent (1983).

The children’s scores on a 23-item comprehension test are illustrated in Figure 1. Statistical analyses identified two main effects: a main effect of hearing status (hearing children scored higher on the comprehension test than D/deaf children) and a second, even more powerful, main effect of captioning. A lack of a statistical interaction between hearing status and captioning indicated that captions were as beneficial to the hearing children as they were to the D/deaf children.

Several other studies demonstrate the same effect: Video with audio and with captions leads to the highest levels of comprehension, both for D/deaf children and for hearing children (Anderson-Inman et al., 2009; Boyd & Vader, 1972; Cambra, Leal, & Silvestre, 2010; Fischer, 1971; Gulliver & Ghinea, 2003; Hertzog, Stinson, & Keiffer, 1989; Murphy-Berman & Jorgensen, 1980; Murphy-Berman & Whobrey, 1983; Nugent, 1983; Steinfeld, 1998; Yoon & Choi, 2010).

Captions Benefit Hearing Children Learning to Read

Even for hearing children, learning to read is a complex process, which requires learning to map sound and meaning onto text (Linebarger, 2001). Soon after captions began appearing on TV shows for D/deaf audiences, educators of hearing children made a striking discovery: Because captions explicitly illustrate the mapping among sound, meaning, and text, captions could also benefit hearing children learning to read (Adler, 1985; Kirkland, Byrom, MacDougall, & Corcoran, 1995; Koskinen, Wilson, & Jensema, 1986; Parkhill, Johnson, & Bates, 2011).

For example, Figure 2 displays the results of a study of 70 hearing children learning to read (Linebarger et al., 2010). Second and third graders were randomly assigned either to watch videos with audio but without captions or to watch videos with audio and with captions. The children watched six ½-hr videos, which were episodes of PBS children’s shows (e.g., Arthur & Friends, Magic School Bus, Zoom).

Figure 2.

Data from Linebarger, Piotrowski, and Greenwood (2010).

As Figure 2 illustrates, watching videos with audio and captions leads to significantly better reading skills. Children who watch captioned videos are better able to define content words that were heard in the videos, pronounce novel words, recognize vocabulary items (which may or may not have been heard in the videos), and draw inferences about what happened in the videos. Other studies demonstrate cumulative benefits from watching videos with captions, for example, cumulative growth in vocabulary both for hearing children (Koskinen et al., 1986) and for hearing adults (Griffin & Dumestre, 1992–1993).

Captions Benefit Hearing Adults

After discovering that captions benefit hearing children learning to read, researchers investigated whether captions also benefit hearing adults learning to read. They do (Koskinen, Knable, Markham, Jensema, & Kane, 1995–1996; Kothari, Pandey, & Chudgar, 2004; Kruger, Kruger, & Verhoef, 2007).

For example, in the late 1990s, researchers encouraged India’s national television network to begin captioning popular Bollywood music videos, which were sung and captioned in Hindi. The literacy of thousands of adults was assessed before the captioned music videos began airing and several years later. The literacy of adults who frequently watched the captioned videos increased at a much greater pace than the literacy of adults who rarely or never watched the captioned videos (Kothari & Bandyopadhyay, 2014).

Even highly literate adults benefit from captions. For example, when highly literate adults watch television commercials that are captioned, they remember brand names better (Brasel & Gips, 2014), and when highly literate college students watch course lectures that are captioned, they remember course content better (Steinfeld, 1998). Captions benefit hearing adults, just as captions benefit hearing children.

Captions Benefit Hearing Persons Learning a Second Language

Captions for D/deaf persons were co-opted from foreign- language subtitles for hearing persons. In the early 1980s, as captions for D/deaf persons became more prominent, second-language instructors began re-co-opting captions for hearing persons, to improve second-language literacy (Price, 1983; Vanderplank, 2013). Scores of studies demonstrate that captions in a second language benefit hearing persons learning that second language; indeed, captions in a second language benefit hearing persons learning that second language even more than captions in the persons’ native language.

For example, Figure 3 displays the results from nearly 150 Japanese junior college and university students learning English as a second language (Yoshino, Kano, & Akahori, 2000). The students watched three types of videos: videos with English audio but without any captions, videos with English audio and Japanese captions, videos with English audio and English captions. In a fourth condition, the students listened to only the English audio.

Figure 3.

Data from Yoshino, Kano, and Akahori (2000).

After watching each type of video (or listening to only the audio) twice, in counter-balanced order, the students recalled as much content as they could using either Japanese and English. The students recalled substantially more content after they watched the videos with English captions than after they watched the same videos with Japanese captions. In fact, after watching the videos with Japanese captions, the students recalled as little as they recalled after not even watching the videos (the audio only condition).

Captions (same-language subtitles) also improve second-language learners’ listening comprehension. Figure 4 displays data from University of Southern California students learning English as a second language (Huang & Eskey, 1999–2000). The students were randomly assigned to watch videos with English audio and English captions or with English audio but without captions. Watching videos with English captions not only improved the students’ performance when tested with a written comprehension test, but also improved the students’ performance when tested with an auditory, listening, comprehension test.

Figure 4.

Data from Huang and Eskey (1999–2000).

Captions benefit hearing persons learning a second language, regardless of genre. Figure 5a displays data from 70 college students learning English as a second language, and Figure 5b displays data from 40 English-speaking college students learning Russian as a second language (Garza, 1991). The students learning English as a second language were randomly assigned to watch videos with English audio and with or without English captions. The students learning Russian as a second language were randomly assignment to watch videos with Russian audio and with or without Russian captions.

Figure 5.

Data from Garza (1991).

As both Figures 5a and 5b illustrate, watching videos with same-language captions leads to significantly better comprehension. Captions benefit comprehension, regardless of the language being learned (Russian or English) and regardless of the genre being watched, from documentaries (The Sharks) to dramas (Hoosiers) to animations (An American Tail) to comedies (The Secret of My Success) to music videos (The Authority Song).

What Are the Policy Implications?

The empirical evidence is clear: Captions, also known as same-language subtitles, benefit everyone who watches videos. More than 100 studies document that captioning a video improves comprehension of, memory for, and attention to videos, for children, adolescents, college students, and adults. Although captions particularly benefit persons watching videos in their non-native language, children and adults learning to read, and persons who are D/deaf or hard of hearing, captions also benefit highly literate, hearing adults.

With so many studies documenting the benefits of captions, why does everyone not always turn on the captions every time they watch a video? Regrettably, the benefits of captions are not widely known. Some researchers are unaware of the wide-ranging benefits of captions because the empirical evidence is published across separate literatures (deaf education, second-language learning, adult literacy, and reading acquisition). Bringing together these separate literatures is the primary purpose of this article.

Reaping the benefits of captions is also impeded by erroneous attitudes (e.g., Weasenforth, 1994). Many people think captions are intended for, and therefore only beneficial to, persons who are D/deaf. For example, in a survey of several hundred K-12 educators across 45 U.S. states, almost all of whom were experienced teachers who frequently showed videos in their classroom, the majority had never turned on the captions on those videos. The minority who had, reported their students having reaped benefits from the captions (Bowe & Kaufman, 2001).

Similarly, faculty and administrators in higher education are unlikely to be aware of the benefits of captions for university students, despite the fact that captions perfectly illustrate the fundamental principle of Universal Design. Like curb cuts and elevators, captions were initially developed for persons with disabilities, and, like curb cuts and elevators, captions benefit persons with and without disabilities. Indeed, the overwhelmingly vast majority of persons who benefit from curb cuts and elevators are not persons with disabilities, and the same could be true for captions.

The Institute of International Education reports that international students are enrolling in U.S. colleges and universities at an all-time high, a whopping 72% increase in only the past decade. Nearly a third of the international students studying in the United States are from China (Redden, 2014). Given the increasing number of students in U.S. institutions of higher education who are not native English speakers and given the powerful benefits of captions to non-native speakers, it would behoove professors to turn on captions.

Unfortunately, a primary reason that everyone who watches videos is not benefitting from captions is that not all videos are captioned. Despite U.S. laws, which cover many workplace and educational contexts, many video audiences and video creators are naïve about the legal mandate to caption, much less the empirical benefit of captions. Some organizations rely solely on automatically generated captions (e.g., the auto- generated captions found on many YouTube videos).

However, as recent litigation (Orzeck, 2015) as well as empirical data (Pan, Jiang, Yao, Picheny, & Qin, 2010) demonstrate, captions generated via automated speech recognition are not yet without interfering error. When auto-generated captions reach parity with human-transcribed captions, further technologies, including real-time captioning of lectures for all students (Bain, Basson, Faisman, & Kanevsky, 2005), will be able to harness the power of captions for the broadest population ever.

Key Points.

• Captions benefit everyone who watches videos, from younger children to older adults.

• Captions are particularly beneficial to persons watching videos in their non-native language, children and adults learning to read, and persons who are D/deaf or hard of hearing.

• Captions generated via automated speech recognition are not yet without interfering error, but when auto-generated captions reach parity with human-transcribed captions, technology will be able to harness the power of captions.

• Despite U.S. laws, which require captioning in most workplace and educational contexts, many video audiences and video creators are naïve about the legal mandate to caption, much less the empirical benefit of captions.