Benefits and Pitfalls of Multimedia and Interactive Features in Technology-Enhanced Storybooks

Abstract

A meta-analysis was conducted on the effects of technology-enhanced stories for young children’s literacy development when compared to listening to stories in more traditional settings like storybook reading. A small but significant additional benefit of technology was found for story comprehension (g+ = 0.17) and expressive vocabulary (g+ = 0.20), based on data from 2,147 children in 43 studies. When investigating the different characteristics of technology-enhanced stories, multimedia features like animated pictures, music, and sound effects were found beneficial. In contrast, interactive elements like hotspots, games, and dictionaries were found to be distracting. Especially for children disadvantaged because of less stimulating family environments, multimedia features were helpful and interactive features were detrimental. Findings are discussed from the perspective of cognitive processing theories.

There is no doubt that stories read to young children are one of the most important sources of literacy development (Bus, van IJzendoorn, & Pellegrini, 1995; Mol & Bus, 2011). Listening to stories, children expand their story comprehension skills and acquire sophisticated language in addition to code-related skills such as phonological awareness or concepts of print. With the emergence of technology in homes and school settings, children can watch a narrative on television, on the computer using a CD-ROM or DVD, or on the Internet; and more recently, they can use a tablet or a smartphone (e.g., apps on the iPad or the iPhone) to access stories. Television only allows for multimedia features (like animated illustrations in addition to music and sound effects); in contrast, it is possible for stories on the computer or tablets to involve the child in the story through interactive features such as questions, dictionaries, games, animations, or sounds to be activated by clicking on or touching a spot in an illustration (often indicated as hotspots).

The Joan Ganz Cooney Center analyzed the 137 most popular American electronic books (e-books) for young children in 2012 (Guernsey, Levine, Chiong, & Severns, 2012) and found that 75% of the e-books included hotspots and 65% included game-like activities. Only about 20% of hotspots and a quarter of the games were related to the story. From the perspective of information processing, this shift from listening to a story to playing during listening might require the child to continuously switch between listening and playing, which could have serious consequences for story comprehension and learning as a result of cognitive overload (Bus, Takacs, & Kegel, 2014).

At the same time, it has been suggested that technology-enhanced stories will enhance children’s comprehension of stories (Salmon, 2014; Zucker, Moody, & McKenna, 2009). Multimedia additions provide nonverbal information that might help story comprehension by visualizing story events congruent with the narration (Sharp et al., 1995; Verhallen, Bus, & de Jong, 2006). Similarly, interactive features that are relevant to the story (e.g., a hotspot with a question that is tightly connected to the story) or aimed at developing literacy skills (e.g., an alphabet game) might enhance the effects of listening to a story (Segers, Nooijen, & de Moor, 2006; Shamir, Korat, & Fellah, 2012; Smeets & Bus, 2014). Additionally, technology-enhanced stories may be more engaging for children in comparison to print storybooks (Adam & Wild, 1997; Chiong, Ree, Takeuchi, & Erickson, 2012; Moody, Justice, & Cabell, 2010; Okolo & Hayes, 1996), especially during repeated readings (Verhallen & Bus, 2009a).

For the purposes of the present meta-analysis of technology-enhanced stories, the effects of different devices and platforms were ignored (see Roskos & Burnstein, 2013, for a study on the role of devices). Instead, the effects of multimedia and interactive elements were examined. Furthermore, the effect of technology was investigated as a function of children’s risk status, because it has been suggested that multimedia may be especially beneficial in risk groups (Kamil, Intractor, & Kim, 2000).

Multimedia Features

The visual superiority hypothesis assumes that salient visual information presented in television programs distracts children from the verbal stimuli (e.g., narration or conversation). This hypothesis, however, has not been confirmed. Research has shown that children pay attention to the verbal information when it is congruent with the visual information (for reviews see Bus et al., 2014; Rolandelli, 1989). However, we still do not know if a presentation of stories that include nonverbal information is better for comprehension than a verbal-only source of information.

The cognitive theory of multimedia learning (Mayer, 2003) proposes that deeper learning occurs when information is presented both verbally and nonverbally. According to the dual coding theory (Paivio, 2007), verbal and nonverbal information are processed in two separate but interconnected channels. Thus, processing the two kinds of stimuli simultaneously does not result in cognitive overload but, on the contrary, facilitates learning. Because illustrations and narration mostly complement each other in picture storybooks, the nonverbal information may support comprehension of verbal information and, vice versa, verbal information may support the interpretation of illustrations and other nonverbal information (Sipe, 1998).

Technology-enhanced books may, even more than traditional print books, enhance children’s story comprehension and word learning from the story due to a closer match between nonverbal and verbal information. When pictures include movements and zooming, each frame might illustrate the oral narration more closely in time than static pictures, resulting in a higher temporal contiguity between the verbal and visual information. In fact, the temporal contiguity principle of the multimedia learning theory predicts deeper learning when the verbal and nonverbal information are presented close to each other in time rather than further apart (Mayer, 2005). The hypothesis is that in the case of high temporal contiguity, children do not need to hold the oral narration and the illustration in working memory in order to integrate them, thus reducing the cognitive load children face when listening to a story. Additionally, it is plausible that sound effects and background music that are often part of technology-enhanced books might, if congruent to the narration, illustrate feelings and mood, thereby facilitating story comprehension and learning abstract words from the narration.

The literature comparing children’s comprehension and memory of the details of animated (television) to audio-only (radio) stories show some evidence that dynamic visualizations enhance story comprehension (Beagles-Roos & Gat, 1983; Gibbons, Anderson, Smith, Field, & Fischer, 1986; Hayes, Kelly, & Mandel, 1986; Pezdek, Lehrer, & Simon, 1984; Sharp et al., 1995). A more recent line of research that compares (a) electronic stories with animated pictures, background sounds, and music to (b) print or print-like presentations that include static illustrations found an advantage for technology-enhanced books on story comprehension and word learning (Smeets & Bus, 2014; Verhallen & Bus, 2010; Verhallen et al., 2006) with some exceptions. For children having difficulties with verbal processing, sound effects might disrupt perception of speech (Smeets, van Dijken, & Bus, 2012).

In sum, as long as they are congruent to the story, animated pictures, sound, and music do not seem to distract children from the story text. On the contrary, meaningful nonverbal additions to stories have been shown to boost story comprehension and word learning. In the present study, the effect of multimedia features was compared to those of oral narration of stories including some or no static illustrations.

Interactive Features

Most technology-enhanced stories are loaded with interactive features such as puzzles, memory tasks, amusing visual or sound effects, dictionary function, or word or picture labels appearing when activating the hotspot (de Jong & Bus, 2003; Guernsey et al., 2012; Korat & Shamir, 2004). As these features are often available not only after, but also during the oral narration (de Jong & Bus, 2003), they might interrupt the flow of the story or draw children’s attention away from listening to the oral narration. In fact, de Jong and Bus (2002) found that when a lot of visual and sound effects are available and children can make a choice between listening to the narration and playing with visual and sound effects, they hardly spend any time listening to the oral narration.

According to the cognitive load theory (Sweller, 2005), working memory capacity is very limited. Instructional designs that do not take this limited capacity into consideration can result in a large cognitive load and disrupt learning. The coherence effect of the cognitive theory of multimedia learning (Mayer, 2003) predicts deeper learning when extraneous materials that are not directly related to the learning material are excluded from the multimedia message. Interactive features, especially the ones that are not tightly connected to the storyline like games or hotspots on irrelevant details, might function as seductive, extraneous materials that can distract children from the story.

In fact, incongruent interactive features have been found to result in the child’s failure to retell the story (Labbo & Kuhn, 2000; Okolo & Hayes, 1996). Ricci and Beal (2002), on the other hand, found that children’s recall of a highly interactive story including unrelated interactive features was better than their recall of an audio-only presentation. Interactive features that support story content may have a potential advantage. Segers et al. (2006) found that an electronic book with games to explain story vocabulary was more beneficial for special needs children’s word learning than a teacher reading a story to them. Korat and Shamir (2008) showed that children reading electronic books with dictionaries improved more in vocabulary than children reading electronic books without interactive features. Smeets and Bus (2014) found that children in the condition including explanations of difficult words in the narration in the form of hotspots outperformed the children in the electronic story condition without interactive features to support word learning.

In sum, interactive elements that are not supportive of story comprehension might function as extraneous material resulting in incidental processing and cognitive overload that disrupts processing of the essential material of the story and learning (Mayer & Moreno, 2003). Constant switching between two different tasks, understanding the story on the one hand and exploring games and hotpots on the other, might place too much extraneous load on the working memory of young children and decrease their performance on both tasks. Specifically, it may result in decreased story comprehension and word learning from the story. Even interactive features that are relevant to the story may disturb story comprehension and language learning. Story comprehension and playing with hotspots or games are two fundamentally different tasks, even when their content is related, and carrying out both requires task switching. On the other hand, the more closely related the story and the interactive additions are, the smaller the cognitive cost of switching between the two tasks is.

Disadvantaged Children

It is plausible that for children who do not fully understand the narration because they lack the language and comprehension skills necessary, nonverbal information from animations and sound effects can fill in the gaps. Similarly, games related to literacy skills in interactive stories can offer an appealing environment to practice and develop literacy skills, which might be especially important for children who are behind or who are having difficulties with these skills. Thus, in the present meta-analysis, special attention was given to the effects of technological enhancements on stories for the different groups of disadvantaged children by testing every effect separately for disadvantaged and nondisadvantaged children.

As we found samples with a wide range of characteristics that might put children at risk for lagging behind in language and literacy development in the primary literature, we used the umbrella term disadvantaged for groups of children from low socioeconomic status (SES) families (e.g., Korat & Shamir, 2007) or immigrant, bilingual families (e.g., Segers, Takke, & Verhoeven, 2004) and children with learning problems, such as struggling readers (e.g., Karemaker, Pitchford, & O’Malley, 2010b), children with special needs (Segers et al., 2006), children with developmental delays (Shamir et al., 2012), or children with severe language impairments (Smeets et al., 2012).

Research Questions

In the present meta-analysis, we were specifically interested in the additional effects of technology as compared to more traditional presentations of stories, like telling a story or reading a print storybook. Thus, only studies contrasting technology-enhanced story presentations to more traditional presentations of the same or a similar story were included in the meta-analysis. In both the technology-enhanced and the comparison conditions, an oral narration of the story had to be included. We considered independent reading of a story as fundamentally different from listening to stories because children need to pay attention to decoding the written text when reading themselves instead of just focusing on comprehending the story.

There were four research questions. Question 1 asked whether technology-enhanced stories foster learning more compared with traditional print-like story presentations. Based on the primary literature, we expected a general advantage of technology-enhanced stories over more traditional presentations on children’s literacy outcomes. Question 2 asked if multimedia-enhanced stories were more beneficial for children’s literacy than traditional story presentations. Based on the theory of multimedia learning, it was hypothesized that multimedia features, congruent to the narration, such as animated pictures, music, and sound, may be beneficial.

Question 3 asked whether interactive features in technology-enhanced stories were distracting at the expense of children’s literacy learning. In contrast to multimedia elements, interactive features, especially the ones that are irrelevant to the story, may be distracting and harmful for story comprehension (Bus et al., 2014). Finally, Question 4 asked if technological additions to stories were more important for disadvantaged groups of children than for nondisadvantaged students. We expected that the addition of multimedia features to stories would be especially important for children who are at risk for or already behind in language development. That is, because of these children’s limited understanding of the story language, they are the ones who might benefit the most from extra nonverbal information. In fact, it is plausible that older and typically developing children with average or above average vocabularies and language skills might not need much, or even any, nonverbal addition to understand a story.

Method

Operational Definitions

The goal of the present study was to compare the effects of technology-enhanced narrative stories to more traditional presentations on young children’s language and literacy development. Technology-enhanced stories were defined as any orally narrated story presented with some digital addition, like multimedia (animated and/or video illustrations, zooming, sound effects, background music) or interactive features (hotspots, questions, games). Our broad definition of technology-enhanced stories included a wide range of electronic stories and television shows and very different devices on which the story was presented, like television sets (e.g., Pezdek & Stevens, 1984), computers (e.g., de Jong & Bus, 2002, 2004; Ricci & Beal, 2002), tablets (Chiong et al., 2012; Noel, 2013), or other platforms like the Microsoft Kinect (Homer et al., 2014). Unlike other reviews (e.g., Zucker et al., 2009), we did not require the technology-enhanced stories to include the print text on the screen similar to print books.

For a study to be included, there had to be a comparison condition in which the same or a similar story was presented in a way that resembled the more traditional circumstances of children listening to stories, that is, listening to someone either tell a story or read one from a picture storybook. For this criterion, a comparison condition with either only orally presented stories or oral text in addition to static illustrations sufficed. Earlier studies assessed the differences between stories presented through television and radio formats, that is, an audiovisual and an audio presentation (e.g., Beagles-Roos & Gat, 1983; Gibbons et al., 1986). Later studies compared technology-enhanced stories to an adult reading the story from a print picture storybook to the child, thus presenting static illustrations during the story. In these studies, the adults were either instructed to keep their interaction with the children to a minimum (e.g., Critelli, 2011) or were encouraged to interact with the child during the reading, imitating a natural interactive shared reading session (e.g., de Jong & Bus, 2004; Homer et al., 2014; Korat & Shamir, 2007). Another alternative was to have the computer read the story while presenting the static pictures on screen without any other technological advancements (e.g., Gong & Levy, 2009; Smeets & Bus, 2014). These comparison conditions were all considered imitations of traditional story sharing activities with young children, even when children listened to a story on the computer but with no other information that is commonly available in a more traditional story sharing session.

Search Strategy

We searched three databases—PsychInfo, ERIC, and the Web of Science—for journal articles, reports, and book chapters with a detailed search string including different terminology for literacy outcomes, technology-enhanced narrative stories, and young children (see Appendix A). Secondary searches involved inspection of the reference lists of review articles and the included articles for other suitable studies, in addition to checking handbooks on technology and children’s literacy development (see Appendix B for the list). We also searched for dissertations and theses reporting data that might be suitable for the present meta-analysis.

When we could not find a full text, authors were contacted. When we could not contact the authors of the original manuscript, we contacted authors who referenced the study to see if they had a copy. Four studies (two conference papers and two reports) were not included in the meta-analysis because we could not locate copies of the manuscripts (George & Schaer, 1986; Hudson, 1982; Meringoff, 1982; Montouri, 1986).

Inclusion Criteria

According to our operational definitions described, intervention studies were included based on the following criteria:

1. The study was experimental or quasi-experimental, either a between- or a within-subject design, and contrasted a technology-enhanced condition with a comparison condition.

2. In one condition, stories were technology-enhanced, including an orally presented narration, multimedia features such as animations, music, sound effects, and/or interactive features (e.g., questions, hotspots, games).

3. The comparison condition involved an orally presented narration with or without static illustrations.

4. Participants were preschool- and/or elementary-school-aged children.

5. The study included at least one outcome measure such as (a) the child’s literacy skills (including story comprehension and vocabulary, and code-related literacy skills such as phonological awareness, letter knowledge, concepts of print, word reading, or general reading skills), or (b) the child’s behavior while listening to the stories (including not only the child’s engagement and attention but also communication initiated by the child).

Parental interaction, as already discussed, was beyond the scope of the present study, so measures of those were not included (e.g., in Chiong et al., 2012). There were no restrictions regarding the publication status of the manuscripts or the participants’ country of origin as long as the article was written in English.

Exclusion Criteria

We excluded correlational studies not comparing a technology-enhanced with a comparison story (Kendeou, Bohn-Gettler, White, & van den Broek, 2008; Kim, Kendeou, van den Broek, White, & Kremer, 2008), studies targeting foreign language learning (Jakobsdottir & Hooper, 1995; Tsou, Wang, & Tzeng, 2006), and studies without an eligible comparison condition (Hayes & Birnbaum, 1980; Matthew, 1996; Trushell, Maitland, & Burrell, 2003). We also excluded technology-enhanced interventions focusing on expository texts (Peracchio, 1992; Silverman & Hines, 2009), programs that targeted explicit literacy training (Penuel et al., 2012), or stories with only written text (Doty, Popplewell, & Byers, 2001; Lewin, 2000; Miller, Blackstock, & Miller, 1994; Neuman, 1992) or sign language (Gentry, Chinn, & Moulton, 2004; Wang & Paul, 2011). Additionally, we excluded studies that overlapped with other studies (Choat & Griffin, 1986; Greenfield & Beagles-Roos, 1988; Reissner, 1996; Vibbert & Meringoff, 1981), presented data already included in another study (Korat, Segal-Drori, & Klein, 2009), or presented data for children and adults together (Pratt & MacKenzie-Keaing, 1985).

In some instances, no data were available on the measure, even after contacting the authors (e.g., the measures of word shape concept and word element concept in Gong & Levy, 2009; the measure of justifications of inferences in Beagles-Roos & Gat, 1983; the measure of picture ordering in Meringoff, 1980; or the measure of child initiated communication in Chiong et al., 2012). We also could not include results when the measure assessed memory for information that was not presented in the comparison condition (e.g., nonverbal information when having an only audio comparison in Pezdek Stevens, 1984; identification of the tutor when the tutor was not included in the comparison condition in Homer et al., 2014), or measures that were outside the scope of this meta-analysis (e.g., creativity in Valkenburg & Beentjes, 1997; characteristics of parent-child interaction in Chiong et al., 2012; or attitude towards computers in Karemaker et al., 2010b, and towards reading in Stine, 1993). See Appendix C for a prisma diagram of the literature search.

Coding

We coded the following information: (a) bibliographic information (e.g., authors, year, and title of study, published or not, kind of publication and the country in which the study was conducted); (b) characteristics of the sample (e.g., basic information such as the number of participants, gender distribution, and mean age in addition to any possible disadvantage factors (e.g., socioeconomic status, intelligence, first or second language learners, language skills, and disabilities or developmental delays); (c) the design of the study (experimental or quasi-experimental and between- or within-subject); (d) materials used in the technology-enhanced condition, including the kind of software used (multimedia story, television program or interactive books), multimedia features (animation, music and sound effects), interactive features (hotspots, games, and questions), and whether those were relevant or irrelevant to story comprehension or other literacy skills, and any other technological features (e.g., highlighting print); (e) the number of repeated interactions with the stories; (f) whether static illustrations were presented in addition to the oral narration in the comparison condition; and (g) outcome measures, including story comprehension (retelling of the story or comprehension questions), vocabulary (expressive or receptive vocabulary, and whether assessing book-based or general vocabulary), code-related literacy skills (alphabet knowledge, concepts of print, name writing, phonological awareness, word writing, word reading and recognition, or reading skills), and child’s engagement during the intervention (e.g., visual attention, skin conductance as indicator of arousal or communication initiated by the child).

To obtain information that was not available in the studies regarding the details of the technology-enhanced stories, we looked the software up on the Internet, for example, checking videos and demos on Youtube.com or other studies reporting on the same software (e.g., Talley, Lancy, & Lee, 1997, for the Stories and More software used in the dissertation of Stine, 1993). When more information was needed, the authors of the study were contacted via e-mail, if possible.

As shown in Table 1, whenever results were reported separately for subgroups of children, based on age (e.g., Pezdek et al., 1984; Williamson & Silvern, 1983), disadvantage status (e.g., Segers et al., 2004), or ability level (e.g., Verhallen & Bus, 2009b), effect sizes were calculated for each subgroup in order to test differences among different groups of children. When studies included two technology-enhanced conditions (e.g., Korat & Or, 2010; Okolo & Hayes, 1996; Robb, 2010), both groups were contrasted with the control group so we could test differences among different features of technology-enhanced stories. In these cases, we divided the number of participants in the comparison group by two in order not to include control group children twice in the analyses (for a similar procedure see Bakermans-Kranenburg, van IJzendoorn, & Juffer, 2003; Mol, Bus, de Jong, & Smeets, 2008). When there were more than one nontechnology comparison condition in a study, the condition most similar to a traditional print book reading activity was chosen (e.g., the adult reading condition in Terrell & Daniloff, 1996, and the text and accompanying illustrations condition in Williamson & Silvern, 1983).

Table 1.

Overview of the studies in the meta-analysis including the moderators and the effect sizes

First author	Year	Disadvantaged	Technology-enhanced condition	Material	Multimedia features (animation and/or music and sound)	Relevant and irrelevant interactive features	Comparison condition	Illustrations	Outcome measures	Average effect size (Hedges’ g)
Beagles-Roos	1983	No	“Television” (n = 24)	A Story, A Story, Strega Nona Animated by Weston Woods Studios	Yes	No	“Radio” (n = 24)	No	Story comprehension (3 measures)	0.34
Caplovitz (dissertation)	2005	No	“Talking book” (n = 48 + 44)	The Story of Clifford, Elmo’s Noisy Day Talking book with plastic case for a paper book	No	Yes, relevant and irrelevant	“Books only” (n = 47)	Yes	Story comprehension (2 measures) Code-related skills (15 measures)	−0.24 0.03
Chiong	2012 Contr 1	No	“Basic e-book” (n = 16)	iPad application	No	Yes, relevant	“Print book” (n = 16)	Yes	Story comprehension (1 measure)	0.17
Chiong	2012 Contr 2	No	“Enhanced e-book” (n = 16)	iPad application	Yes	Yes, relevant and irrelevant	“Print book” (n = 16)	Yes	Story comprehension (1 measure)	−0.81
Critelli (thesis)	2011	No	“E-book” (n = 5)	Bubbles CD-ROM story	Yes	No	“Print book” (n = 5)	Yes	Story comprehension (1 measure) Code-related skills (2 measures)	0.00 −0.10
De Jong	2002	Yes (low SES)	“Computer book – restricted” (n =12)	P.B. Bear’s Birthday Party Bombilla	Yes	Yes, relevant and irrelevant	“Regular book” (n = 12)	Yes	Story comprehension (2 measures) Code-related skills (6 measures)	−0.27 −0.25
De Jong	2004	Yes (low SES)	“Electronic book” (n = 18)	I’ll Make You Well Again Said the Bear, Big Party for Tiger, Tiger and Bear in Traffic Het Spectrum Electronic Publishing	Yes	Yes, relevant and irrelevant	“Printed book” (n = 18)	Yes	Story comprehension (1 measure)	−0.53
Gazella	2003	No	“Audiovisual” (n = 15)	Researcher-constructed story with videotaped puppets	Yes	No	“Audio-only” (n = 14)	No	Story comprehension (2 measures)	−0.11
Gibbons	1986	No	“Audiovisual” (n = 48)	Researcher-constructed stories with animated puppets	Yes	No	“Audio” (n = 48)	No	Story comprehension (1 measure)	0.52
Gong	2009	No	“Bouncing Ball” (n = 24)	Talking books with a bouncing ball above the text being read aloud	No	Yes, only highlighting the text as it is read	“Story control” (n = 24)	Yes	Code-related skills (1 measure)	0.60
Hayes	1986	No	“Television” (n = 22)	How the Whale Got Its Throat Television segment	Yes	No	“Radio” (n = 22)	No	Story comprehension (2 measures)	−0.45
Homer	2014	No	“Kinect with activities” (n = 12)	Children Make Terrible Pets by Microsoft Games Studio for the Kinect	Yes	Yes, relevant and irrelevant	“Book reading” (n = 14)	Yes	Story comprehension (3 measures) Vocabulary – expressive (1 measure) Code-related skills (2 measures) Engagement (1 measure)	0.26 0.37 −0.22 −0.57
Karemaker	2008	No	“ORT for Clicker” (n = 27)	Talking books from the Oxford Reading Tree for Clicker series by Crick Software	No	Yes, relevant	“ORT Big Book” (n = 27)	Yes	Code-related skills (5 measures)	−0.04
Karemaker	2010b	Yes (struggling readers)	“ORT for Clicker” (n = 17)	Strawberry Jam, Kipper the Clown Oxford Reading Tree for Clicker series by Crick Software	No	Yes, relevant	“ORT Big Book” (n = 17)	Yes	Code-related skills (9 measures)	0.07
Karemaker	2010a	No	“ORT for Clicker” (n = 61)	Strawberry Jam, Kipper the Clown Oxford Reading Tree for Clicker series by Crick Software	No	Yes, relevant	“ORT Big Book” (n = 61)	Yes	Code-related skills (5 measures)	0.20
Korat	2010 Contr 1	No	“Educational electronic book - Tractor” (n = 12)	The tractor in the sandbox Researcher-constructed CD-ROM story	Yes	Yes, relevant	“Print book” (n = 12)	Yes	Communication by the child (1 measure)	2.06a
Korat	2010 Contr 2	No	“Commercial electronic book - Grandma” (n = 12)	Just grandma and me Living Books series	Yes	Yes, relevant and irrelevant	“Print book” (n = 12)	Yes	Communication by the child (1 measure)	2.06
Korat	2007 Contr 1	Yes (low SES)	“E-book” (n = 25)	The tractor in the sandbox Researcher-constructed CD-ROM story	Yes	Yes, relevant	“Adult book reading” (n = 25)	Yes	Vocabulary – receptive (1 measure) Code-related skills (2 measures)	0.06 0.29
Korat	2007 Contr 2	No	“E-book” (n = 25)	The tractor in the sandbox Researcher-constructed CD-ROM story	Yes	Yes, relevant	“Adult book reading” (n = 25)	Yes	Vocabulary – receptive (1 measure) Code-related skills (2 measures)	−0.03 −0.18
Korat	2007 Contr 1 & 2	No	“E-book” (n = 50)	The tractor in the sandbox Researcher-constructed CD-ROM story	Yes	Yes, relevant	“Adult book reading” (n = 50)	Yes	Story comprehension (1 measure)	−0.13
Korat	2013	Yes (low SES)	“Electronic book” (n = 30)	Confused Yuval Researcher-constructed CD-ROM story	Yes	Yes, relevant	“Print book” (n = 30)	Yes	Vocabulary – receptive (1 measure) Code-related skills (2 measures)	−0.17 0.11
Meringoff	1980	No	“Television” (n = 24)	A Story, A Story Animated by Weston Woods Studios	Yes	No	“Book” (n = 24)	Yes	Story comprehension (3 measures) Communication by child (1 measure)	−0.21 −1.08
Moody	2010	Yes (low SES)	“Adult-led e-book” (n = 25)	Little Monster at School Living Books series	Yes	Yes, relevant and irrelevant	“Adult-led traditional storybook” (n = 25)	Yes	Engagement (3 measures) Communication by child (5 measures)	0.24 0.04
Neuman	1989	No	“Televised version” (n = 17)	Simon’s Book	Yes	No	“Storybook” (n = 10)	Yes	Story comprehension (3 measures)	0.32
Noel (thesis)	2013	Yes (low SES)	“Electronic book” (n = 3)	How Rocket Learned to Read, Tacky the Penguin, Elmer’s Special Day, Elmer and Rose, The Bremen Town Musicians, Curious George, Jumanji Applications on the iPad	Mix	Mix	“Print book” (n = 3)	Yes	Vocabulary – expressive (1 measure) Engagement (1 measure)	0.15 0.56
Okolo	1996 Contr 1	Yes (learning disabled students or poor readers)	“High-animation condition” (n = 20)	Harry and the Haunted House Living Book series	Yes	Yes, relevant and irrelevant	“Adult-reader” (n = 20)	Yes	Story comprehension (2 measures) Communication by child (1 measure)	−1.19 0.64
Okolo	1996 Contr 2	Yes (learning disabled students or poor readers)	“Low-animation condition” (n = 20)	Thomas’s snowsuit DISCIS book series	No	Yes, relevant	“Adult-reader” (n = 20)	Yes	Story comprehension (2 measures) Communication by child (1 measure)	−0.28 0.53
Parish-Morris	2013 Study 2	No	“Electronic console books” (n = 20)	The Berenstain Bears, Dora the Explorer by Fisher-Price PowerTouch Learning System	No	Yes, relevant and irrelevant	“Traditional book” (n = 20)	Yes	Story comprehension (2 measures)	0.65
Pezdek	1984 Contr 1	No (3rd grade)	“Television” (n = 24)	A Story, A Story, Strega Nona Animated by Weston Woods Studios	Yes	No	“Radio” (n = 24)	No	Story comprehension (3 measures)	1.02
Pezdek	1984 Contr 2	No (6th grade)	“Television” (n = 24)	A Story, A Story, Strega Nona Animated by Weston Woods Studios	Yes	No	“Radio” (n = 24)	No	Story comprehension (3 measures)	0.55
Pezdek	1984	No	“Audiovisual match” (n = 24)	Bert and Ernie and Big Birds segments from the TV show Sesame Street	Yes	No	“Audio only” (n = 24)	No	Story comprehension (2 measures)	0.59
Ricci	2002	No	“Interactive participant” (n = 16)	The Ugly Duckling Interactive story	Yes	Yes, relevant and irrelevant	“Audio only” (n = 17)	No	Story comprehension (4 measures)	0.23
Robb (dissertation)	2010 Contr 1	No	“Interactive reading with parent” (n = 23)	Curious George Goes to a Chocolate Factory Read With Me DVD System	Yes	Yes, relevant	“Print book reading with parent” (n = 12)	Yes	Story comprehension (3 measures) Engagement (1 measure)	0.21 −0.54
Robb (dissertation)	2010 Contr 2	No	“Interactive reading alone” (n = 23)	Curious George Goes to a Chocolate Factory Read With Me DVD System	Yes	Yes, relevant	“Print book reading with parent” (n = 12)	Yes	Story comprehension (3 measures) Engagement (1 measure)	−0.06 0.10
Segal-Drori	2010 Contr 1	Yes (low SES)	“E-book independently” (n = 32)	The tractor in the sandbox Confused Yuval Researcher-constructed CD-ROM stories	Yes	Yes, relevant	“Print book with adult” (n = 16)	Yes	Code-related skills (3 measures)	−0.05
Segal-Drori	2010 Contr 2	Yes (low SES)	“E-book with adult instruction” (n = 32)	The tractor in the sandbox Confused Yuval Researcher-constructed CD-ROM stories	Yes	Yes, relevant	“Print book with adult” (n = 16)	Yes	Code-related skills (3 measures)	1.69
Segers	2006	Yes (special needs, physical disabilities)	“Computer story” (n = 9)	Treasure Chest with the Mouse Researcher-constructed CD-ROM stories	Yes	Yes, relevant	“Teacher reading” (n = 8)	Yes	Vocabulary (2 measures) Engagement (1 measures)	0.62 −0.12
Segers	2004 Contr 1	No	“Computer reading” (n = 41)	Treasure Chest with the Mouse Researcher-constructed CD-ROM stories	Yes	Yes, relevant	“Teacher reading” (n = 41)	Yes	Story comprehension (1 measure) Vocabulary – expressive (1 measure)	−0.08 −0.02
Segers	2004 Contr 2	Yes (immigrant)	“Computer reading” (n = 30)	Treasure Chest with the Mouse Researcher-constructed CD-ROM stories	Yes	Yes, relevant	“Teacher reading” (n = 30)	Yes	Story comprehension (1 measure) Vocabulary – expressive (1 measure)	−0.41 −0.18
Shamir	2012	Yes (developmental delays)	“E-book” (n = 42)	Confused Yuval Researcher-constructed CD-ROM story	Yes	Yes, relevant	“Printed book with adult” (n = 34)	Yes	Vocabulary – receptive (1 measures) Code-related eracy skills (2 measures)	0.45 0.24
Sharp	1995	Yes (low SES, small vocabulary)	“Helpful video” (n = 18)	Commercial video clips with researcher-constructed narratives	Yes	No	“No video” (n = 18)	No	Story comprehension (2 measures)	1.43
Silverman	2013 Study 1	Yes (low SES, limited English proficiency)	“Video” (n = 42)	Arthur, Martha Speaks TV shows	Yes	No	“Read aloud” (n = 36)	Yes	Vocabulary (2 measures)	−0.24
Smeets	2014 Contr 1	No	“Animated e-book” (n = 36)	Pete on the Pavement, Bear Is In Love With Butterfly, Rokko the Crocodile, Bolder and the Boat, Cycling With Grandpa Het Woeste Woud	Yes	No	“Static e-book” (n = 17)	Yes	Story comprehension (3 measures) Vocabulary (3 measures)	0.11 −0.18
Smeets	2014 Contr 2	No	“Interactive animated e-book” (n = 33)	Pete on the Pavement, Bear Is In Love With Butterfly, Rokko the Crocodile, Bolder and the Boat, Cycling With Grandpa Het Woeste Woud	Yes	Yes, relevant	“Static e-book” (n = 16)	Yes	Story comprehension (3 measures) Vocabulary (3 measures)	0.19 0.13
Smeets	2012 Exp 1	Yes (SLI)	“Video book” (n = 28)	Pete on the Pavement, Rokko the Crocodile, Bolder and the Boat, Little Kangaroo, Imitators, Dear Dear Het Woeste Woud	Yes	No	“Static book” (n = 28)	Yes	Vocabulary – expressive (1 measure)	−0.25
Smeets	2012 Exp 2	Yes (SLI)	“Video with music and sounds” (n = 21)	Pete on the Pavement, Rokko the Crocodile, Bolder and the Boat, Little Kangaroo, Imitators, Dear Dear, Bear Is In Love With Butterfly, Sweat-Naughty Bear Baboen Het Woeste Woud	Yes	No	“Static-no music or sound” (n = 21)	Yes	Vocabulary – expressive (1 measure)	0.07
Stine (dissertation)	1993	Yes (low SES)	“CD-ROM interactive computer-based books” (n = 8)	Stories and More by IBM	Yes	Yes, relevant	“Print books” (n = 8)	Yes	Story comprehension (1 measure) Vocabulary (1 measure)	1.31 1.55
Terrell	1996	No	“Videotape” (n = 26)	An animated segment, including music, from a children’s TV show with a researcher-constructed narrative	Yes	No	“Storybook” (n = 26)	Yes	Vocabulary (3 measures)	−0.58
Valkenburg	1997	No	“Television” (n = 64)	Strega Nonna Doctor de Soto Animated by Weston Woods Studios	Yes	No	“Radio” (n = 64)	No	Story comprehension (1 measure)	0.14
Verhallen	2006 Contr 1	Yes (low SES, 2nd language)	“4x Multimedia” (n = 10)	Winnie the Witch Bombilla	Yes	No	“4x Static” (n = 10)	Yes	Story comprehension (1 measure) Vocabulary – expressive (1 measure)	1.16 1.07
Verhallen	2006 Contr 2	Yes (low SES, 2nd language)	“1x Multimedia” (n = 10)	Winnie the Witch Bombilla	Yes	No	“1x Static” (n = 10)	Yes	Story comprehension (1 measure) Vocabulary- expressive (1 measure)	0.36 0.62
Verhallen	2009a Contr 1	Yes (low SES, 2nd language)	“4x video” (n = 22)	Winnie the Witch Bombilla	Yes	No	“4x static” (n = 20)	Yes	Story comprehension (1 measure) Vocabulary – expressive (1 measure) Engagement (1 measure)	0.54 0.54 0.28
Verhallen	2009a Contr 2	Yes (low SES, 2nd language)	“1x video” (n = 21)	Winnie the Witch Bombilla	Yes	No	“1x static” (n = 23)	Yes	Story comprehension (1 measure) Vocabulary – expressive (1 measure)	0.39 0.59
Verhallen (dissertation)	2009b Contr 1	Yes (low SES, 2nd language, low L2)	“Video” (n = 22)	Winnie the Witch Bombilla	Yes	No	“Static” (n = 20)	Yes	Story comprehension (2 measures) Vocabulary – expressive (1 measure)	0.98 0.02
Verhallen (dissertation)	2009b Contr 2	Yes (low SES, 2nd language)	“Video” (n = 13)	Winnie the Witch Bombilla	Yes	No	“Static” (n = 12)	Yes	Story comprehension (2 measures) Vocabulary – expressive (1 measure)	−0.23 0.89
Verhallen	2010	Yes (low SES, 2nd language)	“Video” (n = 34)	Winnie the Witch Bombilla	Yes	No	“Static” (n = 29)	Yes	Vocabulary (2 measures)	0.36
Williamson	1983 Contr 1	No (kindergarten)	“Animated film” (n = 10)	Petunia Animated film of the story while researcher reads it	Yes	No	“Tradebook” (n = 10)	Yes	Story comprehension (2 measures)	0.15
Williamson	1983 Contr 2	No (third grade)	“Animated film” (n = 10)	Petunia Animated film of the story while researcher reads it	Yes	No	“Tradebook” (n = 10)	Yes	Story comprehension (2 measures)	1.00

^aWinsorized value.

One technology-enhanced condition was chosen instead of including both when the control condition included fewer than 10 children (e.g., de Jong & Bus, 2002). In these cases, we chose the most technology-enhanced condition (e.g., the video with music and sound condition in Experiment 2 in Smeets et al., 2012; the Kinect with activities condition in Homer et al., 2014; the interactive condition in Ricci & Beal, 2002; the helpful video condition in Sharp et al., 1995, or the technology condition including an adult such as the adult-led e-book condition in Moody et al., 2010). However, in the study by de Jong and Bus (2002) the restricted/no-game electronic book condition was chosen because when children had the option to play with the games, they hardly spent time listening to the story. Another exception was the study described in Caplovitz (2005); we merged two technology-enhanced story conditions in this study as the difference between the two, instruction for the parents on how to use the talking book, was not considered a potential moderator in the present meta-analysis. In the Gong and Levy (2009) study, the bouncing ball condition was chosen for the technology-enhanced condition because the bouncing ball jumping from word to word while they are read aloud was regarded as highlighting the text. The other conditions in this study, including violations in the written text on screen, were considered fundamentally different from the technology-enhanced story conditions and therefore not included.

All studies were coded by two independent coders to assess interrater reliability. Full agreement was reached for study eligibility. For further coding, agreement was on average κ = .77, ranging from κ = .65 for the materials used in the technology condition to κ = .99 for bibliographic information. Disagreements were settled in discussion.

Meta-Analytic Procedures

The dependent variable in the present meta-analysis was the difference in mean score between the technology-enhanced condition and the condition similar to a traditional print book reading activity. As different outcome measures were included with different scales, the standardized mean difference, Hedges’ g, was calculated for each contrast between the two conditions. To calculate Hedges’ g, raw post-test means and standard deviations were favored over other statistics, but in some cases, only frequency distributions, F, t, chi-square statistics (e.g., Segers et al., 2006), or gain scores in the two conditions (e.g., Critelli, 2011) were available. In the case of gain scores, we calculated the difference between the average gains in the technology-enhanced and comparison condition (Morris & DeShon, 2002). We entered the available statistics in the Comprehensive Meta-Analysis software, Version 2.0 (Borenstein, Hedges, Higgins, & Rothstein, 2005), which calculated Hedges’ g for each contrast for each outcome variable, as presented in Table 1. We preferred Hedges’ g to alternatives because Ns were rather small. If two or more outcome measures were available in one study, the effect sizes for the different measures were averaged to compute an overall effect for the study. Interpretation of Hedges’ g statistics is similar to that of Cohen’s d. In previous meta-analyses of print exposure, effect sizes averaged around d = .50 (Bus et al., 1995; Mol & Bus, 2011). We expected a small advantage of technology-enhanced reading compared to more traditional print book reading.

A positive effect size indicated an advantage for the technology-enhanced condition to a condition more similar to traditional print book reading. The effect sizes for all separate outcome measures were inspected for outliers, which resulted in eight outlying values (with a standardized residual exceeding ± 3.29; Tabachnick & Fidell, 2007). The most extreme value, the effect size for looking time at the screen or the book in the study of Homer et al. (2014; i.e., Hedges’ g = 22.00) was excluded from further analysis. The outlying effect size resulted from the small standard deviation of this variable. All other outliers were winsorized into values of .01 higher, or lower in the case of the one negative effect size, than the highest or the lowest nonoutlying effect size. Results were averaged for four sets of outcome measures: story comprehension, vocabulary, code-related literacy skills, and children’s behavior during reading session. We also differentiated expressive and receptive vocabulary measures because there is some evidence that these two measures reflect different levels of word knowledge (Verhallen & Bus, 2010).

Overall effect sizes and 95% confidence intervals were computed based on the random effects model (Lipsey & Wilson, 2001; Raudenbush, 2009). This model takes into account the variation between studies as a result of differences in participants, study design, and intervention characteristics, in addition to within-study variance (Borenstein, Hedges, Higgins, & Rothstein, 2009). Heterogeneity of the effect sizes was estimated using the Q-statistic, with a significant Q indicating a heterogeneous effect, which means that more variability is found within the included studies than may be expected from sampling error on a subject level only (Lipsey & Wilson, 2001). Studies were weighted by the inverse of their variance, so that studies with larger sample sizes and more accurate estimates of population parameters had a greater weight on the mean effect size (Lipsey & Wilson, 2001; Shadish & Haddock, 2009).

Publication bias occurs when studies with significant findings are overrepresented because these are more likely to be published (Borenstein et al., 2009). Publication bias can be observed by visual examination of the funnel plot. In case of asymmetry around the mean effect size, Duval and Tweedie’s (2000) Trim and Fill procedure was used to adjust the overall effect size for publication bias. Additionally, the classic fail-safe N was calculated to have an indication of the confidence of the effect. The fail-safe N shows how many studies showing null effects would be needed to turn a significant effect size into a nonsignificant one. A fail-safe number of 5k + 10 is considered robust, where k is the number of studies in the meta-analysis (Rosenthal, 1979).

Moderator analyses were performed using a random effects model to contrast subsamples based on different categorical study variables. Only moderator variables that had at least four contrasts in one cell were used (cf. Bakermans-Kranenburg et al., 2003). For continuous study variables, as for example, publication year, a meta-regression analysis was performed. Moderators were significant in cases of categorical variables, if Q_between or, for continuous variables, the regression model was significant.

Results

Preliminary Analyses

The search resulted in 43 studies including 57 effects published between 1980 and 2014. All contrasts are shown in Table 1. Eight contrasts came from dissertations, 2 from a research report, and 47 from journal articles. One of the studies used a quasi-experimental design (Stine, 1993); all other studies had an experimental design. Twenty-four studies were conducted in the United States and 3 in the United Kingdom, all including interventions in English. Eleven studies were conducted in the Netherlands with interventions in Dutch, and 5 studies originated from Israel with interventions in Hebrew. In total, 2,147 children between 3 and 10 years of age were included in the meta-analysis. The average sample size of the primary studies was 38.34 children (SD = 21.52). The mean number of repeated readings of the same story during the interventions was 2.30 (SD = 1.65).

To test for publication bias, all effect sizes were transformed into Fisher’s Z. Inspection of the funnel plot showed an even distribution of the effect sizes and no studies were imputed. The number of missing studies that would turn an overall effect for all contrasts nonsignificant was N_fs = 344, which is a robust effect according to Rosenthal’s (1979) criterion. Publication status (i.e., journal article vs. nonrefereed publications such as dissertations) was not a significant moderator, Q_between(1) = 0.26, p = .61, indicating no evidence of publication bias. To test for other biases, moderator analyses were performed for subject design (within vs. between) and country, and meta-regression analyses were performed for publication date, number of repeated readings, and sample size. No significant regression models or moderators were found, except for design. On average, studies with a between-subject design yielded an average effect of 0.33, k = 40, SE = 0.08, 95% CI = [0.17, 0.48], p < .01, which was significantly higher than studies incorporating a within-subject design, g+ = −0.02, k = 17, SE = 0.11, 95% CI = [−0.24, 0.21], p = .89), Q_between(2) = 6.25, p = .01. A likely explanation for this design effect is the role of interactive features as will be shown hereafter: Two-thirds of the within-subject design experiments included interactive features, in contrast, less than half of the between-subject design studies featured interactive elements.

The Effect of Technology Added to Stories for Young Children

To answer the first research question, we inspected the average effect sizes regarding the differences between technology-enhanced stories and more traditional story presentations on children’s literacy outcomes. See Table 2 and Figure 1 for a summary of the findings.

Table 2.

The results on the different outcome measures overall and for disadvantaged and nondisadvantaged groups

Outcome measure	Samples	Number of contrasts included	Average effect size (g+)	Standard error	95% confidence interval	p	Difference between disadvantaged and nondisadvantaged samples (Qbetween)
Story comprehension
	Overall	38	0.17	0.08	[0.01, 0.34]	.04
	Disadvantaged	13	0.22	0.15	[−0.08, 0.51]	.15	Qbetween(1) = 0.13, p = .72
	Nondisadvantaged	25	0.15	0.10	[−0.04, 0.35]	.13
Expressive vocabulary
	Overall	18	0.20	0.10	[0.01, 0.39]	.04
	Disadvantaged	13	0.27	0.12	[0.04, 0.50]	.02	Qbetween(1) = 1.10, p = .30
	Nondisadvantaged	5	0.05	0.17	[−0.29, 0.39]	.78
Receptive vocabulary
	Overall	9	−0.08	0.12	[−0.31, 0.15]	.51
	Disadvantaged	5	0.07	0.13	[−0.19, 0.33]	.60	Qbetween(1) = 2.97, p = .09
	Nondisadvantaged	4	−0.30	0.17	[−0.63, 0.03]	.08
Code-related literacy skills
	Overall	14	0.16	0.10	[−0.04, 0.36]	.11
	Disadvantaged	7	0.27	0.15	[−0.03, 0.56]	.08	Qbetween(1) = 0.84, p = .36
	Nondisadvantaged	7	0.07	0.15	[−0.21, 0.36]	.62
Engagement and child-initiated communication during reading
	Overall	12	0.26	0.24	[−0.21, 0.74]	.28
	Disadvantaged	6	0.32	0.35	[−0.37, 1.01]	.36	Qbetween(1) = 0.05, p = .82
	Nondisadvantaged	6	0.21	0.35	[−0.48, 0.90]	.55

Figure 1.

The effect of technology added to stories as compared to a more traditional story sharing comparison condition on various outcome measures.

*p < .05.

Story comprehension

Thirteen contrasts assessed story comprehension with story retelling measures, 9 contrasts used story comprehension questions, and 15 were based on a combination of the two. Technology had a small but significant effect on children’s story comprehension (see Table 2). As this effect was heterogeneous, Q(37) = 96.21, p < .01, we conducted a moderator analysis to test the effect of assessment. For one contrast, we were unable to code how story comprehension was measured due to insufficient information. Excluding that contrast, a moderator analysis revealed that there was no significant difference among the contrasts based on retelling, comprehension questions, or a combination of these measures, Q_between(1) = 1.60, p = .45. A second moderator analysis comparing disadvantaged with nondisadvantaged children also did not indicate a significant difference in effectiveness of technology (see Table 2).

Vocabulary learning

For one contrast with vocabulary as outcome measure, there was not sufficient information to code whether the measure assessed receptive or expressive word knowledge, so this contrast was excluded from further analysis. Seven contrasts were based on book-based receptive vocabulary and two contrasts targeted general receptive vocabulary. Technology did not have a significant additional effect on receptive vocabulary as compared to more traditional storybook reading conditions.

With regard to expressive vocabulary, 15 contrasts targeted book-based expressive word knowledge and 3 contrasts were based on a combination of book-based and general expressive vocabulary. The average effect size for expressive vocabulary equaled 0.20. This effect was heterogeneous, Q(16) = 28.81, p = .04, so moderator analyses were performed. There was a significant effect for disadvantaged children, but not for nondisadvantaged children, and this difference was not significant (see Table 2). As the effect found for disadvantaged children was heterogeneous, Q(12) = 25.54, p = .01, we inspected differences between subsamples. A significant effect was found for children who were at risk because of environmental factors like low parental education, g+ = 0.35, k = 10, SE = 0.15, 95% CI = [0.06, 0.65], p = .02. There were only three contrasts including samples with developmental delays or learning problems with a nonsignificant average effect size, g+ = 0.06, SE = 0.27, 95% CI = [−0.47, 0.59], p = .82. Therefore, the kind of disadvantage could not be tested as a moderator for expressive vocabulary outcomes. Due to the low number of studies including a general expressive vocabulary measure, a moderator analysis contrasting only book-based and a mix of book-based and general word knowledge could not be carried out.

Code-related literacy skills

Of the 14 contrasts with code-related literacy as the outcome measure, 1 contrast targeted letter knowledge; 1 phonological awareness measures; 1 word reading skills; and 11 a combination of measures tapping phonological awareness, word reading and recognition, word writing, name writing, letter knowledge, and print concepts. The combined effect for the 14 contrasts measuring the additional effect of technology was not significant. As the effect was heterogeneous, Q(13) = 23.65, p = .03, we tested effects in disadvantaged and nondisadvantaged groups separately. For disadvantaged children, the effect of technology did not attain significance. For nondisadvantaged children, the difference was not significant, neither was the difference between the groups (see Table 2).

Child engagement and communication during reading

Of the 12 contrasts related to engagement and communication, 5 targeted communication initiated by the child; 6 targeted children’s engagement during reading including on-task behavior, looking at the material or skin conductance; and 1 contrast was based on a combination of the two. There was no significant effect of the technology-enhanced condition on child engagement and communication during reading. The effect was heterogeneous, Q(11) = 50.55, p < .01. However, there were not enough contrasts to compare the effect of technology for disadvantaged and nondisadvantaged children.

The Role of Multimedia and Interactive Features

To answer the second and third research questions, the effects of multimedia and interactive features were compared. For a summary of the findings, see Figure 2.

Figure 2.

The effects of multimedia, multimedia-interactive and only-interactive stories on story comprehension and vocabulary measures.

**p < .01.

Story comprehension

As the effect of the technology-enhanced condition on story comprehension was heterogeneous, we tested the differences among stories including only multimedia, only interactive features, and the ones with both multimedia and interactive features. This test revealed a significant contrast, Q_between(2) = 12.10, p < .01. As shown in Table 3, stories including only multimedia had a positive additional effect on story comprehension compared to more traditional story sharing activities, g+ = 0.39, whereas this effect was not significant for stories including both multimedia and interactive features. As the effect in the multimedia condition was heterogeneous, Q(20) = 41.03, p < .01, another moderator analysis was conducted to assess whether the control conditions—only oral text or oral text plus static illustrations—made a difference for the effect of multimedia. However, the presence of illustrations in the comparison condition was not a significant moderator, Q_between = 0.11, p = .74. Multimedia stories had a significant advantage over both only orally presented stories, g+ = 0.43, k = 9, SE = 0.14, 95% CI = [0.15, 0.71], p < .01, and stories presented with static illustrations, g+ = 0.36, k = 12, SE = 0.14, 95% CI = [0.08, 0.64], p = .01.

Table 3.

The effects of the different types of stories on measures of expressive vocabulary and story comprehension

Type of technology story	Number of contrasts included	Average effect size (g+)	Standard error	95% confidence interval	p
Multimedia only
Story comprehension	21	0.39	0.10	[0.20, 0.59]	<.01
Expressive vocabulary	12	0.24	0.12	[0.004, 0.47]	.046
Interactive-multimedia
Story comprehension	13	−0.14	0.13	[−0.38, 0.11]	.27
Expressive vocabulary	6	0.13	0.18	[−0.22, 0.48]	.48
Interactive only
Story comprehension	4	−0.03	0.23	[−0.47, 0.41]	.89
Expressive vocabulary	0	—	—	—	—

As shown in Figure 3, for nondisadvantaged children, the difference between multimedia stories, g+ = 0.28, k = 14, SE = 0.10, 95% CI = [0.10, 0.47], p < .01, and stories that also included interactive features, g+ = −0.04, k = 8, SE = 0.13, 95% CI = [−0.29, 0.21], p = .74, was significant, Q_between(1) = 4.18, p = .04. However, in the disadvantaged group, multimedia stories revealed much higher scores than interactive stories; the difference was slightly less than a whole point. For disadvantaged children the difference was significant, Q_between(1) = 7.22, p < .01, with a strong additional effect of multimedia stories, g+ = 0.66, k = 7, SE = 0.23, 95% CI = [0.21, 1.11], p < .01, and a nonsignificant effect for stories also including interactive features. We could not test differences between children growing up in disadvantaged environments and children with developmental delays or learning difficulties because only one study that included children with developmental delays or learning difficulties assessed story comprehension.

Figure 3.

The effects of multimedia and multimedia-interactive stories on disadvantaged and nondisadvantaged children’s story comprehension.

*p < .05, **p < .01.

To investigate the effect of the congruity of interactive features with the story content on story comprehension, interactive stories with only relevant features were compared with stories including irrelevant interactive elements. Stories including only irrelevant or both relevant and irrelevant features did not have a significant effect, g+ = −0.21, k = 7, SE = 0.17, 95% CI = [−0.54, 0.13], p = .22. More surprisingly, stories with only relevant features did not have a significant additional effect compared to more traditional stories either, g+ = −0.06, k = 10, SE = 0.13, 95% CI = [−0.32, 0.21], p = .67. Relevance was not a significant moderator, Q_between(1) = 0.49, p = .48.

Expressive vocabulary learning

We tested the difference between multimedia-only and multimedia-interactive stories on expressive vocabulary as the overall effect was heterogeneous. Although the contrast was not significant, Q_between(1) = 0.26, p = .61, a similar trend appeared. As shown in Table 3, multimedia-only stories showed a significant advantage over more traditional stories on expressive word learning; in contrast, multimedia-interactive stories did not. We could not test whether characteristics of the control condition, only oral text or oral text plus static illustrations made a difference for the effect of multimedia on expressive word learning because there were no contrasts with only oral text.

For disadvantaged children there were not enough contrasts with multimedia-interactive stories to test the difference between multimedia-only and multimedia-interactive stories. However, for these groups of children, multimedia-only stories showed a significant advantage over traditional story materials on expressive word learning, g+ = 0.32, k = 10, SE = 0.15, 95% CI = [0.03, 0.62], p = .03. We could not test differences between children growing up in disadvantaged environments and children with developmental delays or learning difficulties because only two contrasts including children with developmental delays or learning difficulties targeted expressive vocabulary. For nondisadvantaged children there were only two contrasts including a multimedia-only story and three contrasts including a multimedia-interactive story, so the presence of interactive features could not be tested. The effect of the relevance of interactive features could not be tested on expressive vocabulary because there were only two contrasts including irrelevant interactive features. Again, the average effect size of interactive stories including only relevant features was not significant, g+ = 0.04, k = 4, SE = 0.14, 95% CI = [−0.23, 0.31], p = .77.

Discussion

The present study synthesized the available empirical evidence on how technology added to narratives changes the effects of listening to stories on young children’s literacy development. In 43 studies including 2,147 children, we found a small, significant positive additional effect of technology on measures of story comprehension and expressive vocabulary. Although small, the mean effect size is of great relevance as they reflect the additional effect of technology on top of the benefits of more traditional story presentations. So in reply to the first research question, we found evidence that technology can enhance the effects of storybooks on young children’s literacy development. In addition, it is worth noting that these effects were heterogeneous, which may reflect the wide variety of technology-enhanced stories and measures used in the studies. This result underscores the relevance of investigating the effects of different technological features on literacy development.

We found no significant advantage of technology-enhanced stories on receptive vocabulary, code-related literacy skills, or behavior during listening to the story. The small overall effects of technology on comprehension and expressive word learning are in line with a previous meta-analysis showing small to moderate effects on comprehension-related outcomes (Zucker et al., 2009). The nonsignificant finding for receptive vocabulary might result from ceiling effects: Scores on receptive knowledge of words are high even after a more traditional story presentation (Verhallen & Bus, 2010). Technology-enhanced stories did not have a significant effect on code-related literacy skills, probably because most studies in the meta-analysis measuring these skills used programs with interactive features. Although this finding makes sense given the practice that suffices for the development of code-related skills, it also means that code-related skills and interactive features were confounded in the present study. Finally, technology did not contribute significant additional variance to children’s engagement or communication during the reading session. This outcome suggests that the effects of technology on literacy skills may not be a function of increased attention and excitement while listening to the story, although technology can be beneficial for cognitive processing of the information in the story.

Multimedia and Interactive Features

Multimedia stories had a significant positive effect as compared to more traditional presentations on story comprehension, and expressive vocabulary, whereas interactivity combined with multimedia and interactive-only stories did not significantly differ from the nontechnological comparison conditions. As the moderator, static illustrations available in the comparison condition or not, was not significant, multimedia-only stories had a significant advantage over traditional print books including static illustrations. Thus, the advantage of multimedia-enhanced stories was not due to the addition of illustrations but to features that can only be realized with the help of multimedia (e.g., animated pictures, sounds and music). Children from disadvantaged family environments (low SES and/or immigrant, bilingual families) benefited most from multimedia, which had a moderately strong effect on story comprehension and a small effect on expressive vocabulary. Thus, multimedia elements were found to be beneficial additions to stories with small to moderate effect sizes.

This finding supports our hypothesis that extra nonverbal information such as animated visualizations, background sounds, and music, as long as congruent with the narration, aid children’s comprehension, especially when children are at risk for language delays. This finding also aligns with the multimedia learning theory (Mayer, 2003), which proposes that the stronger match between verbal and nonverbal information in multimedia stories, compared to stories with static pictures, supports learning (Bus et al., 2014). Thus, instead of causing cognitive overload, nonverbal information optimally attuned to the narration is beneficial for learning. Multimedia may not be helpful when the nonverbal information is not designed in a way to attract attention to details that illustrate the story text (Bus et al., 2014). We were unable to test the prediction that only when nonverbal information closely corresponds the narration, multimedia stories enhance effects of story reading because we were unable to code whether animations and sound effects were supportive of the oral text or had a purely decorative function in the primary studies.

Regarding the third research question, interactive elements did not make a significant contribution to the effects of listening to a story, even when combined with multimedia features. Interactive features negatively affected story comprehension and expressive word learning, probably because interactivity may interfere with the line of the story and children’s processing of the narrative. Strikingly, even interactive features designed to develop story understanding and literacy skills do not seem to enhance the effects of listening to stories. These results confirm that interactive features are possible distractors from the story, whether they are relevant to the story and developing literacy skills or not. These findings are in line with the cognitive load theory (Sweller, 2005) and support our conclusion regarding interactivity, that is, interactive elements seem to distract from understanding the story and result in cognitive overload in the child (Bus et al., 2014). This outcome is probably because the processing of games and extra animations can be considered as extraneous materials that interfere with the processing of the story content (Mayer & Moreno, 2003). Bus and colleagues (2014) proposed that interactive features in technology-enhanced stories are distracting, probably most when there is an abundance of possibilities for interaction, because the child is required to juggle two tasks at the same time: listening to a story and engaging with interactive elements like games and hotspots. This finding also may explain why positive effects of multimedia fade out when books include interactivity.

Disadvantaged Children

Larger effect sizes were found in groups of disadvantaged children as compared to the mean effect sizes in the samples as a whole. Although the effect of technology on story comprehension for disadvantaged children was similar to the effect found for nondisadvantaged children, the same effect on expressive vocabulary was only significant for the disadvantaged groups. Likewise, the effects multimedia and interactive features have on story comprehension were larger for disadvantaged groups. There was a trend suggesting that disadvantaged children profited more from multimedia stories on story comprehension as compared to nondisadvantaged children, but the difference was not significant.

Although not significant, disadvantaged children also tended to be more distracted by interactive features than nondisadvantaged children, suggesting not only no advantage but also a disadvantage of interactivity for disadvantaged children but not for nondisadvantaged groups. To further illustrate this, for disadvantaged groups the difference between the effects of multimedia and interactive-multimedia stories on story comprehension was almost a whole point; in contrast, this difference was significant but small for nondisadvantaged groups of children. When results were further inspected for different groups of disadvantaged children, we found that this pattern was most pronounced in the group that was at risk due to environmental factors like SES and immigrant status or growing up in bilingual families. Due to the small number of studies targeting children with developmental delays and learning difficulties, the role of multimedia and interactive features could not be tested for this group. These children might also benefit from multimedia-only stories, but, alternatively, it may be that technological additions to stories do not provide sufficient support for children with serious disabilities.

In the present meta-analysis, children from low socioeconomic status and immigrant families and children already experiencing a lag in language and literacy development were considered disadvantaged. These children might have smaller vocabularies and may be experiencing difficulties understanding the sophisticated language of narrative stories, which seem to make them more sensitive to the effects of multimedia and interactive features. In sum, both the benefits of multimedia and the pitfalls of interactive features tend to be elevated for disadvantaged children.

Limitations

Due to the limited number of primary studies available, we could not assess the separate effects of different kinds of multimedia (e.g., animation, music and sound effects) and interactive features (e.g., games, hotspots, dictionary function), nor the effects of how well they correspond to the narration. Moreover, the participants consisted of a broad range of disadvantaged children with different risk factors like low SES; second language learner immigrants; children with small vocabularies in addition to struggling beginning readers; and children with learning disabilities, severe language impairments, special needs and developmental delays. Thus, they were not a homogeneous group of children, and technological additions may have different effects for different risk statuses (e.g., Smeets et al., 2012). More specific results were reported for groups of disadvantaged children who are at risk for developing language delays and learning problems and for groups showing delays and difficulties. Still, a larger number of primary studies may enable more fine-grained analyses leading to a thorough understanding of the effects of different technological features, specifically for different groups of at-risk children.

Conclusion

Technology provides a small but significant addition to the effects of listening to stories on young children’s literacy development and especially on story comprehension and expressive word learning, evidencing the potential of electronic stories and books. Multimedia features such as animated illustrations and music and sound effects were found to be beneficial; in contrast, interactive elements such as hotspots and games—even the ones that are intended to facilitate understanding of the story content—were not. Moreover, children who were at risk for language and literacy delays, especially due to disadvantaged family backgrounds, were shown to be more sensitive to both the benefits and the pitfalls of technological additions: Multimedia elements were especially helpful and interactive features were especially distracting for these children. Developers of technology-enhanced stories and individuals who have the responsibility for selecting high-quality electronic stories should choose ones without interactive features that might distract children from the story and opt for stories with multimedia support that is congruent with the story and provides nonverbal scaffolding for children to understand the story.

Authors

ZSOFIA K. TAKACS is a doctoral student at the Institute of Education and Child Studies at Leiden University.

ELISE K. SWART is a doctoral student at the Institute of Education and Child Studies at Leiden University.

ADRIANA G. BUS is a full professor in the Institute of Education and Child Studies at Leiden University.

Appendix A

Search Terms for Electronic Database Search

(literacy OR “emergent literacy” OR “early literacy” OR reading OR “early reading” OR “beginning reading” OR language OR vocabulary OR “story comprehension” OR “story retelling” OR attention OR engagement OR attitude) AND (computer OR technology OR e-book OR eStorybook OR digital OR CD-ROM OR multimedia OR multimodal OR “talking book” OR “electronic book” OR “living book” OR “living storybook” OR “dynamic book” OR animated OR animation OR video OR software OR tablet OR iPad OR television) AND (storybook OR book OR “picture storybook” OR narrative OR narration OR story OR stories) AND (children or kindergartner or preschooler)

Appendix C

PRISMA Diagram of the Literature Search