An Attentional Goldilocks Effect: An Optimal Amount of Social Interactivity Promotes Word Learning from Video

Abstract

Television can be a powerful education tool; however, content-makers must understand the factors that engage attention and promote learning from screen media. Prior research suggests that social engagement is critical for learning and that interactivity may enhance the educational quality of children’s media. The present study examined the effects of increasing the social interactivity of television on children’s visual attention and word learning. Three- to 5-year-old (M_Age = 4;5 years, SD = 9 months) children completed a task in which they viewed videos of an actress teaching them the Swahili label for an on-screen image. Each child viewed these video clips in four conditions that parametrically manipulated social engagement and interactivity. We then tested whether each child had successfully learned the Swahili labels. Though 5-year-old children were able to learn words in all conditions, we found that there was an optimal level of social engagement that best supported learning for all participants, defined by engaging the child but not distracting from word labeling. Our eye-tracking data indicated that children in this condition spent more time looking at the target image and less time looking at the actress’s face as compared to the most interactive condition. These findings suggest that social interactivity is critical to engaging attention and promoting learning from screen media up until a certain point, after which social stimuli may draw attention away from target images and impair children’s word learning.

Since the premier of Sesame Street in 1969, many stations have aired programming that aims to both entertain and educate preschool-aged audiences. But the extent to which children can learn through television remains unclear. Research suggests that social interactivity may enhance the educational quality of children’s television (Richert, Robb, & Smith, 2011; Rice, Huston, Truglio, & Wright, 1990). However, the direct effects of increasing television interactivity on children’s learning have not been demonstrated.

Past studies on learning through media have focused on the role of social information. Many studies have revealed a “video deficit” — a gap between the amount of information children learn from real people compared to the amount they learn from people talking to them via television (Anderson & Pempek, 2005; Kuhl, 2007; Troseth, 2003; Troseth, Saylor, & Archer, 2006). In one study, 9- and 10-month-olds exposed to Mandarin, through reading children’s books and playing with toys with a native Mandarin speaker, performed significantly better on tests of Mandarin speech sound discrimination compared to peers exposed only to English (Kuhl, Tsao, & Liu, 2003). However, infants exposed to movies of Mandarin speakers reading books and playing with toys for the same amount of time did not demonstrate phonetic learning. In another study, 2-year-olds were able to locate a toy after watching an experimenter hide it in person but not when they watched it being hidden through a video feed (Troseth & DeLoache, 1998).

Younger children may face a particularly strong need for live social interaction in order to learn language. In a study of verb learning, children between the ages of 2 and 4 years were able to learn novel verb-action pairs through a combination of educational television clips and live teaching demonstrations by an experimenter (Roseberry, Hirsh-Pasek, Parish-Morris, & Golinkoff, 2009). However, when the live interaction component was removed from the learning phase, only older children learned the new verbs. The researchers hypothesized that the benefits of live interaction derived from children’s heightened arousal in social settings and the experimenter’s ability to establish joint gaze to better direct children’s attention toward the named action. This idea is supported by the work of Krcmar, Grela, and Lin (2007), who examined 15- to 24-month-olds’ learning of object names under different conditions, including among others, a joint reference condition in which an experimenter labeled the object when the child attended to it, and a children’s programming condition, in which children learned an object label from an edited Teletubbies clip. Children learned most successfully in their joint reference condition, highlighting the important role of social engagement in directing attention toward relevant stimuli to promote learning. Taken together, this work supports the hypothesis that learning may be “gated” by live social interaction (Kuhl, 2007).

Still, research suggests real-world digital media can promote word learning. Allen and Scofield (2010) found, for example, that 2-year-olds could learn object labels from videos and then transfer those labels to real-world settings. Additionally, children who spent more time watching age-appropriate television programming demonstrated greater vocabulary growth than peers who watched fewer hours of these shows (Rice, Huston, Truglio & Wright, 1990; Linebarger & Walker, 2005). In one study, 3-year-olds who watched Sesame Street developed larger vocabularies than their non-viewing peers (Rice et al., 1990). In another study, time exposed to certain children’s programs was correlated with larger vocabularies and higher expressive language scores in 2.5-year-olds (Linebarger & Walker, 2005).

One explanation that may reconcile these conflicting findings is that children learn from television only when it effectively captures their attention and elicits active participation during viewing, an idea supported by the finding that infants were more responsive to television when parent co-viewers asked questions, provided labels, or made abstractions about video content (Barr, Zack, Garcia, & Muentener, 2008). Similarly, children were more capable of finding a toy after watching it being hidden through a live video feed when experimenters interacted with them through the feed prior to object retrieval (Troseth et al., 2006). Three and 4-year-olds were also able to learn more letters and numbers from Sesame Street clips when adults sat with them, asked them reinforcing questions (e.g. “What letter is that?”), and provided contingent feedback during viewing (Reiser, Tessmer, & Phelps, 1984).

Furthermore, integrating contingent social interaction into screen-based learning may reduce, or even erase, the “video deficit” (Lauricella, Pempek, Barr, & Calvert, 2010; Nielsen, Simcock, & Jenkins, 2008; Roseberry, Hirsch-Pasek, & Golinkoff, 2014). For example, 2- to 3-year-old children who interacted with an experimenter over Skype prior to engaging in a Skype-based word-learning task learned novel verbs equally as well as children who engaged in the learning task with a live experimenter (Roseberry et al., 2014). But children who watched a video recording of an experimenter’s interactions with a different child prior to the word-learning task failed to learn the novel verbs. Nonetheless, as of yet, television programs cannot broadcast contingent social interactions into children’s homes. Thus it is important to examine strategies to promote word learning that can more feasibly be implemented in existing children’s programs. To our knowledge, the effects of parametrically increasing the social interactivity of prerecorded digital media on word learning have not been tested directly.

Here we examined the effects of increasing the interactivity of a television clip on children’s word learning. To do so we designed video clips in which a television actress taught children Swahili words across four different conditions that parametrically manipulated social engagement and interactivity. The majority of the reviewed literature, as well as children’s programming, is focused on preschool-aged children. As such, we tested 3-, 4-, and 5-year-old preschoolers on our novel paradigm. Based on the findings of Roseberry et al. (2009), we reasoned that the effects of social interactivity may impact younger word learners differently relative to older, more language-experienced preschoolers. Specifically, we expected the more interactive conditions to elicit greater social engagement and enhanced attention to the video clips for all subjects in our sample. But we hypothesized that younger children may benefit more from this greater attentional engagement during word learning, and thus demonstrate a greater difference in their ability to learn from video clips in more versus less interactive conditions.

Method

Participants

Sixty-four children (range 3;2 – 5;11; 40 boys; M_Age = 4;5, SD = 9 months) participated in the study. Of these, N = 22 were 3-year-olds (M = 3;7, SD = 2.8 months), N = 25 were 4-year-olds (M = 4;6 SD = 3 months) and N = 17 were 5-year-olds (M = 5;4, SD = 3 months). An additional three children were tested but excluded from analyses for failure to complete the task. Participants were screened via parental report to ensure that they had no previous or current known learning disabilities, uncorrected visual or auditory impairments and/or preterm birth. Participants were recruited from the community and through the Department of Health. Parents gave consent for their children’s participation, and families were compensated for their time. According to parental report, 78.1% of participants lived in homes in which only English was spoken and 15.6% of participants lived in homes in which English and one other language were spoken. No participants had any exposure to Swahili. The parents of four children did not provide language information.

Stimuli & Script

The stimuli for this study comprised four different video clips each filmed in four different ways, for a total of 16 unique clips. All clips were 30-seconds in duration and featured the same actress. The background of every video was the same, featuring a cartoon sun, cloud, giraffe, and tree spread across the four corners (Figure 1). The goal of each video was to teach participants to name one of these objects in Swahili. All of the Swahili words were two syllables with the emphasis placed on the first syllable. All videos began and ended with the actress on a safari and taking pictures of her surroundings.

Figure 1.

Example of a video task screen. Each 30-second video clip featured the same actress and background. In each video, the actress taught participants the name of one of the background images in Swahili — wingu [cloud], jua [sun], miti [tree], or twiga [giraffe].

In the Very Low Interactivity Condition, the actress did not make eye contact with the camera, and said, for example, “I know how to say ‘giraffe’ in Swahili,” and then took a picture of the relevant giraffe. When the actress clicked her camera, a red circle appeared around the giraffe. The actress then said the word in Swahili twice, and then, “That’s how you say ‘giraffe’ in Swahili.” In the Low Interactivity Condition, the script remained the same as in the Very Low Interactivity Condition, except the actress made eye contact with the camera after repeating the Swahili word twice and while looking out at the participant added, “Do you know how to say giraffe in Swahili?” The actress then immediately resumed taking pictures. Medium Interactivity Condition videos were nearly identical to those of Low Interactivity, except before the actress took pictures at the end, she paused for four seconds and looked at the camera presumably waiting for the participant’s response. High Interactivity Condition videos were nearly identical to those of Medium Interactivity, except after the pause, the actress said, “That’s right! You’re speaking Swahili, just like me.” Sixteen videos were created so that each of the four words appeared in each of the four conditions. All videos were identical in length.

Each word was presented once per child for a total of four trials per participant. Condition order was counterbalanced across participants. Test trials displayed four images in a row in the center of the touch screen (Figure 2). These four images included either all four pictures that corresponded to the Swahili words taught in video clips or three pictures that corresponded to the familiar Swahili words and one novel catch image, which was either a leaf or flower, designed to increase task difficulty. Novel catch images appeared equally across conditions. Each test screen was accompanied by an audio recording of the actress’s voice that said, “Touch the Swahili word.” The location of the correct image was counterbalanced across participants.

Figure 2.

Example of a test screen. Test screens comprised four images. While participants viewed the screen, a recording of the actress’ voice instructed them to touch one of the images. For example, they might hear, “Touch the twiga,” to which the correct response would be pressing the image of the giraffe.

Video Task Procedure

Participants were randomly assigned to one of four video groups. They then watched the experimenter do a demonstration trial, which consisted of a High Interactivity Condition video that taught the Swahili word for truck: lori. During the demonstration, the experimenter said the Swahili word out loud during the actor’s pause to demonstrate the interactive nature of the videos. After the video, a test screen came up, accompanied by an audio recording of “Touch the lori.” The experimenter touched the truck and explained that she was doing so because lori means truck in Swahili. Participants were then told that they had an opportunity to practice what they had just seen. They then watched another practice video that used the Swahili word for elephant: tembo. When the test screen came up, the experimenter did not advance forward until the participant had correctly pressed the image of the elephant. Participants then saw a screen with images of the four words they were going to hear in Swahili and the experimenter ensured the participants knew the English names of all four images. Participants were told they were going to watch more videos, and that they would learn two words in a row and should try to remember both of them.

Participants then watched a set of two 30-second videos, each presenting a different word in a different condition. A one-second-long blank screen followed each video to separate them from each other. After participants viewed two videos, they saw two test screens, each asking them to touch the picture of one of the words they had just learned. Participants had an unlimited amount of time to respond and were told to “make a guess” if they were unsure. After participants responded to the second test screen, they were told they were going to watch two more videos. The testing procedure was then repeated with a different set of two videos, which included the remaining two words in the remaining two conditions. The pairing of the videos was counterbalanced within each video group, across participants. The order of the videos within each video pair was randomized for each participant. The order of the test screens within each testing pair was also randomized for each participant. The touch screen recorded the accuracy per trial.

Eye-tracking Procedure

Of the 64 participants who completed the study, the final subset of N = 25 (M_Age= 4;3, SD = 9 months) were eye tracked while they completed the task. The eye-tracking condition was added after the initial behavior-only data collection revealed that word learning was better in the Medium Interactivity Condition than the High Interactivity Condition. We added an additional 25 participants with eye tracking to better understand the reasons for these results and to specifically investigate how patterns of visual attention in the Medium and High Interactivity conditions may have been contributing to differential word learning.

A remote eye tracker (SensoMotoric Instruments RED system) was used to record participants’ eye movements. Prior to testing, each participant’s point-of-gaze (POG) was calibrated using a 5-point protocol provided by the SMI Experiment Center software and then validated using a 4-point procedure. Based on the data from validation, we computed the deviation between each subject’s point of gaze and the location of a screen target. We used the tools provided by the SMI BeGaze analysis software to draw Areas of Interest (AOI) around the four background target images and the actress’s face. When the face AOI was closest to one of the background AOIs, the visual angle between them was 2.5°. Thus we used the deviations computed from the validation procedure to identify participants whose calibrations were not sufficiently accurate (<2.5°) to determine whether they were looking at the actress’s face or one of the background AOIs (N = 10 participants). For the remaining participants included in our analyses (N = 15, M age = 4;0, SD = 8 months), average horizontal deviation was .89° (SD = .40°) and average vertical deviation was .97° (SD = .37°), enabling assessments of eye movements to the face and background images.

Language Measure

Participants were administered Tests 1–7 and Test 9 from the Woodcock-Johnson III Tests of Cognitive Abilities. Test 1 yielded a raw verbal comprehension score for N = 62 participants, which was calculated by summing participants’ scores on the picture vocabulary, synonyms, antonyms, and verbal analogies subtests.

Results

Effects of Condition by Age

Our primary aims were a) to determine whether increasing the level of interactivity in a video clip would enhance children’s word learning, and b) whether this effect differed across our age range. Trials were coded categorically as either correct or incorrect and treated with statistical methods appropriate for binary variables. A Cochran’s Q test identified significant differences between Interactive Conditions, χ² (3, N = 64) = 10.43, p = .015. Planned McNemar’s tests revealed significantly better in accuracy for Medium Interactivity videos relative to Very Low Interactivity videos, M₃ = .63, CI₃ = [.50, .74], M₁ = .38, 95% CI₁ = [.26, .50]; χ² (1, N = 64) = 8.65, p = .003, and a trend-level improvement between Medium Interactivity videos and Low Interactivity Videos, M = .45, 95% CI = [.34, .58], χ² (1, N = 64) = 3.23, p = .072. No other accuracy differences between Very Low, Low, or High Interactivity Conditions were significant (all ps > .12)

We further examined the effects of Interactive Condition as a function of age (Figure 3). Using one-sample binomial tests, we compared participants’ performance on the task to chance level accuracy (4 alternatives on each test screen results in a chance level accuracy of .25). Three-year-olds performed above chance only on Medium Interactivity videos (p = .01). In contrast, 4-year-olds performed above chance on Low, Medium and High Interactivity videos (all ps < .005). Five-year-olds performed above chance on all interactive conditions (all ps < .005). These findings suggest that older children were able to learn from the video stimuli regardless of their level of social interactivity. However, increasing social interactivity helped learning in younger age groups. In support of this interpretation, we found that the three age groups performed significantly differently from each other in the Low Interactivity condition, Pearson χ² (2, N = 64) = 6.93, p = .03, and trended toward performing significantly differently from each other in the Very Low Interactivity condition, Pearson χ² (2, N = 64) = 5.37, p = .068, with accuracy in these conditions improving with age. There were no group differences in the Medium Interactivity condition, χ² (2, N = 64) = 2.26, p = .32, and the High Interactivity condition, χ² (2, N = 64) = 1.30, p = .523.

Figure 3.

Condition accuracy by age group. Error bars indicate 95% confidence intervals. Asterisks indicate performance that was significantly above chance (.25). All ps ≤ .01.

Visual Attention

An unexpected pattern emerged in the data. Performance in the Medium Interactivity condition surpassed performance in the Very Low Interactivity condition, while performance on the High Interactivity condition, which was designed to be more interactive due to its inclusion of non-contingent positive feedback, did not. We thus examined the extent to which differences between the Medium Interactivity condition and the High Interactivity condition specifically elicited different distributions of attention orienting. We eye tracked a subset of participants across ages. For each interactive condition, we calculated the proportion of time participants spent looking at the actress’s face and the target image (Face or Target Image/Total time spent looking at the screen for that trial). A two-way within-subjects analysis of variance was conducted with the within-subjects factors of Interactive Condition (Medium Interactivity Condition and High Interactivity Condition) and Attention Distribution (object and face).¹ We found a main effect of Attention Distribution, F(1,14) = 96.63, p < .001, as well as an Interactive Condition x Attention Distribution interaction effect, F(1, 14) = 4.47, p = .05. Figure 4 shows that while watching High Interactivity videos, children demonstrated an increased proportion of looking to the face relative to the object. These results suggest that increased social interactivity of High Interactivity videos compared to Medium Interactivity videos may have impaired word-learning by drawing children’s attention away from the target image during encoding.

Figure 4.

Proportion of total looking time to the face and object of interest in the Medium Interactivity condition vs. the High Interactivity condition.

Role of language ability

Finally, we examined the agents of change in performance across age groups. Specifically, older children may have performed better than younger children on the less interactive conditions due to their advanced language expertise. We divided our sample into three groups based on raw verbal comprehension scores from the Woodcock-Johnson III Test of Cognitive Abilities: (a) weak verbal scorers (N = 22), who correctly answered between 8 and 13 items (M = 11.3, SD = 1.42); (b) average verbal scorers (N = 20), who correctly answered between 14 and 21 items (M =17.75, SD = 2.2); and (c) strong verbal scorers (N = 20), who correctly answered between 22 and 29 items (M = 24.45, SD = 2.46). Though raw verbal comprehension scores correlated with age, r(60) =.67, p <.001, the weak, average and strong scoring groups contained children from different age groups, with respective mean ages for weak verbal scorers of 3;10 (SD = 7 months, range 3;2 – 5;1), average verbal scorers of 4;5 (SD = 7 months, range 3;4 – 5;8), and strong verbal scorers of 5;0 (SD = 6 months, range 4;3 – 5;10). We compared group performance in each condition to chance level (.25) using one-sample binomial tests. Weak verbal scorers performed above chance in the most interactive conditions, the Medium Interactivity condition, p = .03 and the High Interactivity condition, p = .003. Average verbal scorers performed above chance in the Low Interactivity condition, p = .035, the Medium Interactivity condition, p < .001, and the High Interactivity condition, p = .01, while strong verbal scorers performed above chance on all conditions, (all ps < .005). This pattern suggests older children’s ability to learn words under all levels of social interactivity was in part driven by their greater experience and facility with language.

Discussion

Our findings indicated that children are able to learn novel words through digital media created in the style of popular children’s programming. Though 5-year-olds were able to learn under all conditions, increasing the social interactivity of the clips promoted word learning in 3- and 4-year-olds. Interestingly, participants only benefited from increasing interactivity to a point, after which the social elements of the stimuli seemed to draw visual attention away from the object of interest. Thus there seems to be an attentional Goldilocks effect for learning from digital media — an optimal amount of social interactivity that engages children without distracting them from target information. This term ‘Goldilocks effect’ refers to Kidd, Piantadosi and Aslin (2012) who found that infants spent the most time looking at sequences of images that were neither too simple nor too complex.

The idea that an optimal amount of social interactivity promotes learning from digital media is supported by past work that found that social stimuli are useful for learning in so far as they cue attention toward relevant information (Wu & Kirkham, 2010; Yurovsky, Wade, & Frank, 2013). In Yurovsky et al.’s (2013) word-learning paradigm, attention to the face was a negative predictor of learning — children who followed social cues toward their referents learned new word-mappings, while those who remained focused on the speaker’s face did not. Other work has shown that shared gaze and joint attention, in which gaze is mutually focused on a target object, promote word learning in both typical and atypical development (Brooks & Meltzoff, 2005; Brooks & Meltzoff, 2008; Mundy, Sigman, & Casari, 1990). Similarly, Roseberry et al. (2009) found that social engagement was especially critical to promote younger children’s learning from media. Our results are consistent with this work, suggesting that an optimal level of interactivity may be especially important to support learning from video in young preschool children, where word learning is still developing rapidly (Gathercole, Willis, Emslie, & Baddeley, 1992). Our own data indicated that age may be a proxy for the number of words or language experience children have. We found that children with lower verbal comprehension scores benefitted the most from increasing levels of social interactivity during word learning.

Krcmar et al. (2007) helped explain the “video deficit” by arguing that children’s programs do not successfully draw attention toward relevant information. However, our findings provide an alternative view. We suggest that any general claims about the attentional patterns elicited by children’s programming should be based in the specific contents and structure of a children’s television clip. The Medium Interactivity and High Interactivity conditions elicited different patterns of visual attention that differentially affected word learning even as they were identical until the positive feedback was given. The idea that subtle differences in the design of children’s television can affect word-learning is supported by past research that suggests that differences between children’s programs can affect their ability to support vocabulary growth, with shows like Blue’s Clues and Dora the Explorer leading to greater vocabulary growth than shows like Teletubbies and Barney & Friends. (Linebarger & Walker, 2005).

The idea that social engagement affects learning from digital media through modulating attention is further supported by previous research on the effects of sound on children’s imitation from television. This work specifically points to the ways in which programs’ educational efficacy can be helped or hindered by subtle changes in design that may influence attention. For example, 12- and 18-month-olds were able to imitate actions demonstrated via video clips when they included matched sound effects, but unable to do so when the sound effects were out of sync with the visual demonstration (Barr, Wyss, & Somander, 2009). Additionally, though infants had difficulty learning from videos with cartoon soundtracks, adding sound effects matched with actions shown on screen improved their ability to imitate actions from the videos 24 hours later (Barr, Shuck, Salerno, Atkinson, & Linebarger, 2010). In this case, the soundtrack seemed to draw attention away from relevant information, while the added matched sound effects successfully directed attention back to the target stimuli. These data also suggest that our finding that social interactivity is beneficial for learning to the extent that it draws attention toward target information may not be specific to the social domain — any elements of media that aim to increase overall engagement may detract from learning unless they are specifically designed to draw attention toward relevant information.

Our study did not include a live person condition and so we cannot make any definitive claims about whether increasing interactivity can completely close the “video deficit” noted in past research (Anderson & Pempek, 2005; Troseth, 2003; Troseth et al., 2006; Kuhl, 2007). Additionally, in our design, children’s learning may have benefitted from the demonstration trial. In real-world settings, children may not watch television alongside adult coviewers who explicitly tell them to try to learn words from screen media. Of course, the use of a demonstration trial could not have influenced the pattern of results observed across conditions. Children’s better performance in the Medium Interactivity condition suggests the social content of television clips themselves played a critical role in driving the attentional patterns that supported word learning. Our results indicate that the beneficial attentional effects of live social interaction can be replicated on screen to at least diminish, if not completely close, the digital learning gap.