Impact and Characteristics of Positive and Fearful Emotional Messages during Infant Social Referencing

Abstract

Studies of infant social referencing have indicated that infants might be more influenced by vocal information contained in emotional messages than by facial expression, especially during fearful message conditions. The present study investigated the characteristics of emotional channels that parents used during social referencing, and corresponding infants’ behavioral changes. Results of Study 1 indicated that parents used more vocal information during positive message conditions. Unlike previous findings, infants’ behavioral change was related to the frequency of vocal information during positive condition. For fearful messages, infants were more influenced by the number of multi-modal channels used and the frequency of visual information. Study 2 further showed that the intensity of vocal tone was related to infant regulation only during positive message conditions. The results imply that understanding of social context is important to make sense of parent-infant’s emotional interaction.

Infants’ ability to recognize emotion signals from facial and vocal expressions has been extensively studied in the last three decades. Studies of facial expressions have generally indicated that babies as young as 7 months of age can discriminate facial expressions of happiness, fear, and anger (Kobiella, Grossmann, Reid, & Striano, 2008; Neslon & Dolgin, 1985; Nelson, Morse, & Leavitt, 1979). Infants are also good at understanding emotional information through vocal messages, as studies have demonstrated that 5-month-old babies can successfully distinguish sad and happy vocal expressions (Walker-Andrews & Grolnick, 1983). When infants acquire locomotion and start to learn some social communicative skills, such as understanding gestures and language, their ability to interpret emotional signals is more reliably observed. One outcome is social referencing, in which an infant uses another’s emotional reaction to a situation to construct his or her own interpretation of that situation (Campos, 1983; Feinman, 1982; Walden, 1991).

Several studies have investigated the impact of facial and vocal information during social referencing. Mumme, Fernald and Herrera (1996) found that vocal messages influenced infants’ behavior during a fearful message condition, even without facial expressions. However, happy vocal signals alone did not encourage babies to approach novel stimuli. Using the visual cliff paradigm, Vaish and Striano (2004) reported that infants crossed the cliff faster during a voice-only condition and a voice plus face condition than during a face-only condition.

While the results of the previous studies imply that that vocal information may be more potent than face signals especially during negative (fearful) conditions, several issues cloud interpretation of those results. Most importantly, infants’ familiarity with positive and fearful emotional signals may differ (Baldwin & Moses, 1996). Negative expressions are less familiar to infants than positive expressions in general (Kaufmann & Schweinberger, 2004; Kuchuk, Vibbert, & Bornstein, 1986), and to receive negative emotional signals from facial expression is even rarer, as implied from previous studies (Malatesta, & Haviland, 1982; Nelson, & Dolgin, 1985. cf, Peltola, Leppanen, Palokangas, & Hietanen, 2008). Walden and Ogan (1988) found that some infants who participated in their study looked to be ‘confused’ especially during fearful message conditions. The authors indicated that the infants might feel this situation strange, as fearful situations in everyday contexts usually include some parental actions to accompany the fearful expressions. Walden and Kim (2005) reported that 18-month-old infants tended to look toward strangers more than mothers especially during fearful message conditions, the least familiar event for infants. Taken together, it is possible to argue that infants’ behavioral change followed by voice-only signals during fearful message conditions in previous studies might be partly due to unfamiliarity, not due to the impact of emotional messages per se.

The present study compared the characteristics of parents’ messages during positive and fearful conditions, and investigated whether the impact of each emotional signal differed by message condition. We tested whether and how the frequency and intensity of vocal, facial, and multi-modal manifestations of parental emotion expression influence infant’s reactions toward ambiguous events when mothers delivered positive and fearful messages.

STUDY 1

In Study 1, we focused on two aspects of parental emotional expressions during social referencing. First, we examined how mothers used each emotional channel during social referencing. Second, we are testing whether the frequency of each emotional channel used is related to infants’ behavioral change during positive and fearful social referencing. We wondered whether parental use of different emotional channels was related to changes in infants’ behaviors, and whether the pattern differed for positive and fearful messages.

Method

Participants

Forty-one typically developing infants between 17 and 25 month of age (M=20.6, SD=2.5) participated. Thirty-eight infants were Caucasian and 3 were African-American. Fourteen males participated with their mothers, two males with fathers. Twenty-three females participated with their mothers, one female with her father, and one with an aunt who provided daily care for the infant. Seventeen infants were randomly assigned to the positive message condition, while other 24 infants were assigned to the fearful message condition.

Stimuli

The experimental stimuli were several remote-controlled mechanical toys, both of which made unusual noises. The stimuli were ranged from 22 to 62cm in height. They were chosen based on previous studies (Walden & Kim, 2006; Walden & Ogan, 1988), so that none elicited extreme spontaneous positive or fearful reactions before infants received an emotional message. A pilot test and a previous study (Walden & Kim, 2006) has shown that most infants fell into the neutral range of affective states when they encountered the toys without emotional messages. Some studies have implemented a more complicated criterion in choosing stimuli (See Feinman et al., 1992), but others argued back that the uncertainty of a stimulus would not restrict referencing behaviors (Klinnert et al., 1986). For the present study, infants’ emotional states were the only criteria for choosing stimuli. One toy was used for the positive condition, and the other was used for the fearful condition (the selection of toys for each condition was counterbalanced across participants).

Procedure

Five infants were observed at their childcare centers and 36 at the Child Study Center at Vanderbilt University. The child was allowed to explore while procedures were described to the parent. Each parent was shown Izard’s (1971) photographs of posed happy and fearful emotions and practiced producing the expressions. Verbal messages were also modeled for each parent; positive messages were “Oh look at that! Nice toy!”, and fearful messages were “Ooo, look at that! Scary toy!” Parents were allowed to deviate from these scripts with the restriction that they not tell the child to do or not do anything.

Each dyad was randomly assigned to one message condition (positive message, N=17, or fearful message, N=24), thus one infant was exposed to only one emotional expression. The parent and infant were seated on the floor, and after a few minutes of play the parent took a position to the side and slightly behind the child, with both facing a table covered by a cloth. Each stimulus was presented in a block of seven 30-sec trials. During each trial the toy approached the child from behind the cloth and moved around for 20 sec, then the stimulus remained still and silent for 10 sec. The stimulus withdrew for 3-sec inter-trial intervals.

Parents were told to pose a neutral facial expression when each trial started, but to deliver assigned positive or fearful messages only when the child was looking toward them while looking at the child and the stimulus back and forth.

Measures

Emotional message channels

Four emotion channels, two visual (face, gesture) and two verbal messages (vocal tone, vocal content), were rated separately. Each videotape was coded by one of three trained coders who were blind to the hypotheses. Eight tapes were coded by two coders to monitor reliability. Agreements and disagreements were determined on a frame-by-frame basis (1/30^th sec) to compute Kappa.

Whenever the child looked toward the parent during the trials, the parent’s emotional expression was assessed. Each parental facial expression was categorized as being absent (the child did not look at the parent’s face), neutral, positive, negative, mixed or off-camera (Kappa=.61). Parents’ body cue/gestures were categorized as absent, neutral, positive, negative, mixed, or off-camera (Kappa=.76). The content of parental verbal communications was categorized as absent, neutral, positive, negative, mixed or unintelligible (Kappa=.82). Vocal tone cues associated with verbal communications were categorized as neutral, positive, negative, or mixed (Kappa =.79).

The parents’ visual and verbal communications were linked and segmented into units that consisted of the visual and verbal information present in each communicative channel for each message. Independent raters rated the appropriateness of parents’ signals for each condition on four fixed-choice categories (absent, appropriate, wrong/mixed, and neutral). Since parents gave a higher proportion of appropriate messages in the positive condition (87% vs. 61%, p<.05), only message units having at least one appropriate message in each message block were used for analyses. Eleven dyads had been excluded for failing to meet this criterion.

Infant behavior

During the trials, each instance in which the child touched the toy or made an intentional proximal movement to the toy was coded (Kappa=.95). The proportion of time the child exhibited proximal toy-directed behavior following the first parental message in the positive and fearful trial was computed.

Results and Discussion

The first set of analyses explored the characteristics of parental messages in the positive and fearful message conditions. An oneway ANOVA on the proportion of time parents gave messages indicated no significant differences between the positive and fearful message trials. Parents gave messages 26 percent of the time in the positive condition and 24 percent of the time in the fearful condition (SDs=.20 and .15, respectively). No sex, presentation order, or stimulus difference was found in any subsequent analyses.

A 2 (positive/negative) × 4 (communicative channels) ANOVA on the proportion of appropriate message time indicated a significant effect of communicative channels; F(3,120)=135.1, p<.001; and a channel by message interaction; F(3, 120)=30.3, p<.01. Post-hoc analyses revealed that a larger proportion of fearful messages contained appropriate gestures and vocal content than did positive messages. A greater proportion of appropriate positive messages contained vocal tone cues than did appropriate fearful messages. Facial expressions were judged to be positive for positive messages equally as often as they were judged to be negative for fearful messages. Relative to the other channels, facial expressions were less likely to be used and verbal content and vocal tone were more common (See Table 1). Thus, parents’ positive messages contained more vocal signals than negative messages, whereas their fearful messages contained more visual signals, but the signal tended to be gestural rather than facial.

Table 1.

Mean proportion of positive and fearful messages in which each channel was rated as appropriate (SD in parentheses).

	Message Condition
Communicative Channel	Positive	Fearful
Facial Expressions	.14 (.11)	.08 (.18)
Body Cues / Gestures*	.11 (.11)	.39 (.26)
Vocal Tone*	.87 (.12)	.59 (.34)
Verbal Content*	.62 (.25)	.74 (.20)

Note:

^*asterisk by the number indicated that the proportion in the positive message condition was significantly different from that in the fearful condition.

Correlations among the variables indicated that gesture cues are significantly related to infants’ behavior change only during fearful conditions. Vocal tone and verbal content cues are significantly correlated with infants’ behavior change only in positive messages (Table 2).

Table 2.

Correlations between behavior regulation and characteristics of parental messages in positive and fearful message conditions.

	Message Condition
	Positive	Fearful
Number of Channels used
All appropriate messages	.27*	-.26*
1-component Messages	-.11	-.12
2-component Messages	.33**	-.14
3 or 4 component Messages	.10	-.39**
Message Characteristics
Facial Cues	-.03	-.11
Body Cues	.02	-.37**
Vocal Tone Cues	.28*	-.24
Verbal Content Cues	.31**	-.24

Note:

^*p< .05,

^**p< .01

A stepwise regression analysis showed that proportion of time that verbal content occurred in the positive message condition was related to infants’ behavior F(1, 38)=16.2, p<.001. That is, more verbal content is related to a more approach to an ambiguous toy. Proportion of time that gestures were rated as negative in the fearful condition was also related to behavior change F(1,38)=13.7, p<.001. Infants withdrew from a toy more when they received more fearful gestures.

Messages containing either 1-, 2-, or 3 or more-communicate channels rated as appropriate were included as predictors of infant behavior. Variance in infants’ behavior was best accounted for by a two-variable model including 3 or more channel messages in the fearful condition (F(1,38)=11.2, p<.01) and 2-channel messages in the positive message conditions (F(1, 38)=9.8, p<.01). On the other hand, behavior change was not significantly related to message characteristics when only one channel was used (See Table 2). Thus, infants changed their behaviors more when parents delivered messages using multi-modal channels during fearful message conditions. For positive messages, two channels produced behavior change.

In sum, results of Study 1 indicated that parents tended to deliver more gesture signals during fearful conditions, whereas they used more vocal signals during positive conditions. Infants’ behavioral change also differed in positive and fearful conditions. More multi-modal emotion signals were related to infant negative behavior during fearful conditions. Infants tended to be more affected by vocal information only during positive conditions. In general, unlike previous findings (e.g. Mumme et al., 1996; Vaish & Striano, 2004), our results suggested that vocal signals might be more frequent and effective in positive messages than in fearful messages.

STUDY 2

Study 1 showed that vocal information may play a different role during positive and fearful emotional messages in guiding infants’ behaviors toward ambiguous stimuli. This finding was based on the relative frequency of affective channels used. It is still unknown which aspect of vocal information is most crucial.

Characteristics of vocal cues are an under-studied area, not only in developmental psychology but psychology in general, despite a long-standing idea that characteristics of vocal cues may represent different kinds of emotional states (Banse & Scherer, 1996). The present study specifically focused on one aspect of vocal cues; adult studies have shown that higher intensity of vocal tone may generate better accuracy in decoding emotions (Juslin & Laukka, 2001). In the present study, we tested whether intensity in hedonic tone is related to infants’ response in social referencing. Characteristics of vocal tone in a positive message condition and a fearful message condition were also compared. Because a previous study (Hornik & Gunnar, 1988) and our pilot work showed that infants were more intrigued by social or live objects than robot toys, we used a replica of a live animal to increase infants’ interest in the stimuli for this study.

Method

Participants

Twenty-six 18-month-old infants (15 boys, 11 girls) participated in an active social referencing experiment. All infants participated with their mothers. Twenty-five participants were Caucasian, and one was African-American. Three parents failed to give appropriate fearful messages and their data were excluded from analyses.

Stimuli

The experimental stimuli for this study were three replicas of live animals; a toy weasel with a ball, a wooden snake, and a plastic turtle. They were all similar in height and length (about 20cm), except for the snake, which was longer than other stimuli (about 60cm) but shorter in height. Every stimulus was randomly assigned to each trial, and there was no significant difference according to the type of toy for any dependent measures.

Procedure

Procedures for Study 2 were almost identical to those of Study 1, except that trials were lengthened to 60 sec, and every child for this study was exposed to both positive and fearful messages (that is, two emotional messages for this study were set as a within-subject factor). A female experimenter brought the stimuli to the room in which infants and mothers were located, and then left the room. Trials started when the experimenter left the room and closed the door. Instructions for parents were identical to those of Study 1. Immediately after the termination of the first trial, the next trial followed. The order of emotional expressions and toy selections for each trial was counterbalanced, that is, when a child was assigned to the positive message trial first, then the next trial would be the fearful message condition, and vise versa. The entire procedure was videotaped.

Measures

Intensity of vocal tone

Two graduate students who were different from those for Study 1 and were unaware of the hypotheses of the study coded the intensity of vocal tone. Both the ‘negativity’ and the ‘positivity’ of the first three parental messages in each condition were coded. We chose the first three messages because those contained the most ‘typical’ positive or fearful vocal expressions. Some infants behaved unexpectedly while trials continued (such as to cling to parents), and some parents often forgot to deliver instructed messages during the later time of a trial as well. On the other hand, only three parents who were excluded from the analyses failed to delivered appropriate messages for the first three messages. Thus, to maximize the convergent evidence for our measures while maintaining as many participants as possible, the first three messages emerged as the most suitable choice. All subsequent analyses were results from this duration of time.

Coders were trained to rate the intensity of one’s emotional vocal tone. They listened to only the audio clips from the videotapes, and asked to rate how high the intensity level of the message was on a 7-point scale (7: very intense, 1: no sign of any given emotional tone). Both positivity and negativity were measured regardless of message conditions, although it was almost meaningless to measure positivity during fearful conditions as almost all responses were 1(no fearful emotional tone), and vise versa. The final intensity scores of positive and fearful messages were calculated by taking an absolute value of their positivity-negativity score. Since only appropriate messages were included for the present study, high scores in the positive condition indicated highly intense positive messages; high scores in the fearful condition reflected highly intense fearful messages. Intercoder reliability for intensity of positive messages was r=.85, and r=.73 for fearful messages.

Affect and behavior regulation

A coding system was developed which comprised two scores, one reflecting affect and the other reflecting approach-avoidance behavior. Affect after the infant received emotional messages was coded every 10 seconds from videotape and ranged from “1”, extremely positive, to “5”, extremely negative. Criteria for affect included infant facial expression, overt laughing or crying behavior, and verbalizations. The overall affect score was generated by averaging the affect ratings for each condition. The Kappa reliability coefficient for the overall affect score was .75. Behavior regulation was also coded from videotape, based on infant’s behavior such as touching a stimulus or parents, extending their arms, pointing, reaching toward the parents, and other body locomotion related to stimuli, ranged from “1”, touch the toy, to “7”, touch the parents. Kappa coefficients for overall behavior score were .81.

Infant looking frequency

During each trial, infant looks were recorded whenever the infants turned and looked to parents. When more than two looks occurred within 3s, they were coded as one occurrence. Thus, the unit of infants’ looking data was a 3 sec ‘interval.’ Two trained graduate students who were blind to the hypotheses coded looks. Kappa was .75.

Results and Discussions

A t-test for parent message indicated that parents tended to deliver more intense messages in the positive (M=4.88) than in the fearful condition (M=4.25); t(22)=1.91, p=.069. The correlation between intensity of vocal tone in the positive and fearful conditions was not significant (r=.33, p>.11).

For infant regulation, correlations between the intensity of vocal tone and the two regulation scales were examined. The intensity of vocal tone was significantly correlated with both affect regulation (r=.56 p<.005) and behavior regulation (r=.65, p<.001) in the positive message condition. However, none was significant in the fearful condition (see Table 3). Contrary to previous studies, no correlation between intensity of vocal tone and the frequency of looks was significant. That is, infants did not necessarily look at parents more when they received more intense vocal signals.

Table 3.

Correlations between parental message’s vocal intensity and infant reactions

	Vocal intensity
	Positive	Fearful
Infant Regulation
Affect Regulation	.56**	-.30
Behavior Regulation	.65**	-.10
Infant Looking
Frequency of looks to parents	.31	-.21
Frequency of looks to stimuli	.25	.22

^**p< .01

On the other hand, different relations between the frequency of looks and the intensity of vocal tone during positive and fearful conditions were shown when infants were divided into two groups. Based on median intensity scores, a group of infants who received high intensity of vocal tone during the experiment and those who received low intensity of vocal tone were created. The high intensity group during the positive condition (M=5.5, n=15) showed more looks to parents (t(21)=2.35, p<.05), and tended to show more affect regulation (t(21)=2.05, p=.053), that is, showed more positive affect in the positive condition, than the low intensity group (M=3.7, n=11). However, the high intensity group during the fearful condition (M=5.5, n=13) did not differ in regulation or looking from the low intensity group (M=2.9, n=13). Finally, for looking frequency to parents, infants looked at parents more often during the fearful condition (.29) than the positive condition (.25). t(22)=2.15, p<.05. In conclusion, consistent with the results of Study 1, parents tended to deliver stronger vocal messages during positive than fearful messages. Intensity of vocal tone was related to infant regulation only during the positive message condition.

GENERAL DISCUSSION

The present study examined the characteristics of parents’ messages during social referencing, and whether visual and vocal information each played a role in changing infants’ behaviors following positive and fearful messages. In general, vocal characteristics of messages were more common during positive messages, whereas visual and multi-modal information was more frequent during fearful conditions. Parents provided more vocal cues during positive conditions, and the vocal tone of the messages tended to be more intense. Infants’ regulation toward stimuli following positive messages (maintaining positive affect toward the stimuli) was also related to both quantity and the strength of the vocal cues. On the other hand, infant regulation following fearful messages (withdrawing from the stimuli) was best predicted by multi-modality of the messages and gestural cues parents delivered.

Two noticeable findings from the present study include strong effects of vocal messages during positive conditions, and infrequent use of facial cues relative to other cues. Our finding implies that infants’ experience with positive and fearful situations may influence their reactions. As previously mentioned, receiving face-only signals hardly occurs during everyday situations, and although receiving vocal-only signals may occur occasionally, those cases are predominantly positive (e.g. mom is exchanging verbal conversation with a baby while preparing meals). During fearful situations, on the other hand, the most affective interaction is likely to be multi-modal (e.g. when a baby is touching an electrical outlet, parents may call to, run to, and show a fearful facial expression to the baby). The results of the present study are consistent with this observation, as vocal cues were adequate to change infant’s behaviors only during positive message conditions.

The relatively weak impact of facial expression was counterintuitive, as facial expressions have been considered one of the most powerful emotion-conveying signals in adult studies (Ekman & Friesen, 1969), and also a key factor in discriminating the valence of emotional expressions in infant studies (D’Entremont & Muir, 1999). The result of the present study, however, does not mean that facial information was unimportant; rather, facial expressions were crucial in changing infants’ behavior when combined with other emotional cues, especially during fearful conditions. Results indicated that infant regulation during fearful conditions was best predicted when three or more emotional cues were present, as opposed to two channels being effective during the positive conditions. That is, facial expressions might be most effective in a context in which other emotional cues are also available, which is typically what infants encounter during everyday situations.

Most previous studies investigating infant social referencing for facial and vocal messages used single emotional expression, that is, displaying either facial or vocal information (e.g. Mumme et al., 1996). On the other hand, the present study allowed parents to use multimodal channels when delivering emotional messages. The methods the present study used possessed both pros and cons. Since each emotional cue was not experimentally manipulated, the interpretation of the results was based on correlations. On the other hand, results of previous studies in which visual and vocal signals were arbitrarily separated have also been criticized because the methods could generate unfamiliar contexts lacking external validity (Baldwin & Moses, 1996; Caron et al., 1985; Flavell, 1985). We are suggesting that previous results showing a strong impact of vocal information during negative conditions might indeed be due to infants’ unfamiliarity of the event. This conclusion would not be obtained if vocal and visual cues were delivered separately, failing to capture the multi-modal nature of emotional transactions in which parents and children participate.

At the same time, we would also like to emphasize that this study was still an experimental study, not a naturalistic observation. That is, it is still likely that the situation was unusual. For example, mothers were told not to initiate interactions with the infants, to make sure that infants could actively initiate social referencing. Although this is common in social referencing studies (cf. Rosen, Adamson, Bakeman, 1992), there have been controversies whether the inactiveness of mothers might produce unfamiliarity to infants, and thus might influence their reactions (e.g. Walden & Ogan, 1988). At the same time, if parents had freely interacted with the infants, that could also create interpretational issues. Infants’ reaction might have no longer been a social referencing behavior, as they might be affected by specific parental behaviors, not by emotional expressions parents delivered. In addition, some parental actions involved physical movement, which could draw infants’ attention and thus distract their focus on experimental settings and parents’ faces through which emotional expressions were delivered (Bahrick, Gogate, & Ruiz, 2002). Thus, the findings of the present study should be interpreted with full understanding of advantages and disadvantages of experimental designs.

In sum, the findings of the present study implied that vocal information, which was considered a strong variable in changing behavior during fearful condition in previous studies, may indeed be more related to infant behavioral changes during positive situations. Emotions expressed on the face are important when accompanied by other emotional cues. The present findings imply that infants’ regulation of emotion is a result of dynamic process in which infants and the social partners communicate in social environment. Thus, it is imperative to consider the affordance of the social environment in which the emotional interactions occur. Understanding emotion is not a mere perception of a particular physical stimulus, but it is the understanding of people, people’s minds, and interactions in context.