Skip to main content
PLOS One logoLink to PLOS One
. 2020 May 29;15(5):e0233592. doi: 10.1371/journal.pone.0233592

Gradual positive and negative affect induction: The effect of verbalizing affective content

Charlotte Out 1,*, Martijn Goudbeek 1, Emiel Krahmer 1
Editor: Hedwig Eisenbarth2
PMCID: PMC7259663  PMID: 32469910

Abstract

In this paper, we study the effect of verbalizing affective pictures on affective state and language production. Individuals describe (Study I: Spoken Descriptions of Pictures) or passively view (Study II: Passively Viewing Pictures) 40 pictures for the International Affective Picture System (IAPS) that gradually increase from neutral to either positive or negative content. We expected that both methods would result in successful affect induction, and that the effect would be stronger for verbally describing pictures than for passively viewing them. Results indicate that speakers indeed felt more negative after describing negative pictures, but that describing positive (compared to neutral) pictures did not result in a more positive state. Contrary to our hypothesis, no differences were found between describing and passively viewing the pictures. Furthermore, we analysed the verbal picture descriptions produced by participants on various dimensions. Results indicate that positive and negative pictures were indeed described with increasingly more affective language in the expected directions. In addition to informing our understanding of the relationship between (spoken) language production and affect, these results also potentially pave the way for a new method of affect induction that uses free expression.

1. Introduction

Speaking about emotionally meaningful events is not a neutral act: it affects us. When we experience something positive, like getting a raise, we feel happy, and, conversely, explaining that we did not get a promotion makes us feel sad. This relationship between speaking and feeling is partly there because the event we talk about is inherently affective, but it might also be because verbalizing an emotionally meaningful fact (“I got a raise. I finally got a raise!”) amplifies or even induces the emotions we experience.

In this paper, we investigate the relationship between language production and affect. Inspired by the Velten method [1], we present a procedure where participants are exposed to pictures that increase or decrease in valence as the experiment progresses. In contrast to the Velten method that relies on reading aloud affective fixed self-referential statements, we use spontaneous spoken descriptions of affectively charged pictures. To elucidate the relationship between affective state and language production, we explore the content of the verbal descriptions, using the affective categories of the Linguistic Inquiry and Word Count (LIWC; [2]). To gauge the role of verbal description in eliciting affective reactions, we explore the difference between describing these pictures and silently viewing them.

1.1. The Velten method

The theoretical aim of Velten’s [1] study was to find evidence for the efficacy of a type of cognitive therapy, focusing on making the patient aware of how their own verbal interpretations of events influence their affective responses. Predicting that affective phrases would indeed elicit corresponding responses, Velten created an affect induction method to elicit positive (elation) or negative (depressive) affect by having participants read out loud 60 sentences that gradually increased in affective content. The first sentence in both conditions was ‘Today is neither better nor worse than any other day’. In the positive condition, it was followed by sentences such as ‘Things look good, things look great!’ with the final sentence being ‘God, I feel great!’. In the negative condition, it was followed by sentences such as ‘It often seems that no matter how hard I try, things still go wrong’, with the final sentence being ‘I want to go to sleep and never wake up’. Velten also included a neutral condition, containing sentences as ‘The review is concerned with the first three volumes’ and ‘West Samoa gained its independence in 1965’. After reading the statements, various measures were obtained, including several cognitive and behavioral tasks. In one of the tasks, participants were asked to choose from a long list of adjectives which adjectives applied to them (Multiple Affect Adjective Check List, Today Form (MAACL) [3], and the experiment leader kept track of the number of words the participant uttered during the tasks.

As Velten predicted, participants who read the negative statements, compared to those who read the positive statements, ticked significantly more adjectives in the Depression Scale, one of the five emotion subscales of the MAACL, and in general uttered less words [4]. Velten concluded that these results, together with his other measurements, indicated that the induction method was effective: participants reading the negative statements felt more depressed, and participants reading positive statements felt more elated. Participants reading the neutral statements generally fell between the scores of elation and depression, implying that no effect on emotion took place [1]. After its initial development, the Velten method has been widely used in the following decades (e.g., [5]), and the effects have frequently been replicated (e.g., [57])

1.2. The relationship between verbal expression and affective state

In general, the Velten method appears to be highly relevant for researchers studying the relationship between affect and language, which has been under scholarly debate for only about a decade [8], resulting in various hypotheses. One hypothesis is based on the psychological constructionist approach. According to this approach, language has a pivotal, although not sufficient, role in perceiving and experiencing emotions. The approach suggests that the lexicon of (affective) words at our disposal is essential to make meaning of, and therefore shape, our affective experiences, for example, by turning general, vague feelings of displeasure (‘this doesn’t feel right’) into concrete emotions (‘I feel lonely’, [8]). According to Barrett [9], this categorization is learned from infancy, and depends on the social (and cultural) environment. To a certain extent, emotions are created by naming, and therefore categorizing (and experiencing) them.

However, the literature shows conflicting results with respect to the impact of verbalization on affective state. On the one hand, there is support that expressing affective content can attenuate the affective experiences, e.g., via ‘affect labelling’. When individuals use affect labelling and put their emotions into words this can result in a decrease in the intensity of the affective, often negative, experience (see, e.g., [1012]). Therefore, some consider affect labelling to be an unintentional or incidental form of affect regulation (see e.g., [13]). In one study, Fan et al. [10] studied affect labelling in naturally occurring, spontaneous emotional expression on Twitter, looking at tweets starting with ‘I feel…’, followed by an adjective or adverb, written by approximately 74.487 different Twitter users. A dictionary based affect detection algorithm, VADER (Valence Aware Dictionary and Sentiment Reasoner; [14]) was used to detect possible changes in affective content of the tweets six hours after (and before) the affect labelling took place. Their results showed that for most individuals, immediately after using affect labelling in a tweet, the level of affective content of their tweets decreased, before returning to baseline [10]. Negative emotions returned to baseline fairly rapidly, with a decay half-life of five minutes, while for positive emotions a less rapid reduction was observed with a decay half-life of eleven minutes. The authors conclude that their findings are in line with literature on the attenuating effects of affect labelling.

In contrast to the findings summarized above, putting emotions into words can also result in an enhanced affective experience. For example, Ortner [15] presented participants with neutral and negative pictures. First, they merely viewed 10 neutral and 10 negative pictures. Then, for the next 10 neutral and 10 negative pictures, participants were asked to either passively view, reappraise (reinterpret the pictures in a way that it no longer seemed negative) or emotionally label pictures (observe which emotions they experience and utter their labels, e.g., ‘there is… anger’). The results showed that participants using affect labelling reported stronger affective states than those reappraising or only viewing the pictures. Ortner [15] suggests that the individuals who verbally described their emotional reactions to the affective pictures in their own words created a heightened awareness of them and therefore, experienced more intense affect.

Finally, in expressive writing, the verbalizing of emotions often results in an initial increase, followed by a decrease in emotional intensity. Expressive writing is a well-known and successful technique to deliberately reduce (unwanted) negative affect and distress in the long run (e.g., [16, 17]). For this technique, individuals are asked to write 15–20 minutes about a traumatic or personally emotional event for several consecutive days [18]. In contrast to Fan et al. [10], the decrease in negative emotions was not immediate: during, and immediately after writing, individuals usually reported feeling worse (e.g., [16,19]), although at least one study reports that individuals scored higher on positive disposition shortly after the last expressive writing episode [17]. In all these studies, participants experienced a decrease in negative affect in the long run, even while feeling worse immediately after the linguistic expression of the emotional event.

1.3. Describing affective pictures

The findings discussed above indicate the existence of a crucial, although unclear, relationship between language production and affective state and vice versa, especially when individuals are allowed to use their own words. Asking participants to describing affectively laden pictures might be an excellent way to bring about an affective state, while simultaneously allowing the free production of linguistic content that, in turn, might affect the extent to which the pictures induce an affective state compared to a situation where participants are merely passive observers of the pictures.

For our study, we selected pictures from a well-known and validated set of affectively laden pictures, the International Affective Picture System (IAPS; [20, 21]). The IAPS is a large set of pictures of varied content, including arousing and (un)pleasant (e.g., snakes and spiders, romantic couples, and extreme sports) as well as more neutral pictures (e.g., flowers, objects, and portraits). The pictures have been rated on valence or pleasure (negative/positive), arousal (low/high) and dominance (dominated/in control; [20]), as well as for discrete emotion categories [22].

Most studies using the IAPS pictures to induce affective state select a number of IAPS pictures within certain ranges of valence to create subsets of positive, negative and sometimes neutral pictures (e.g., [23]). Per the Velten method, we aimed to gradually induce positive and negative affect by exposing individuals to sets of IAPS pictures that start with neutral content, and gradually become more positive or negative, based on their valence ratings [20].

1.4. The current studies

In order to study the effect of language production on affect, we devised a method where individuals expressed themselves in their own (possibly affective) language as a way to induce affect. Our method is inspired by the incremental nature of the Velten method, but asks participants to describe affectively evocative pictures instead of read out loud sentences. This modification is prompted by the desire to investigate the relationship between language and affect, while simultaneously using a more natural paradigm that is less prone to demand characteristics.

To assess whether verbally describing the pictures would indeed result in an enhanced affective experience we contrast the effect of this with the effect of passively viewing pictures, which is the more conventional way of using the IAPS pictures (e.g., [23]). To our knowledge, this is the first study that investigates the effect of describing affectively charged pictures on the affective state of the speaker. Given that previous work found that describing personal affective experiences either decreases (e.g., [12]), increases (e.g., [15]), or increases and then decreases [16] the intensity of the experienced affect, this comparison could go either way. However, given that the verbalizing self-referential statements were effective in the Velten method, we hypothesize that, compared to passively viewing them, verbally describing affectively laden pictures will enhance the affective experience.

In sum, to investigate our research questions, we conduct two studies investigating whether there is an additive effect of verbalizing the content of pictures on affective state (compared to merely viewing them). For this, we used pictures taken from the IAPS, gradually increasing in affective content (positive, negative) or remaining neutral. In Study I (called “Spoken Descriptions of Pictures”), participants view and describe the pictures out loud. In Study II (called “Passively Viewing Pictures”), participants passively view the pictures, and do not describe them.

Finally, to elucidate the relationship between affective state and the language that is used in the descriptions, we will explore the content of the verbal descriptions of the pictures of Study I, comparing the frequency of affective word use in the three (affective) content categories, and word count, using LIWC [2]. We preregistered the methods, hypotheses, and analyses of this study at the Open Science Foundation: https://osf.io/kv8g3.

1.5. Hypotheses

With respect to our gradual affect induction procedures, we have the following hypotheses:

  1. Irrespective of whether they describe the pictures out loud, participants in the condition with positive pictures will report higher levels of pleasant affect. In the condition with negative pictures, they will report higher levels of unpleasant affect. No differential effect is expected for the neutral pictures.

  2. We expect that describing affective pictures will enhance the effect on affective state compared to passively viewing the pictures. Specifically, we predict that participants viewing and describing positive pictures (Study I) will report higher levels of pleasant affect than participants passively viewing positive pictures (Study II), and participants viewing and describing negative pictures (Study I) will report higher levels of unpleasant affect than participants passively viewing negative pictures (Study II). To determine if this hypothesis is true, the results from Study I and Study II will be compared.

2. Study I: Spoken Descriptions of Pictures

Study I investigated the effect of viewing and describing (out loud) pictures gradually increasing in affective content on (self-reported) affective state.

2.1. Method

2.1.1. Design

The study had a 2 (Time: pre-test, post-test) x 3 (Condition: positive, neutral, and negative) design, with time as within-subjects variable and condition as between-subjects variable.

2.1.2. Participants

In total, 122 participants were recruited at a Dutch university and participated in the experiment for course credit. One participant was excluded because they did not consent to their data being used. Our final sample included N = 121 participants (41 male; M age = 22.22, SD age = 2.90), each randomly assigned to one of the conditions (positive condition: n = 41; neutral condition: n = 40; negative condition: n = 40).

All procedures performed were in accordance with the ethical standards of the institutional research committee, the Research Ethics and Data Management Committee of Tilburg School of Humanities and Digital Sciences, Tilburg University. All participants gave written informed consent in accordance with the Declaration of Helsinki (1964) and its later amendments or comparable ethical standards.

2.1.3. Materials

Stimuli. We used the 2008 variant of the IAPS, containing 1194 pictures [24]. In order to create three conditions, we selected 40 IAPS pictures per condition based on the procedure described below.

First, two sets of 600 positive and negative pictures each were created. For the positive condition, we started from the 600 pictures with the highest valence rating (range 5.22–8.34). For the negative condition, we started from the 600 pictures with the lowest valence ratings (range 1.31–5.24). Indeed, the ranges of positive and negative pictures partly overlap. This mirrors the Velten method, which starts with the same sentence in both the positive and negative set of statements.

Next, both sets were divided into 40 bins of fifteen pictures, with each bin increasing in pleasant (positive condition) or unpleasant (negative condition) content. From each bin, one picture was randomly selected, resulting in two sequences of 40 pictures, which gradually increased in (un)pleasant content.

To create the picture set for the neutral condition, we selected 301 pictures with an average valence rating (range 4.62–5.92). Forty random bins of fifteen pictures were created, and from each bin one picture was randomly selected.

While selection of pictures from bins was random in principle, sometimes a picture was deemed inappropriate and replaced by another randomly selected picture from the same bin. Exclusion criteria were: erotic or sexually suggestive (but not non-erotic nudity), too gruesome or disgust-inducing, repetitive content, or culturally sensitive content (e.g., traditions and rituals). Based on these criteria, we excluded six pictures and replaced them with more appropriate pictures from the same bin (S1 Table). Our final sample can be found in S2 Table.

Finally, to check if the three sets of pictures did not contain any outliers that would disturb the gradual increase (positive and negative condition), or would interfere with a consistent level (neutral condition) of affective content of the pictures, the sets of pictures were inspected for their valence and arousal in two line plots (S1 Fig and S2 Fig). As shown in S1 Fig, both the positive set, and the negative set, displayed a near perfect gradual increase in affective content in the expected directions. For the neutral set, a (very) slight decrease in pleasant content can be observed. As can be found in S2 Fig, compared to the valence ratings in S1 Fig, the arousal ratings were less distinct in their sequential direction, with a strong increase in arousal for negative pictures, and no substantial in- or decrease for both the positive pictures, as the neutral pictures.

For the three sets of pictures, the range, mean (with standard deviation) and median of the valence and arousal scores can be found in Table 1.

Table 1. Statistical characteristics of the final sample of pictures.
Valence Arousal
Pictures Range M (SD) Median Range M (SD) Median
Positive 5.22–8.05 6.52 (0.77) 6.52 2.63–7.31 4.33 (1.04) 4.07
Neutral 4.95–5.22 5.08 (0.82) 5.07 2.00–6.23 3.52 (0.93) 3.22
Negative 1.51–5.22 3.53 (1.13) 3.57 1.72–7.07 4.73 (1.55) 4.97

Viewing IAPS pictures. In Study I, participants were given instructions to describe each picture out loud, inspired by the MS COCO instructions [25], which is a well-established method of eliciting picture descriptions. In our study, participants were instructed to describe all the important aspects and details of the pictures, describe them in a way that another person could recognize this picture out of the set of 40 pictures, and use full sentences when describing the pictures.

After piloting with various timeframes (6, 8, and 10 seconds), 10 seconds viewing time per picture appeared to be sufficient to describe the pictures. Each participant started with two practice trials describing two neutral pictures. In order to encourage participants to actively engage in the task, we presented them with a bogus purpose of the study: memorizing the pictures. The study was introduced as a memory experiment, and participants were told that they would be asked to indicate pictures they had, and had not, seen before from a set of new and old (already seen) pictures.

Video- and audio recording. Audio was recorded for content analysis of the picture descriptions. In addition, we video recorded facial expressions for possible future analysis.

Affect questionnaire. Before and after viewing the series of pictures, participants indicated their current affective state on six 7-point Likert scales: sad/happy, unpleasant/pleasant, unsatisfied/satisfied, discontent/content, sullen/cheerful, low-spirited/in high spirits ([26], based on [27]; [28]; English translations of Dutch originals). They were instructed to choose a number per scale; the closer the numbers were to the words, the stronger they match the feeling described the word in question. Low numbers indicated the degree of negative affect (e.g., unpleasant), high numbers indicated the degree of positive affect (e.g., pleasant). In a previous study by Krahmer et al. [26], the internal consistency of this questionnaire was good, α = .88. We assessed the reliability of the current scale with Cronbach’s α as well. This analysis indicated that the items of our affect questionnaire had excellent internal consistency, for both Study I (pre-test, α = .90; post-test, α = .94) and Study II (pre-test, α = .94; post-test, α = .95). Based on these results, the six items were merged into one scale, ‘Affect’, resulting in one pre-test and one post-test score per participant, indicating (self-reported) affective state, ranging from 1 (negative affect) to 7 (positive affect). Based on these results, the six items were merged into one scale, ‘Affect’, resulting in one pre-test and one post-test score per participant, indicating (self-reported) affective state, ranging from 1 (unpleasant) to 7 (pleasant).

Procedure. After participants signed the informed consent form, the experiment leader explained the procedure and turned on the camera, including audio recording. If needed, the camera was adjusted to an appropriate height to record the participant’s face. Participants reported their gender and age. They then filled out the affect questionnaire for the first time (pre-test). Then, starting with two practice trials, participants were asked to view and describe 40 pictures out loud. After the task, they filled out the affect questionnaire again (post-test). Then, participants were asked to indicate which pictures they had seen before, and which ones they had not. Pictures were selected beforehand, by randomly picking three numbers between 1 and 40, using the corresponding bin to select one ‘old’, and one ‘new’ picture. Ninety percent (n = 220) of all participants (N = 245) correctly identified all six pictures as ‘old’ or ‘new’.

After the experiment, participants in the negative condition viewed a light-hearted, short video displaying a jumping competition for bunnies [29]. This video was shown to rise their spirits, in case participants felt especially low after the experiment. Participants in the positive and neutral condition did not watch the video. At the end, the participants were debriefed and thanked for their participation.

2.2. Results

2.2.1. Descriptive statistics

Fig 1 displays the individual scores on affect (y-axis) for the pre- and post-test (x-axis), sorted by condition (positive, neutral, and negative pictures). On the y-axis, lower scores indicate the degree of unpleasant affect; higher scores indicate the degree of pleasant affect. In Fig 1, the results for the three conditions show a clear pattern. Participants viewing negative pictures generally report feeling unpleasant after describing the pictures. Participants viewing positive pictures generally report feeling slightly more pleasant describing the pictures, and participants viewing the neutral pictures did not seem to report a change in affective state. In general, participants in all conditions seem to start the experiment in fairly good spirits (possibly partly explaining the limited effect in the positive condition), scoring roughly 5 to 5.5 on the 7-point Likert scale.

Fig 1. Individual affect scores for participants viewing and describing pictures.

Fig 1

The dots of the bold lines represent the mean scores.

2.2.2. Change in affective state

We performed a repeated measures analysis of variance with time (pre-test and post-test) as within-subjects factor, condition (positive, neutral, or negative pictures) as between-subjects factor and affective state as dependent variable. Mean scores, standard deviations, difference scores (posttest-pretest) and range can be found in Table 2. A main effect was found for time, F (1, 118) = 28.03, p < .001, ηp2 = .19, and for condition, F (2, 118) = 8.05, p = .001, ηp2 = .12. However, these two main effects were qualified by a (predicted) interaction effect for time and condition, F (2, 118) = 37.23, p < .001, ηp2 = .39. Post-hoc tests revealed that participants in the negative condition reported lower levels of pleasantness after viewing and describing the negative pictures. Participants in the positive or neutral condition did not report a significant change in affective state after viewing and describing the pictures.

Table 2. Affective state scores of participants in Study I and Study II.

Mean scores (with standard deviations), difference scores and range are displayed.

Condition Time Spoken Descriptions of Pictures Passively Viewing Pictures
M (SD) M (SD)
Positive Pre-test 5.20 (0.88) 4.94 (1.17)
Post-test 5.44 (0.83) 5.06 (1.08)
Difference 0.24 (0.62) 0.12 (0.86)
Range -1.50, 2.67 -3.17, 2.17
Neutral Pre-test 5.28 (0.89) 5.13 (0.92)
Post-test 5.13 (0.89) 4.92 (0.97)
Difference -0.15 (0.54) -0.21 (0.36)
Range -1.33, 0.67 -1.17, 0.50
Negative Pre-test 5.25 (0.91) 4.88 (1.03)
Post-test 3.97 (1.22) 3.58 (1.12)
Difference -1.28 (1.17) -1.30 (1.04)
Range -4.00, 0.67 -3.83, 0.83

2.3. Conclusion

As predicted, participants viewing and describing negative pictures reported to experience lower levels of pleasantness, and participants viewing and describing neutral pictures did not report a change in affective state after completing the task. In contrast to our prediction (but in line with other, earlier studies reporting unsuccessful positive affect induction, as discussed below), participants viewing and describing positive pictures did not report (significantly) higher levels of positive affect after the task.

3. Study II: Passively Viewing Pictures

Study II studied the effect of passively viewing, but not describing out loud, pictures gradually increasing in affective content on (self-reported) affective state. We used the same sets of pictures as in Study I.

3.1. Method

3.1.1. Design

The design was identical to Study I.

3.1.2. Participants

Participants were recruited at the same Dutch university as in Study I. A total of 126 participants participated in the experiment for course credit; none of them participated in Study I. Two participants did not consent to have their data published in scientific journals; therefore, we excluded their data. Our final sample included N = 124 participants (43 male; M age = 23.50, SD age = 4.00), again, each assigned to one of the conditions (positive condition, n = 41, neutral condition, n = 41; negative condition, n = 42). Again, all procedures were in accordance with the ethical standards of the local research committee. Written informed consent was obtained from all individual participants included in the study.

3.1.3. Materials

Stimuli. The materials we used were identical to those used in Study I, but in contrast to Study I, participants could do the experiment in Dutch or English, because they did not verbally describe the pictures, the language they spoke became irrelevant. Participants received informed consent, instructions and debriefing, and fill out the questionnaires, in their language of choice.

Viewing IAPS pictures. The viewing time per picture was identical to Study I.

Procedure. The procedure was identical to that of Study I, except that participants only passively viewed the pictures, instead of viewing them and describing them out loud. For this reason, no audio recording took place.

3.2. Results

3.2.1. Descriptive statistics

As in Fig 1, Fig 2 displays the individual scores of affective state (y-axis) for the pre- and post-test (x-axis), sorted by condition (positive, neutral, and negative pictures). On the y-axis, lower scores indicate the degree of unpleasant affect; higher scores indicate the degrees pleasant affect.

Fig 2. Individual affect scores for participants Passively Viewing Pictures.

Fig 2

The dots of the bold lines represent the mean scores.

Notice that Fig 2 looks very similar to Fig 1, showing the same pattern as described above: participants viewing negative pictures generally reported feeling unpleasant, and participants viewing positive or neutral pictures generally did not report a substantial change in affective state. Akin to the participants in Study I, participants in Study II generally started the experiment in fairly good spirits, scoring roughly 5 to 5.5 on the 7-point Likert scale.

3.2.2. Change in affective state

A repeated measures analysis was performed with time (pre-test and post-test) as within-subjects factor, condition (positive, neutral or negative pictures) as between-subjects factor and affective state as dependent variable. Mean scores, standard deviations, difference scores and range can be found in Table 2. As in Study I, a main effect was found for time, F (1, 121) = 40.22, p < .001, ηp2 = .25, and condition, F (2, 121) = 8.99, p < .001, ηp2 = .13. However, again, these two main effects were qualified by a (predicted) interaction effect for time and condition, F (2, 121) = 35.16, p < .001, ηp2 = .37. Identical to Study I, post-hoc tests revealed that participants in the negative condition reported lower levels of pleasant state after viewing the negative pictures. Again, participants in the positive or neutral condition did not report a significant difference in affective state after viewing the pictures.

3.3. Conclusion

Similar results to Study I were found: participants viewing negative pictures reported negative affect after viewing the pictures, and participants viewing positive or neutral pictures did not report a significant change in affective state.

4. Comparing Spoken Descriptions of Pictures and Passively Viewing Pictures

To determine if viewing and describing the positive and negative pictures out loud (Study I), compared to passively viewing them (Study II), evoked higher levels of (positive or negative, respectively) affect, the ratings from Study I and Study II were compared using an ANOVA.

4.1. Individual changes in affective state

To explore our dataset, we looked at the changes in affective state for all individual participants (both Study I and Study II). As can be inferred from both Fig 1 and Fig 2, there is a substantial amount of variation in the effectiveness of the manipulation, with only the negative condition showing a consistent pattern for the majority of the participants.

Generally speaking, participants viewing positive pictures (n = 82) reported feeling more pleasant after the task (n = 47), albeit the change was modest (≤ 1 on a 7 point scale) for the majority of participants (n = 40). Regarding participants viewing neutral pictures (n = 81), the majority (n = 75) reported a small change in affective state, feeling more positive (≤ 1) or negative (≤ -1). Participants viewing negative pictures (n = 82) showed the same pattern, but contrary to participants exposed to the neutral pictures, the variation between participants was much larger: 74 participants reported more unpleasant affect, of which 50 individuals reported a decrease of ≥ -1 on the affect scale.

There were no large (individual) differences between the individuals describing the pictures out loud, or only passively viewing them.

4.2. Results

To test the hypothesis that verbally describing affective pictures, compared to only viewing them, enhances the effect on affective state, a mixed ANOVA was performed with time (pre-test and post-test) as within-subjects factor, type of study (Study I: Spoken Descriptions of Pictures, or Study II: Passively Viewing Pictures) and condition (positive, neutral or negative pictures) as between-subjects factors, and affective state as dependent variable.

A main effect was found for type of study, F (1, 239) = 6.39, p = .012, ηp2 = .03 (Study I: M = 5.04, SD = .91; Study II: M = 4.75, SD = 0.91), indicating that affective state was overall more positive for participants in Study I, compared to Study II. A main effect was also found for time, F (1, 239) = 67.50, p < .001, ηp2 = .22 (pre-test: M = 5.11, SD = .97; post-test: M = 4.68, SD = 1.22), indicating that participants experienced more negative affect after engaging in the task (reflecting the effective manipulation in the negative condition). Finally, a main effect was found for condition, F (2, 239) = 16.82, p < .001, ηp2 = .12 (positive: M = 5.16, SD = .94; neutral: M = 5.11, SD = .91; negative: M = 4.42, SD = .94), indicating that affect was lower overall in the negative condition, both for participants that described the pictures and for those that did not. However, no three-way interaction of time, type of study, and condition was found, F (2, 239) = .09, p = .915, indicating that describing or only viewing affective pictures did not influence affective state significantly for one or more of the conditions.

Given that there was no significant change in affective state for participants after viewing positive or neutral pictures, we wanted to rule out the possibility that these null effects obscured a possible difference for the negative condition. Selecting only the negative condition, a repeated measures ANOVA was performed with time (pre-test and post-test) as within-subjects factor, type of study (Study I or Study II) as between-subjects factor and affective state as dependent variable. As predicted, a main effect was found for time, F (1, 80) = 111.95, p < .001, ηp2 = .583, with participants becoming more negative during the experiment. However, we found no effect for type of study, F (1, 80) = 3.49, p = .065, and, importantly, no interaction between time and study, F (1, 80) = 0.001, p = .98. The results of this secondary analysis again indicate that individuals experienced worse affective state after exposure to the pictures, regardless of whether they described the pictures out loud or not.

5. An exploratory content analysis of the picture descriptions

In order to investigate the language use of the participants, and get more insight in how individuals describe (affective) content, we explored the verbal picture descriptions of Study I, using the word counting software LIWC [2]. LIWC is a text analysis software program for counting words and calculating percentages of words, grouping them in various categories, including cognitive- and affective processes. For our current analysis, we used the Dutch LIWC dictionary [30] to keep track of the words in the LIWC-categories 'affective processes' (to which we will refer to as “affective words”, e.g., dirty, help), 'positive emotion' words (e.g., beautiful, hug), 'negative emotion' words (e.g., sad, cry), as well as the word count per picture description.

Verbalizations of descriptions were transcribed by five individuals outside the project. The utterances of N = 122 participants were transcribed, resulting in 40 x 122 = 4880 descriptions. Forty-three descriptions, less than 1% of the dataset, were missing: all descriptions from one participant (in the neutral condition), two descriptions from one participant, and one description from one participant. One participant was excluded because she did not consent to her data being used. Our final sample included 4797 picture descriptions by n = 120 speakers, with a mean word count of 18.89 (SD = 6.73) words per description.

5.1. Descriptives

Table 3 provides the mean percentages (with standard deviations) of total words used per picture description, in the corresponding LIWC categories, per condition. Fig 3A–3D depict the average scores per item in the respective LIWC category, represented by dots (the average score per item) and trend lines, including bands, representing Confidence Intervals. As can be seen in Fig 3A–3C, most individuals tend to use no (0) or few (1, 2) affective, positive, or negative emotion words to describe a picture. Participants gradually used more positive emotion words to describe positive pictures and negative pictures, but not neutral pictures (Fig 3A). The same pattern was found for negative emotion words, although the increase was less steep (Fig 3B). Affective word use gradually increased to describe positive and negative pictures, but not neutral pictures (Fig 3C). In all conditions, the data suggest that the number of words increases with subsequent pictures, a trend which is most clear for negative pictures (Fig 3D). However, we should be cautious interpreting this pattern since there is also substantial variation between participants.

Table 3. Mean scores, standard deviations and confidence intervals per condition for word count, and percentages of affective words, positive emotion words, and negative emotion words.

Condition Descriptions n Affective words M (SD) CI Pos. words M (SD) CI Neg. words M (SD) CI Word count M (SD) CI
Positive 1638 1.54 (3.21) 1.39–1.70 1.26 (2.85) 1.13–1.40 0.13 (0.93) 0.09–0.18 20.16 (6.52) 19.84–20.47
Neutral 1560 0.75 (2.42) 0.63–0.87 0.45 (1.98) 0.35–0.55 0.27 (1.27) 0.21–0.33 17.58 (6.90) 17.24–17.92
Negative 1599 1.69 (3.43) 1.52–1.86 0.53 (1.70) 0.45–0.61 1.03 (2.76) 0.90–1.17 18.88 (6.53) 18.56–19.20
Total 4797 1.33 (3.08) 1.25–1.42 0.75 (2.27) 0.69–0.82 0.48 (1.87) 0.43–0.53 18.89 (6.73) 18.70–19.09

Fig 3. Trend lines for average scores per item (represented by dots), per condition for positive emotion words, negative emotion words, affective words, and word count.

Fig 3

Bands represent confidence intervals.

5.2. Results

To statistically analyze word count and affective words used in the picture descriptions, four separate one-way ANOVAs were performed, with condition as independent factor, and word count, (percentage of) affective-, positive- and negative word use, as dependent variables. Data were aggregated on individual level, combining individual scores on each picture description to one mean score for each LIWC category. We tested for homogeneity of variances using Levene’s tests. Results of ANOVAs and Levene’s tests can be found in Table 4. Levene’s test results indicated that equal variances were assumed for word count, but not for affective-, positive-, and negative word use. Differences between the conditions were assessed with Tukey’s (equal variances assumed) and Games Howell (equal variances not assumed) post hoc comparisons.

Table 4. ANOVAs and Levene’s tests for condition on affective-, positive-, and negative word use, and word count.

One-way ANOVA Levene’s test
Affective word use F (2, 117) = 20.78, p < .001 F (2, 117) = 9.19, p < .001
Positive word use F (2, 117) = 25.57, p < .001 F (2, 117) = 10.31, p < .001
Negative word use F (2, 117) = 134.87, p < .001 F (2,117) = 6.77, p = .002
Word count F (2, 117) = 3.30, p = .040 F (2, 117) = 0.22, p = .801

5.2.1. Affective words

Although our positive affect induction was not successful, participants describing positive pictures generally used more affective words in their descriptions, compared to participants describing neutral pictures, p < .001. Negative pictures were described with more affective words than neutral pictures, p < .001. No difference was observed for affective word use between positive and negative pictures, p = .670.

5.2.2. Positive emotion words

Positive emotion words were used significantly more when describing positive pictures, compared to negative pictures, p < .001, and neutral pictures, p < .001. There was no significant difference between negative and neutral pictures, p = .483.

5.2.3. Negative emotion words

A similar pattern was observed for negative word use: participants describing negative pictures used significantly more negative emotion words, compared to positive pictures, p < .001, and neutral pictures, p < .001. Additionally, neutral pictures were described with more negative emotion words than positive pictures, p = .012.

5.2.4. Word count

Individuals used more words describing positive pictures compared to neutral pictures, p = .031, but not compared to negative pictures, p = .409. No significant difference was found between neutral and negative pictures, p = .405

5.3. Conclusion

The results of this exploratory content analysis are in line with what would be intuitively expected. Individuals viewing affective pictures used more affective words in their descriptions, compared to when they are describing neutral pictures. Positive pictures, compared to negative and neutral pictures, were described with more positive emotion words, and conversely, negative pictures were described with more negative emotion words than neutral pictures, and neutral pictures were described with more negative emotion words than positive pictures. Speakers used more words to describe positive pictures than neutral pictures.

6. General discussion

In this study, we aimed to study the effect of free verbal expression on affect induction, by investigating the effectiveness of affect induction methods, inspired by Velten [1], where pre-defined self-referential statements are replaced with IAPS pictures, gradually increasing in affective content (positive, negative) or remaining neutral. In Study I, ‘Spoken Descriptions of Pictures’, individuals verbalized the content of the pictures out loud. In Study II, ‘Passively Viewing Pictures’, participants passively viewed the pictures, and did not describe them out loud. Our first hypothesis was partly confirmed: as predicted, for both studies, negative affect induction was effective, and the neutral condition did not evoke a change in affective state. However, in both studies, positive affect induction did not result in a significant enhancement of positive affective state when compared to the neutral condition. Our second hypothesis was not confirmed: describing the pictures out loud did not enhance, nor did it temper, affective state.

Additionally, the linguistic content of the verbal descriptions of the IAPS pictures was explored with LIWC [2]. For positive and negative pictures, we observed a gradual increase in affective word use over time. Specifically, positive pictures were described with more positive emotion words, and negative pictures were described with more negative emotion words. No effects were observed for the neutral pictures. A large variation between pictures was observed for the number of words speakers used to describe the pictures. In general, speakers used more words to describe positive pictures than neutral pictures, but not negative pictures.

6.1. Inducing positive and negative affect

Verbally describing and passively viewing affective pictures successfully induced negative affective states (in the negative condition), but not positive affective states (in the positive condition). Our findings did not support the hypothesis that verbally describing affective pictures would induce stronger affective states than passively viewing them. The finding that the positive affect induction turned out to be less successful than the negative affect induction is found more often (e.g.,[21, 31, 32]), and other studies using IAPS pictures (e.g., [21]) and the Velten method (e.g. [31, 3335]) have faced this problem as well. Given the fairly positive affective state of the participants before being exposed to the pictures, a possible explanation for this lack of an effect might be that the participants’ positive affect was already at ceiling [21, 31].

6.2. Affect and language

Participants reported slightly more pleasant affect after describing the positive pictures, compared to passively viewing them. However, this difference was small and not significant. As described above, the literature shows mixed results regarding the effect of affect labelling on affective (and emotional) experience. Putting emotions into words can enhance the affective experience (e.g., [15]) or decrease it (e.g., [12]). However, we realize that our affect induction procedure is somewhat unique, and might not be directly comparable to affect labelling. First of all, in experiments studying affect labelling, participants are asked to describe their own affective state. In our Study I, individuals were asked to describe the (affective) content of affective pictures, not their own emotions. Second, our participants were asked to view and describe the pictures simultaneously, making it harder to distinguish between the effect of viewing the pictures and verbalizing the content. But given that the effect of verbalizing was small, we doubt whether it would have made a significant difference to first expose individuals to the pictures to our participants, and asking them to describe them only after viewing.

To date, only a limited amount of work has been done on the description of affective content where speakers could use their own words [36, 15]. Study I adds to this relatively new field that combines questions from affective science and psycholinguistics.

6.3. Verbal descriptions

Corresponding to the gradual increase in affective content of the positive and negative pictures, we observed a gradual increase in affective language use in the expected directions: over time, positive pictures were described with more affective and positive emotion words; negative pictures were described with more affective and negative emotion words. The descriptions of neutral pictures were described with few affective, positive- and negative emotion words. Interestingly, individuals viewing and describing positive pictures did not (self)report enhanced positive affect, but they did use substantially more positive emotion words in their descriptions, compared to the negative and neutral pictures.

The usage of emotional words could be attributed to and explained by the specific (affective) content of the pictures. Therefore, we also studied a phenomenon that could not be attributed to the emotional content of pictures—the general number of words uttered to describe the pictures. Our results indicated that individuals did not use more words to describe positive than negative pictures. This is in contrast to the literature: happy individuals tend to talk faster (e.g., [37], but see also [38]) and sad individuals tend to talk slower (e.g., [39]). Additionally, happy individuals have been found to utter more words spontaneously [1]. However, keeping in mind that positive affect induction was not successful, this finding is not unexpected. Another explanation might be that the content of the affective pictures was more complex, compared to the neutral pictures. Indeed, many neutral pictures included depictions of objects, patterns and portraits, whereas the affective pictures often composed scenes of multiple components, e.g., individuals in various situations (e.g., plane crash, cycling), diverse backgrounds (e.g., nature, city, living room).

However, individuals used more words to describe positive pictures than neutral pictures. Assuming that affective pictures, both positive and negative, are more arousing than neutral pictures, this might explain why individuals used more words to describe affective than neutral pictures, because highly aroused speakers compared to lowly aroused speakers tend to have an increased speech rate [40] and thus might use more words. We found that the IAPS arousal scores were indeed positively correlated to word use, both for positive (r = .07) and negative (r = .16) pictures. However, we also found correlations between the IAPS valence scores and word use, which were systematically larger than those between arousal and word use, for both the positive (r = .12) as well as negative pictures (r = -.21). Concluding, both affective and arousing content was correlated to the number of words used to describe the pictures.

6.4. Strengths and limitations

Our study has a few limitations that need to be acknowledged. First, while we deliberately chose to use an incremental procedure, the incremental nature of the affect induction procedure might pose various issues. Given that order effects are at the base of our study, participants might be influenced to a lesser extent by the pictures, because they were exposed to pictures gradually increasing in affective content, instead of viewing a random selection of affective pictures (e.g., [23, 41, 42]) that might be, on average, more positive or negative. The temporal place of a stimulus in an array of pictures can influence how the stimulus is processed in the viewer, e.g. habituation effects (e.g., [43]) might reduce the effectivity of the stimuli, while recency bias (e.g., [44]) might enhance the effect of the last (few) affective pictures. However, showing affective IAPS pictures in a fixed (or non-incremental) order is not uncommon, and has been shown to effectively induce desired emotions and affective states (e.g., [15, 45, 46]). Therefore, while we acknowledge this limitation, we do not think the incremental nature of the stimuli is responsible for the absence of an effect of, for example, the positive condition. Nevertheless, there is certainly a possibility that the last few pictures were the most effective at inducing affect, and the previous pictures’ affective impact was limited. For future research, it might be interesting to compare the effectivity of exposure to highly positive or negative rated IAPS pictures, compared to exposure to pictures gradually increasing or decreasing in valence.

Based on the available evidence in the literature, predicting the precise effectivity of the incremental procedure was difficult. Hence, we were ambivalent in our predictions: the gradual increase in valence could result in a weaker effect, a stronger effect, of perhaps even no effect at all. For example, Van der Zwaag et al. [47] compared the effectivity of gradual versus abrupt change in happy music to sad music. They found that both emotion induction procedures were equally effective, lowering both valence and energy (i.e. feeling more tired, according to self-report of the participants).

Gradually increasing affective content of stimuli might have several advantages. First, as Velten argued, the gradual emotion induction was favorable, ‘to overcome the subjects’ presumable reluctance to experience unpleasant mood’ [4] (p. 68). Indeed, recent research shows that noncompliance with an affect induction procedure is more common viewing negative videos than positive videos [48]. Given that we started the series of negative pictures with neutral stimuli, this might prevent the initial reluctance of participants to engage in the negative affect induction procedure. Additionally, for some populations, the startle effect might be specifically unethical, because they could cause serious psychological or physiological harm, for example, to individuals with certain mental disorders (e.g., PTSD, panic disorder) or cardiovascular diseases.

Second, verbalizing the content of the pictures adds additional challenges–for example, participants likely vary in their degree of verbal skills and consequently differ in how difficult they considered the task. We did try to take this into account in the selection of our participants by excluding participants with a speech disorder or a limitation in the ability to speak fluently (e.g., stuttering). However, to check whether having Dutch as a first language had an effect on the effectivity of the affect induction of verbally describing the pictures, we repeated our analysis of Study I, excluding the participants who did not have Dutch as their first language (n = 5), but did not find substantial differences. Additionally, given that we tested a relatively homogenous group of participants (young Dutch students), we expect that individual differences in verbal fluency, attention, and other cognitive and communicative abilities are small and randomly distributed throughout our sample.

Third, an additional benefit of our study is the collection of human, realistic verbal descriptions for the content of the subset of IAPS pictures we used. While these descriptions are not yet validated, it is a valuable first step to the possible creation of a verbal IAPS, which might be useful in certain specific populations, e.g., visually impaired individuals.

Lastly, we conjectured that asking individuals to use their own words describing the pictures (instead of uttering pre-defined affective sentences), would reduce the awareness of the goal of the procedure (affect induction) and therefore reduce the chance of participants reporting to feel the change in affect they think they ‘should’ experience (e.g., social desirability or task demands), even when they do not actually experience a shift in affective states (e.g., [49]).

6.5. Future research

Affective processes often take place in a social setting, but in laboratory settings, they are generally induced in individual participants [50]. Our affect induction method might be a useful, naturalistic method to induce affective states in more than one individual at the same time. For example, participants could take turns in a dialogue setting describing out loud the affective IAPS pictures to each other. This might create more naturalistic opportunities in affective research to study affect induction in dyads.

Contrary to the Velten method, our method describing pictures was not self-referential in nature. Recent literature suggests that self-referencing might play a critical role in affective word processing. Soares et al. [51] found that in a masked priming paradigm, individuals categorize positive adjectives faster when they are primed by self-related primes, compared to other-primes. In light of these findings, the Velten method might be more effective inducing positive affect than our affect induction method. Upon inspecting the verbalizations, indeed, only 26% percent of the descriptions are self-referential (e.g., ‘I see…’), and less than 1% is other-referential (‘Here you see…’; see S3 Table). This might be one of the reasons that our pleasant affect induction was not effective. For future research, it might be interesting to compare a condition where participants are instructed to provide a self-referential description (‘I see a happy couple’) to a condition where participants are instructed to provide a non-self-referential description of the pictures (‘This is a picture of a happy couple’).

Finally, our affect induction method is not inherently limited to valence, but also could be applied to specific emotion categories that are present in picture datasets (cf., [22], for IAPS). By replacing the current pictures with pictures that induce a specific emotion (e.g., disgust, tenderness and anger), we think our method might be able to successfully induce specific emotions and their accompanying verbal descriptions.

6.6. Implications

This study contributes to the sparse literature on verbalizing affective content, implying that an engaging task as verbalizing negative content, using free expression, can be an effective method to induce negative affect in a possibly more ecologically sound manner (e.g., viewing affective videos). The results indicated that verbalizing or passively viewing affective content are equally effective methods to induce negative affective state.

We contributed to the scientific literature on the relationship between affect and language, aiming to gain understanding of the critical, but unclear relationship between language production and (un)pleasant affect.

Supporting information

S1 Table. Positive and negative IAPS pictures that were initially selected but excluded based on our exclusion criteria.

(DOCX)

S2 Table. Final selection of positive (increasing in valence), negative (decreasing in valence), and neutral IAPS pictures.

(DOCX)

S3 Table. Self-referential and other-referencing in the pictures descriptions, sorted by condition, counted by LIWC.

(DOCX)

S1 Fig. Valence (y-axis) of IAPS pictures by bin (x-axis).

(TIF)

S2 Fig. Arousal (y-axis) of IAPS pictures by bin (x-axis).

(TIF)

Data Availability

The datasets and syntax can be found in the folder ''Gradual emotion induction with a visual Velten method'' --> ''Datasets and syntax'' at the Open Science Foundation: https://osf.io/7zgct/

Funding Statement

This study was funded by the Netherlands Organization for Scientific Research [360-89-050]. The award was granted to EK (principal investigator) and Mariët Theune and MG (co-applicants). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. More information on the project can be found at https://www.nwo.nl/onderzoek-en-resultaten/onderzoeksprojecten/i/45/13545.html.

References

  • 1.Velten Jr E. A laboratory task for induction of mood states. Behaviour research and therapy. 1968;6: 473–82. [DOI] [PubMed] [Google Scholar]
  • 2.Pennebaker JW, Francis ME, Booth RJ. Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates; 2001;71: 2001. [Google Scholar]
  • 3.Marcusson-Clavertz D, Kjell ON, Persson SD, Cardeña E. Online validation of combined mood induction procedures. PloS one. 2019. June 4;14(6):e0217848. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Velten Jr EC. The induction of elation and depression through the reading of structured sets of mood-statements. Doctoral dissertation, University of Southern California; 1967. Available from: 10.1016/0005-7967(68)90028-4 [DOI] [Google Scholar]
  • 5.Jennings PD, McGinnis D, Lovejoy S, Stirling J. Valence and arousal ratings for Velten mood induction statements. Motivation and Emotion. 2000;24: 285–97. [Google Scholar]
  • 6.Scherer KR. The nature and dynamics of relevance and valence appraisals: Theoretical advances and recent evidence. Emotion Review. 2013;5: 150–62. [Google Scholar]
  • 7.Wilting J, Krahmer E, Swerts M. Real vs. acted emotional speech. Ninth International Conference on Spoken Language Processing 2006. [Google Scholar]
  • 8.Lindquist KA. The role of language in emotion: existing evidence and future directions. Current opinion in psychology. 2017;17: 135–9. [DOI] [PubMed] [Google Scholar]
  • 9.Barrett LF. How emotions are made: The secret life of the brain. Houghton Mifflin Harcourt. 2017. [Google Scholar]
  • 10.Fan R, Varol O, Varamesh A, Barron A, van de Leemput IA, Scheffer M, et al. The minute-scale dynamics of online emotions reveal the effects of affect labeling. Nature Human Behaviour. 2019;3: 92. [DOI] [PubMed] [Google Scholar]
  • 11.Lieberman MD, Eisenberger NI, Crockett MJ, Tom SM, Pfeifer JH, Way BM. Affect labeling disrupts amygdala activity in response to affective stimuli. Psychological Science. 2007;18: 421–8. [DOI] [PubMed] [Google Scholar]
  • 12.Torre JB, Lieberman MD. Putting feelings into words: Affect labeling as implicit emotion regulation. Emotion Review. 2018;10: 116–24. [Google Scholar]
  • 13.Burklund LJ, Creswell JD, Irwin M, Lieberman M. The common and distinct neural bases of affect labeling and reappraisal in healthy adults. Frontiers in Psychology. 2014;5: 221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Hutto C, Gilbert E. VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text. In: International AAAI Conference on Weblogs and Social Media. AAAI; 2014. p. 216–225.
  • 15.Ortner CN. Divergent effects of reappraisal and labeling internal affective feelings on subjective emotional experience. Motivation and Emotion. 2015;39: 563–70. [Google Scholar]
  • 16.Baikie KA, Wilhelm K. Emotional and physical health benefits of expressive writing. Advances in psychiatric treatment. 2005;11: 338–46. [Google Scholar]
  • 17.Soliday E, Garofalo JP, Rogers D. Expressive writing intervention for adolescents' somatic symptoms and mood. Journal of Clinical Child and Adolescent Psychology. 2004;33: 792–801. [DOI] [PubMed] [Google Scholar]
  • 18.Pennebaker JW. Writing about emotional experiences as a therapeutic process. Psychological science. 1997;8: 162–6. [Google Scholar]
  • 19.Pennebaker JW, Beall SK. Confronting a traumatic event: toward an understanding of inhibition and disease. Journal of abnormal psychology. 1986;95: 274. [DOI] [PubMed] [Google Scholar]
  • 20.Lang PJ, Bradley MM, Cuthbert BN. International affective picture system (IAPS): Technical manual and affective ratings. NIMH Center for the Study of Emotion and Attention. 1997;1: 39–58. [Google Scholar]
  • 21.Uhrig MK, Trautmann N, Baumgärtner U, Treede RD, Henrich F, Hiller W, et al. Emotion elicitation: A comparison of pictures and films. Frontiers in psychology. 2016;7: 180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Mikels JA, Fredrickson BL, Larkin GR, Lindberg CM, Maglio SJ, Reuter-Lorenz PA. Emotional category data on images from the International Affective Picture System. Behavior research methods. 2005;37: 626–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Capecelatro MR, Sacchet MD, Hitchcock PF, Miller SM, Britton WB. Major depression duration reduces appetitive word use: An elaborated verbal recall of emotional photographs. Journal of psychiatric research. 2013;47: 809–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Lang PJ, Bradley MM, Cuthbert BN. International affective picture system (IAPS): affective ratings of pictures and instruction manual. University of Florida, Gainesville: Tech Rep A-8; 2008. [Google Scholar]
  • 25.Chen X, Fang H, Lin TY, Vedantam R, Gupta S, Dollár P, et al. Microsoft COCO captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325. 2015. [Google Scholar]
  • 26.Krahmer E, Van Dorst J, Ummelen N. Mood, persuasion and information presentation. Information Design Journal. 2004;12: 19–32. [Google Scholar]
  • 27.Mackie DM, Worth LT. Processing deficits and the mediation of positive affect in persuasion. Journal of personality and social psychology. 1989;57: 27. [DOI] [PubMed] [Google Scholar]
  • 28.Bohner G, Crow K, Erb HP, Schwarz N. Affect and persuasion: Mood effects on the processing of message content and context cues and on subsequent behaviour. European Journal of Social Psychology. 1992;22: 511–30. [Google Scholar]
  • 29.Vice. Cute bunny jumping competition! [Video]; 2012. Available from: https://www.youtube.com/watch?v=qM9YWm6T_hc
  • 30.Zijlstra H, Van Middendorp H, Van Meerveld T, Geenen R. Validiteit van de Nederlandse versie van de Linguistic Inquiry and Word Count (LIWC). Nederlands Tijdschrift voor de Psychologie en haar Grensgebieden. 2005. June 1;60(3):50–8. [Google Scholar]
  • 31.Westermann R, Spies K, Stahl G, Hesse FW. Relative effectiveness and validity of mood induction procedures: A meta‐analysis. European Journal of social psychology. 1996;26: 557–80. [Google Scholar]
  • 32.Ferrer RA, Grenen EG, Taber JM. Effectiveness of Internet-Based Affect Induction Procedures: A Systematic Review and Meta-Analysis. Emotion. 2015. December:16:1–11. [DOI] [PubMed] [Google Scholar]
  • 33.Göritz AS, Moser K. Web-based mood induction. Cognition and Emotion. 2006;20: 887–96. [Google Scholar]
  • 34.Gerrards‐Hesse A, Spies K, Hesse FW. Experimental inductions of emotional states and their effectiveness: A review. British journal of psychology. 1994;85: 55–78. [Google Scholar]
  • 35.Göritz AS, Moser K. Mood and flexibility in categorization: A conceptual replication. Perceptual and motor skills. 2003;97: 107–19. [DOI] [PubMed] [Google Scholar]
  • 36.Castro N, James LE. Differences between young and older adults’ spoken language production in descriptions of negative versus neutral pictures. Aging, Neuropsychology, and Cognition. 2014;21: 222–38. [DOI] [PubMed] [Google Scholar]
  • 37.Laukka P, Juslin P, Bresin R. A dimensional approach to vocal expression of emotion. Cognition & Emotion. 2005;19(5):633–653. [Google Scholar]
  • 38.Kamiloğlu RG, Fischer AH, & Sauter DA. Good vibrations: A review of vocal expressions of positive emotions. Psychonomic Bulletin & Review. 2020. January 2:1–29. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Siegman A, Boyle S. Voices of fear and anxiety and sadness and depression: The effects of speech rate and loudness on fear and anxiety and sadness and depression. Journal of Abnormal Psychology. 1993;102(3):430–437. [DOI] [PubMed] [Google Scholar]
  • 40.Goudbeek M, Scherer K. Beyond arousal: Valence and potency/control cues in the vocal expression of emotion. The Journal of the Acoustical Society of America. 2010;128(3):1322. [DOI] [PubMed] [Google Scholar]
  • 41.Hot P, Sequeira H. T50ime course of brain activation elicited by basic emotions. Neuroreport. 2013. November 13;24(16):898–902. [DOI] [PubMed] [Google Scholar]
  • 42.Dhaka S, Kashyap N. Explicit emotion regulation: Comparing emotion inducing stimuli. Psychological Thought. 2017;10(2):303–314. [Google Scholar]
  • 43.Balada F, Blanch A, Aluja A. Arousal and Habituation Effects (Excitability) on Startle Responses to the International Affective Picture Systems (IAPS). Journal of Psychophysiology. 2014;28(4):233–241. [Google Scholar]
  • 44.Hsiao E, Schwartz M, Schnur T, Dell G. Temporal characteristics of semantic perseverations induced by blocked-cyclic picture naming. Brain and Language. 2009;108(3):133–144. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Limonero JT, Fernández-Castro J, Soler-Oritja J, Álvarez-Moleiro M. Emotional intelligence and recovering from induced negative emotional state. Frontiers in psychology. 2015. June 19;6:816. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Carretero LM, Latorre JM, Fernández D, Barry TJ, Ricarte JJ. Effects of positive personal and non-personal autobiographical stimuli on emotional regulation in older adults. Aging clinical and experimental research. 2020. January:32(1):157–164. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.van der Zwaag MD, Janssen JH, Nass C, Westerink JH, Chowdhury S, de Waard D. Using music to change mood while driving. Ergonomics. 2013. October 1;56(10):1504–14. [DOI] [PubMed] [Google Scholar]
  • 48.Shevchenko Y, Bröder A. Noncompliance with online mood manipulations using film clips: how to detect and control for it. Heliyon. 2019;5(4):e01438. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Kenealy PM. The Velten mood induction procedure: A methodological review. Motivation and Emotion. 1986;10: 315–35. [Google Scholar]
  • 50.Gilam G, Hendler T. With love, from me to you: embedding social interactions in affective neuroscience. Neuroscience & Biobehavioral Reviews. 2016:68:590–601. [DOI] [PubMed] [Google Scholar]
  • 51.Soares AP, Macedo J, Oliveira HM, Lages A, Hernández-Cabrera J, Pinheiro AP. Self-reference is a fast-acting automatic mechanism on emotional word processing: evidence from a masked priming affective categorisation task. Journal of Cognitive Psychology. 2019. May 22:31(3):1–9. [Google Scholar]

Decision Letter 0

Hedwig Eisenbarth

29 Aug 2019

PONE-D-19-17931

Gradual positive and negative emotion induction using images: the effect of verbalizing emotional content

PLOS ONE

Dear Mrs. Out,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Based on mainly Reviewer 1's comments I encourage a revision of you manuscript focussing on those aspects of framing.

We would appreciate receiving your revised manuscript by Oct 13 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Hedwig Eisenbarth

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

3. We noted in your submission details that a portion of your manuscript may have been presented or published elsewhere: "Some results in the current manuscript have been published in an abstract for the CERE 2018 conference. This abstract focuses on the main effect of emotion induction on affective state of the participants. However, in the current manuscript, we also:

1) studied the effectiveness of the emotion induction on individual level, instead of only group level

2) compared our data to another study using the Velten method (Wilting et al., 2006)

3) Included tables and figures that are not in the CERE abstract

4) Discussed the literature on emotion and language production in more depth.

Additionally, the abstract is barely 1.5 pages, compared to our current manuscript of about 30 pages."

Please clarify whether this [conference proceeding or publication] was peer-reviewed and formally published. If this work was previously peer-reviewed and published, in the cover letter please provide the reason that this work does not constitute dual publication and should be included in the current manuscript.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Review of Gradual positive and negative emotion induction using images: the effect of verbalizing emotional content (PONE-D-19-17931)

In this manuscript, the authors conducted two emotion induction experiments inspired by the Velten method. Using stimuli from IAPS, the authors placed participants in one of three emotion induction conditions: negative, positive, or neutral, where images either increased in negative valence, increased in positive valence, or remained neutral. In the first experiment, participants saw each image and described what they saw for 10 seconds in one of the three valence conditions. In the second experiment, participants passively viewed the images for 10 seconds each. The authors reported successful emotion induction, most successful for the negative condition. They found the induction results comparable to the Velten method.

The methods, analyses, and conclusions were sound. My main issue with the manuscript is the novelty of the question. To me this does not demonstrate a new method of emotion induction. Rather, it introduces additional complications that may not be necessary. One critical test of the method’s novelty would be to ask why an experimenter who wants to induce negative emotions would use this method rather than showing a random set of highly negative IAPS pictures?

The gradual increasing of affective images might complicate emotion induction, such as imposing order effects and taking longer (400 seconds in this case). Alternatively, other studies have randomized blocks of negative images distributed throughout a task to remove order effects and to sustain negative affect induction. By using one long series of images, it is also possible that participants are just more responsive to the last portion of the most negative images, or that the most negative images at the end are made less salient because of participants are slowly ramped up to them.

Describing while viewing emotional images adds another layer of complication, such as attention and verbal skills. This may be more relevant as a comparison to other induction methods, or for explicitly testing the influence of verbalizing when exposed to emotional stimuli, which the authors partially reviewed but was not the intended question.

To clarify my issue, for example, if a gradual increasing method showed a more sustained or a different kind of negative affect induction, it would demonstrate novelty. Alternatively, if the authors could demonstrate specific behavioral consequences of this emotion induction method, or a particular reason for which a gradual increase of valence or the description of emotional stimuli is useful, or a particular set of questions or dependent measures that are differentially targeted by description of pictures rather than passive viewing, it would demonstrate novelty.

Other notes:

-what was the dependent variable of the ANOVAs? If it was valence ratings, using a magnitude change may reveal a greater and the interaction effect smaller. If it's not a magnitude change, if both negative and positive inductions were perfectly and equally effective, you would hypothesize a null time effect and predict only the interaction.

-the other component of the Velten method is the self-referential nature of the stimuli. There is now a literature on the impact of affective words when they influence the self vs. another. If the main thrust of the question is a direct comparison with the Velten method, this would ideally be addressed.

-despite comparable ratings, the positive valence IAPS pictures are sometimes considered to be not as significant as the negative valence pictures, possibly related to the greater ease in inducing negative emotions (which might account for the reduced positive induction). Also within the negative pictures, I’ve noticed the disgust images are more effective inducers than the fear pictures.

-Figure 3 should be bar plots. Line suggests an ordinal or continuous relationship between the three experiments

-the comparison of the emotion induction results with another study that used the Velten method could be placed in the Results section.

Reviewer #2: This is a sound and well presented study. It builds on earlier work on the elicitation of changes in emotional states via linguistic stimuli, and presents the results of two experiments involving visual stimuli. However, a limited amount of space is devoted to a discussion of the findings. I suggest that the authors include more explanation of the findings and more comments on their implications. It should also be clarified why the use of images, as opposed to linguistic stimuli, is described as more 'natural'.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 May 29;15(5):e0233592. doi: 10.1371/journal.pone.0233592.r002

Author response to Decision Letter 0


3 Mar 2020

Reviewer #1

Review of Gradual positive and negative emotion induction using images: the effect of verbalizing emotional content (PONE-D-19-17931)

In this manuscript, the authors conducted two emotion induction experiments inspired by the Velten method. Using stimuli from IAPS, the authors placed participants in one of three emotion induction conditions: negative, positive, or neutral, where images either increased in negative valence, increased in positive valence, or remained neutral. In the first experiment, participants saw each image and described what they saw for 10 seconds in one of the three valence conditions. In the second experiment, participants passively viewed the images for 10 seconds each. The authors reported successful emotion induction, most successful for the negative condition. They found the induction results comparable to the Velten method.

The methods, analyses, and conclusions were sound. My main issue with the manuscript is the novelty of the question. To me this does not demonstrate a new method of emotion induction. Rather, it introduces additional complications that may not be necessary. One critical test of the method’s novelty would be to ask why an experimenter who wants to induce negative emotions would use this method rather than showing a random set of highly negative IAPS pictures?

We thank the reviewer for their insightful and valid comments. We understand the main issue highlighted by the reviewer.

We realize now that we probably have not been clear enough in describing our aims and would like to clarify these below. The primary motivation was to study the effect of free verbal expression on emotion induction. Given that the relationship between language production and emotion induction is not well understood, we set out to investigate whether verbally describing emotional images would result in an enhanced emotional experience, compared to passively viewing the images. For this, we decided to pick a relatively natural and unconstrained approach, by presenting individuals with emotional IAPS images, asking them to describe the content freely.

To bring out this focus more clearly in the revised manuscript, we have adapted the title and we have now included a section of the paper exploring the content of participant’s verbal descriptions of the IAPS images. These descriptions were transcribed, and analyzed using Linguistic Inquiry and Word Count (LIWC), focusing on the production of emotional words as a function of the three conditions (positive, neutral, and negative), and over time (from the first image to the last image).We describe these in the Results section of a new subsection, ‘Exploratory analyses: Content analysis of image descriptions’.

A secondary motivation of our study is indeed exploring alternative emotion induction methods. As highlighted by, for example, Schmidt et al. (2018), most emotion induction methods are passive and not very ecologically valid. In our study, participants actively engaged in the induction method, especially in the setting where they were asked to verbally describe the images in such a way that other people would be able to identify the described image. Instead of reading out loud fixed statements (as in the Velten method), participants were asked to think for themselves and use their own words. We think this contributes to the need for more ecologically sound emotion induction methods. One attractive feature of the Velten method is its incremental nature. Somewhat surprisingly, this unique property of the procedure has not been studied or discussed in the literature, leaving us wondering whether the incremental manner of inducing mood is also effective when using (IAPS) images with affective content.

In the new manuscript, we now describe our focus more clearly in the Introduction.

The gradual increasing of affective images might complicate emotion induction, such as imposing order effects and taking longer (400 seconds in this case). Alternatively, other studies have randomized blocks of negative images distributed throughout a task to remove order effects and to sustain negative affect induction. By using one long series of images, it is also possible that participants are just more responsive to the last portion of the most negative images, or that the most negative images at the end are made less salient because participants are slowly ramped up to them.

Thank you for this comment. Indeed, for emotion induction purposes, it is common to present IAPS images in a random order (e.g., Capecelatro et al., 2013; Dhaka & Kashyap, 2017).

We conjecture that gradually increasing emotional valence of stimuli might have several advantages. First, the first stimulus is unlikely to shock or startle participants. When participants are startled by images, they might feel excessively tense and want to quit engaging in the study, which is both detrimental to the quality and quantity of the research, as well as ethically questionable. Velten argued in favor of gradual emotion induction, ‘to overcome the subjects’ presumable reluctance to experience unpleasant mood’ (Velten, 1967, p.68). Additionally, for some populations, the startle effect might be specifically unethical, because they could cause serious psychological or physiological harm, for example, individuals with certain mental disorders (e.g., PTSD, panic disorder), cardiovascular diseases, and the elderly.

Incidentally, we were unsure what the reviewer referred to with ‘taking 400 seconds longer’. We reckoned that they referred to the Spoken Descriptions of Images study, where we state that each image was viewed for 10 seconds, resulting in 400 seconds of viewing images (40 images x 10 seconds). We did not include information on the viewing time of the images in the Passively Viewing Images study, which might have been confusing. The viewing time per image was identical in both studies. We have now added this information to Passively Viewing Images study (Materials).

Describing while viewing emotional images adds another layer of complication, such as attention and verbal skills. This may be more relevant as a comparison to other induction methods, or for explicitly testing the influence of verbalizing when exposed to emotional stimuli, which the authors partially reviewed but was not the intended question.

First, we would like to clarify that we did intend to explicitly test the influence of verbalizing when exposed to emotional stimuli (as we described in response to the first comment from this reviewer). One of our initial hypotheses was that describing emotional images would enhance the effect on emotional state compared to passively viewing the images. We compared the two methods and concluded that there was no substantial difference in effect on mood.

We agree that attention and verbal skills could play a role in our ‘Describing and viewing images’ study. We did consider this, for example, by defining relevant exclusion criterion, such as having a speech disorder or limitation in the ability to speak fluently (for example, stuttering). Moreover, given that we tested a relatively homogenous group, we expect that other variabilities (cognitive and communicative abilities) will be randomly distributed throughout the sample, and therefore not have a major impact on our results.

We have reframed the revised manuscript, focusing more on the effect of verbalizing on affective state. We have added a new section, ‘Exploratory analyses: Content analysis of image descriptions’, exploring the content of the verbal descriptions of the images using LIWC, and also briefly discuss these results in the General Discussion.

To clarify my issue, for example, if a gradual increasing method showed a more sustained or a different kind of negative affect induction, it would demonstrate novelty. Alternatively, if the authors could demonstrate specific behavioral consequences of this emotion induction method, or a particular reason for which a gradual increase of valence or the description of emotional stimuli is useful, or a particular set of questions or dependent measures that are differentially targeted by description of pictures rather than passive viewing, it would demonstrate novelty.

We understand the reviewer’s point. As described above, in the revised manuscript we describe more explicitly that our primary interest is to study the effect of spoken language production on affective state. To demonstrate the novelty of our study more clearly, we have now transcribed and analyzed the verbal descriptions of the images using LIWC, which offers an additional dependent measure. Additionally, we have strengthened our discussion of the possible uses of our approach as an emotion induction method.

Other notes:

What was the dependent variable of the ANOVAs? If it was valence ratings, using a magnitude change may reveal a greater and the interaction effect smaller. If it's not a magnitude change, if both negative and positive inductions were perfectly and equally effective, you would hypothesize a null time effect and predict only the interaction.

In our analysis, we used a composite score (which we called "Affect") based on the 6 scales introduced in Mackie and Worth. This score ranges from 1 (very negative) to 7 (very positive) (see Materials for details). We have added this information now to ‘Change in emotional state' (both 2.2.2. and 3.2.2.). We agree with the reviewers interpretation of the possible effects; if both the negative and positive mood inductions were equally effective, we would get a null effect of mood induction (because the effects of the positive and negative condition cancel each other out) and only get an interaction between time and mood (because over time, the scores for the positive and the negative condition do start to diverge). However, as discussed in both Results sections (2.2 and 3.2), in our studies, only the negative mood induction was found to be successful.

-the other component of the Velten method is the self-referential nature of the stimuli. There is now a literature on the impact of affective words when they influence the self vs. another. If the main thrust of the question is a direct comparison with the Velten method, this would ideally be addressed.

The reviewer has a good point; indeed, our study and the Velten method differentiate with respect to self-referencing. Even though our main focus is on language production (as explained above), we do feel it is interesting to address this topic. We now briefly discuss recent work on this (such as Soares et al., 2019). When inspecting the picture descriptions, we find that only a small percentage (26%) is self-referential (e.g., ’I see…’), as opposed to many utterances in Velten’s original method. This might partially explain why our positive emotion induction was not effective. We now briefly touch upon this in the General Discussion (‘Future research’). For future research, it might be interesting to compare a self-referential condition (‘I see a happy couple’) to a non-self-referential condition of verbal image descriptions (‘This is a picture of a happy couple’). We have included this suggestion in the Future Research section.

-despite comparable ratings, the positive valence IAPS pictures are sometimes considered to be not as significant as the negative valence pictures, possibly related to the greater ease in inducing negative emotions (which might account for the reduced positive induction). Also within the negative pictures, I’ve noticed the disgust images are more effective inducers than the fear pictures.

We thank the reviewer for this clear observation. Indeed, this is common problem inducing positive emotions with various emotion induction procedures, and we refer to this problem in the General Discussion. We now briefly touch upon this issue regarding the effectivity of the positive valence images of the IAPS in specific.

-Figure 3 should be bar plots. Line suggests an ordinal or continuous relationship between the three experiments

The reviewer is, of course, correct. We have replaced Figure 3 with a bar plot.

- the comparison of the emotion induction results with another study that used the Velten method could be placed in the Results section.

We have moved the comparison with the Wilting study to ‘5. Comparing our results to the original Velten method’.

Reviewer #2

This is a sound and well presented study. It builds on earlier work on the elicitation of changes in emotional states via linguistic stimuli, and presents the results of two experiments involving visual stimuli.

However, a limited amount of space is devoted to a discussion of the findings. I suggest that the authors include more explanation of the findings and more comments on their implications.

It should also be clarified why the use of images, as opposed to linguistic stimuli, is described as more 'natural'.

We thank the reviewer for their compliments, and agree with their suggestions. We have expanded our General Discussion section, now explicitly discussing the limitations and strengths of our study. We discuss the possibility of order effects, the additional challenges of the verbalization of the stimuli, and how our ‘Describing Images’ study contributes to the need for more ecological and engaging mood induction procedures. We have rewritten the future research section, and added an implications section. We have also added new material, now dedicating a section of the paper to the analysis of the content of the verbal descriptions of the IAPS images (‘Exploratory analyses: content analysis of image descriptions’).

We also explain in somewhat more detail what we mean when we claim that our emotion induction method is more natural than the Velten method. This is because participants in our study were able to use their own words to describe affective stimuli, as opposed to the Velten method in which they read out loud fixed affective statements.

Attachment

Submitted filename: Comments reviewers _revision.docx

Decision Letter 1

Hedwig Eisenbarth

30 Mar 2020

PONE-D-19-17931R1

The effect of language on emotion: verbalizing images gradually increasing in emotional content

PLOS ONE

Dear Mrs. Out,

Sorry for the delay with getting back to you. Unfortunately we did not have the chance to obtain reviewer of the two reviewers who had previously reviewed your manuscript. However, we were lucky to find two experts in the field who were able  to review your revised manuscript, also in light of earlier comments and the original version of your manuscript.

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The comments of reviewer 3 might be new but seem highly valuable in terms of consistent wording and conceptualisation of emotion, valence and arousal. Those clarifications along side the suggested edits will definitely strengthen your manuscript, therefore I suggest you to follow those points.

We would appreciate receiving your revised manuscript by May 14 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Hedwig Eisenbarth

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: (No Response)

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: Partly

Reviewer #4: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: No

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: This paper introduces new data that can be brought to bear on existing debates about the influence of language on (the intensity of) affective experience. The authors have made some valuable changes and additions in their revisions - in particular, the examination of the language used to describe the images. Yet there are still some points where clarifications and perhaps new analyses are needed. Please note that, although I did not serve as a reviewer during the first round, I read both versions of the manuscript before preparing my comments on the revision.

1. I would be cautious with the use of the word ‘emotion(al)’ throughout the manuscript. What is being induced and measured is a change in affect, valence, or un/pleasantness. The ‘affective content’ of the stimuli is therefore a more appropriate description. Likewise, I would refer to an ‘affect induction’ rather than an ‘emotion induction’, to ‘affective language’ rather than ‘emotional language’, etc.

2. At several points in the manuscript, the authors appear to conflate valence, arousal, and intensity (e.g. page 9 line 221). In relation to the selection of stimuli, I would also like to see the authors comment on how the “strong increase in arousal for negative images” (page 9, line 226) does or does not impact their results and conclusions. Could it be that this arousal difference is driving the negative affect induction? Relatedly, the authors suggest that positive and negative images may be more arousing than neutral images, which may explain differences in the number of words used in image descriptions (page 24, line 569). This conjecture could be directly tested.

3. Can the authors clarify why they analyzed pre- and post-induction ratings separately, rather than creating change scores? It seems that change scores would preserve individual differences while streamlining the analyses and narrative. If the authors believe change scores are not appropriate, then I would highly recommend transforming the affect ratings to be centered on 0, such that negative ratings indicate negative affect, and positive ratings indicate positive affect. This will greatly assist in the interpretation of results and figures. For example, lines 276-8 (“lower scores indicate higher levels of negative emotion; higher scores indicate higher levels of positive emotion”) would be much easier to follow if scores were centered on 0.

4. There appears to be a slight mis-interpretation of the main effects reported on page 16 of the results. On lines 385-6, the authors state that “describing images enhanced emotional state in the participants”. To the contrary, a main effect of study type merely indicates that affect was more pleasant overall in Study 1 – this effect alone does not indicate an effect of the induction. Similarly, on lines 390-1, the authors state that “the emotion manipulation was effective for individuals exposed to negative images”. A main effect of condition simply means that affect was lower overall in the negative condition; causality cannot be inferred without examining change in affect.

5. Much of the authors’ argument throughout the manuscript hinges on the effectiveness of having participants verbally describe the evocative images. Yet this argument isn’t always supported by the analyses and results. For example, in comparing the present studies with the work of Wilting et al. (pages 17-8), the authors only analyze the post-induction affect ratings for all 3 studies. Without taking pre-induction ratings into account, however, this comparison isn’t meaningful (i.e. without controlling for where participants started, we don’t know how much they were influenced by the task). Similarly, the authors state that “verbally describing emotional images is an effective method to induce, especially negative, emotional states” (page 22, lines 525-6), while the results indicate no effect of study (page 17, lines 401-2). Also, the claim that “participants generally reported stronger emotions after describing emotional images compared to only viewing them” (page 23, lines 533-4) conflates valence with intensity – higher ratings indicate more pleasant affect, which actually works against the effectiveness of a negative affect induction. These aspects should be revised to clarify the findings and contribution of the present work.

6. Miscellaneous:

a. I prefer the original title, as the revised title seems to place undue emphasis on the effect of language on emotion (see above), and is also easy to misread as “verbalizing images gradually increases emotional content”

b. The original version of the manuscript included a brief description of the Velten method. This seems to have been removed in the revision, and I think it should be added back to help readers such as myself who were not otherwise familiar with the method.

c. Page 5, line 123: “especially for individuals who are free to use their own words” – I would rephrase as “especially when individuals are allowed to use their own words”

d. “Self-referral” (e.g. page 3, line 68) should be “self-referring” or “self-referential”

e. Page 9, line 212: what does it mean that the neutral images were pseudo-randomly selected?

f. Page 10, line 236: “electing” should be “eliciting”

g. Page 20 line 469: “Data were aggregated” (data is a plural noun)

h. Page 22, line 530: I would replace “happiness” with “positive affect”

Reviewer #4: The Authors did addressed all the points raised by reviewer 1. However, some of the authors' responses raise more questions. In particular, regarding the claims of incremental mood induction being "ecologically superior", the authors also pointed out that incremental induction of emotion has not been tested before on affective content images. Thus, only further investigations will support their statement of ecological validity. I suggest deleting lines 615-624 of the draft, and reference to "ecological sound" as a suggestion and not as a fact (lines 658-660 of the draft).

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: No

Reviewer #4: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 May 29;15(5):e0233592. doi: 10.1371/journal.pone.0233592.r004

Author response to Decision Letter 1


28 Apr 2020

Dear Dr. Eisenbarth,

Please find enclosed a revision of our manuscript “Gradual positive and negative affect induction: the effect of verbalizing affective content” (PONE-D-19-17931R1). We would like to thank you and the two reviewers of our revised manuscript for the time and consideration. We were happy to read that reviewer #4 feels all initial comments have been adequately addressed, and agree that the last comments and suggestions from reviewer #3 are valuable in their attention to consistent wording and the clear conceptualisation of emotion, valence and arousal. Below we describe how all separate comments by reviewer #3 and #4 were dealt with.

We hope to have addressed all comments and suggestions appropriately, and that the current revision will be judged favourably as a contribution to PLOS One.

Kind regards,

Charlotte Out, Emiel Krahmer and Martijn Goudbeek

-------------------------------------------------------------------------------------------------------------------

Reviewer #3

This paper introduces new data that can be brought to bear on existing debates about the influence of language on (the intensity of) affective experience. The authors have made some valuable changes and additions in their revisions - in particular, the examination of the language used to describe the images. Yet there are still some points where clarifications and perhaps new analyses are needed. Please note that, although I did not serve as a reviewer during the first round, I read both versions of the manuscript before preparing my comments on the revision.

We thank the reviewer for their comments and compliments.

1. I would be cautious with the use of the word ‘emotion(al)’ throughout the manuscript. What is being induced and measured is a change in affect, valence, or un/pleasantness. The ‘affective content’ of the stimuli is therefore a more appropriate description. Likewise, I would refer to an ‘affect induction’ rather than an ‘emotion induction’, to ‘affective language’ rather than ‘emotional language’, etc.

The reviewer is correct and we have changed this accordingly.

2. At several points in the manuscript, the authors appear to conflate valence, arousal, and intensity (e.g. page 9 line 221). In relation to the selection of stimuli, I would also like to see the authors comment on how the “strong increase in arousal for negative images” (page 9, line 226) does or does not impact their results and conclusions. Could it be that this arousal difference is driving the negative affect induction? Relatedly, the authors suggest that positive and negative images may be more arousing than neutral images, which may explain differences in the number of words used in image descriptions (page 24, line 569). This conjecture could be directly tested.

Regarding the conflation of valence, arousal and intensity, we have rewritten the manuscript, carefully considering the appropriate use of these terms. For example, we have changed line 221 to ‘affective content’ instead of ‘intensity’.

Indeed, the strong increase in arousal for negative images might impact our results and conclusions. To control for this, we followed the suggestion of the reviewer, now also looking at the correlations between the IAPS valence and arousal scores for the positive and negative images and the average number of words used by participants to describe these images. The results indicate that both valence and arousal were correlated to the number of words used to describe the images in the affective conditions, however, correlations between word use and valence were substantially higher. We describe these results in the Discussion.

3. Can the authors clarify why they analyzed pre- and post-induction ratings separately, rather than creating change scores? It seems that change scores would preserve individual differences while streamlining the analyses and narrative. If the authors believe change scores are not appropriate, then I would highly recommend transforming the affect ratings to be centered on 0, such that negative ratings indicate negative affect, and positive ratings indicate positive affect. This will greatly assist in the interpretation of results and figures. For example, lines 276-8 (“lower scores indicate higher levels of negative emotion; higher scores indicate higher levels of positive emotion”) would be much easier to follow if scores were centered on 0.

To some extent, this might to be a matter of subjective preference. Given that the effect of time (pre- and posttest) is important for answering our hypothesis (does affective state change after viewing/describing positive, neutral or negative images?), we prefer the repeated measures analyses keeping both the pre- and posttest scores, because we feel the results are more intuitive and easier to interpret than change scores. Change scores also make it harder to see how participants actually reported to feel in the different conditions.

However, we understand the concern of the reviewer and agree that the results sections could be improved to make them more easy to interpret. Therefore, we have now added change scores to Table 2, and clarified our explanations of the results by streamlining our language following the suggestions of the reviewer.

4. There appears to be a slight mis-interpretation of the main effects reported on page 16 of the results. On lines 385-6, the authors state that “describing images enhanced emotional state in the participants”. To the contrary, a main effect of study type merely indicates that affect was more pleasant overall in Study 1 – this effect alone does not indicate an effect of the induction. Similarly, on lines 390-1, the authors state that “the emotion manipulation was effective for individuals exposed to negative images”. A main effect of condition simply means that affect was lower overall in the negative condition; causality cannot be inferred without examining change in affect.

The reviewer is correct; thank you for spotting this. Regarding the first comment, we have changed this to “(…) indicating that affective state was overall more positive for participants in Study I, compared to Study II”. Regarding the second comment, we have changed this to “(…) indicating that affect was lower overall in the negative condition, both for participants

that described the images and for those that did not.’

5. Much of the authors’ argument throughout the manuscript hinges on the effectiveness of having participants verbally describe the evocative images. Yet this argument isn’t always supported by the analyses and results. For example, in comparing the present studies with the work of Wilting et al. (pages 17-8), the authors only analyze the post-induction affect ratings for all 3 studies. Without taking pre-induction ratings into account, however, this comparison isn’t meaningful (i.e. without controlling for where participants started, we don’t know how much they were influenced by the task).

Unfortunately, the pre-test scores of the Wilting et al. study were never collected. We understand the reviewer’s concern and have therefore decided to removed this comparison.

Similarly, the authors state that “verbally describing emotional images is an effective method to induce, especially negative, emotional states” (page 22, lines 525-6), while the results indicate no effect of study (page 17, lines 401-2).

We have rephrased this sentence to “Verbally describing and passively viewing affective pictures successfully induced negative affective states (in the negative condition), but not positive affective states (in the positive condition). Our findings did not support the hypothesis that verbally describing affective pictures would induce stronger affective states than passively viewing them.’

Also, the claim that “participants generally reported stronger emotions after describing emotional images compared to only viewing them” (page 23, lines 533-4) conflates valence with intensity – higher ratings indicate more pleasant affect, which actually works against the effectiveness of a negative affect induction. These aspects should be revised to clarify the findings and contribution of the present work.

We have changed this sentence to ‘Participants reported slightly more positive affect after describing the positive images, compared to passively viewing them. However, this difference was small and not significant.’

6. Miscellaneous:

a. I prefer the original title, as the revised title seems to place undue emphasis on the effect of language on emotion (see above), and is also easy to misread as “verbalizing images gradually increases emotional content”

b. The original version of the manuscript included a brief description of the Velten method. This seems to have been removed in the revision, and I think it should be added back to help readers such as myself who were not otherwise familiar with the method.

c. Page 5, line 123: “especially for individuals who are free to use their own words” – I would rephrase as “especially when individuals are allowed to use their own words”

d. “Self-referral” (e.g. page 3, line 68) should be “self-referring” or “self-referential”

e. Page 9, line 212: what does it mean that the neutral images were pseudo-randomly selected?

f. Page 10, line 236: “electing” should be “eliciting”

g. Page 20 line 469: “Data were aggregated” (data is a plural noun)

h. Page 22, line 530: I would replace “happiness” with “positive affect”

We have changed the title to “Gradual positive and negative affect induction by describing versus viewing affective pictures’’. Moreover, we have added a brief description of the original Velten method to the Introduction. The small errors have been corrected.

Regarding e), we understand the confusion and changed ‘pseudo-random’ to ‘random’, because neutral images were selected in a similar fashion as the affective images, i.e. at random.

Reviewer #4:

The Authors did addressed all the points raised by reviewer 1. However, some of the authors' responses raise more questions. In particular, regarding the claims of incremental mood induction being "ecologically superior", the authors also pointed out that incremental induction of emotion has not been tested before on affective content images. Thus, only further investigations will support their statement of ecological validity. I suggest deleting lines 615-624 of the draft, and reference to "ecological sound" as a suggestion and not as a fact (lines 658-660 of the draft).

We thank the reviewer for this observation and understand the need for clarification.

We have substantially rephrased lines 615-624, now suggesting that the engaging nature of the method is a potential strength, and we have removed mentioning of ecological superiority. In the Implications section of the Discussion, we now suggest that our method is possibibly more ecologically sound, instead of stating this as a fact.

Attachment

Submitted filename: Comments revision_2.docx

Decision Letter 2

Hedwig Eisenbarth

11 May 2020

Gradual positive and negative affect induction: the effect of verbalizing affective content

PONE-D-19-17931R2

Dear Dr. Out,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Hedwig Eisenbarth

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: Yes

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: The authors have addressed my comments on the manuscript. I thank them for their responsive and attentive revisions. I especially appreciate the streamlined analyses and new title.

Reviewer #4: The authors have met the reviewers' recommendation . I consider the paper now ready for publication.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: No

Reviewer #4: No

Acceptance letter

Hedwig Eisenbarth

18 May 2020

PONE-D-19-17931R2

Gradual positive and negative affect induction: the effect of verbalizing affective content

Dear Dr. Out:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Hedwig Eisenbarth

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Table. Positive and negative IAPS pictures that were initially selected but excluded based on our exclusion criteria.

    (DOCX)

    S2 Table. Final selection of positive (increasing in valence), negative (decreasing in valence), and neutral IAPS pictures.

    (DOCX)

    S3 Table. Self-referential and other-referencing in the pictures descriptions, sorted by condition, counted by LIWC.

    (DOCX)

    S1 Fig. Valence (y-axis) of IAPS pictures by bin (x-axis).

    (TIF)

    S2 Fig. Arousal (y-axis) of IAPS pictures by bin (x-axis).

    (TIF)

    Attachment

    Submitted filename: Comments reviewers _revision.docx

    Attachment

    Submitted filename: Comments revision_2.docx

    Data Availability Statement

    The datasets and syntax can be found in the folder ''Gradual emotion induction with a visual Velten method'' --> ''Datasets and syntax'' at the Open Science Foundation: https://osf.io/7zgct/


    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES