Emotional content and sharing on Facebook: A theory cage match

Abstract

While emotional content predicts social media post sharing, competing theories of emotion imply different predictions about how emotional content will influence the virality of social media posts. We tested and compared these theoretical frameworks. Teams of annotators assessed more than 4000 multimedia posts from Polish and Lithuanian Facebook for more than 20 emotions. We found that, drawing on semantic space theory, modeling discrete emotions independently was superior to models examining valence (positive or negative), activation/arousal (high or low), or clusters of emotions and was on par with but had more explanatory power than a seven basic emotion model. Certain discrete emotions were associated with post sharing, including both positive and negative and relatively lower and higher activation/arousal emotions (e.g., amusement, cute/kama muta, anger, and sadness) even when controlling for number of followers, time up, topic, and Facebook angry reactions. These results provide key insights into better understanding of social media post virality.

Many specific emotions predict social media sharing better than clusters of emotions, valence, or activation.

INTRODUCTION

Recent estimates suggest that 59.4% of the global population (and 77.8% of those aged 18 and higher) uses social media and that the average user spends roughly 2.5 hours a day on social media platforms (1). Social media platforms are increasingly a medium where people, from ordinary citizens to famous celebrities and politicians, share their opinions and their lives, learn about current events, attempt to influence each other, and spread misinformation [e.g., (2–4)]. Individuals, governments, and companies are interested in what content causes readers to share posts because of the power of social media to influence public and consumer opinion [e.g., (5, 6)]. One aspect of posts thought to cause message virality is emotional content (7, 8), and the general presence of emotions can encourage message sharing (8–10). However, recent emotion theory has not been applied to the question of message sharing. Competing emotion theories, by virtue of what emotions they cover, imply different predictions as to what aspects of emotions might be related to virality: the valence (positive/negative aspect) of emotions, the activating/arousal potential (energy) of those emotions, or different, specific emotions themselves. The goal of this study is to compare these theories on their relationship to post sharing. Rather than being the final word about which discrete emotions are related to social media sharing, we hope to leverage advances in the psychology of emotions. Conducting this research can help inform both future studies of social media post sharing and contribute to an understanding of how emotions are expressed in social media.

Psychology of emotions

Different psychological theories have debated the nature of emotion (11–20). Most current theories have converged on three insights: (i) emotions arise from interpreting and evaluating what is happening (e.g., the situation); (ii) emotions have variability within and between cultures and individuals; and (iii) humans share some universal aspects of emotions (11, 12, 14). Core affect theory, also categorized as a constructivist theory, distinguishes emotions along two dimensions: valence (negative to positive) and activation/arousal [levels of alertness or energy from deactivating to activating; (11, 13)]. Sadness, for example, is negative and deactivating, whereas happiness is positive and activating (13). One rival theory raised by Ekman and others identifies a small list of emotions defined as “basic emotions”: anger, happiness, disgust, fear, surprise, and sadness, with contempt added later (15, 16). Basic emotions are of short duration and have distinctive experiences, specific nonverbal (e.g., facial) expressions, and particular physiology (15, 16). This theory has been criticized on several grounds, including that the facial expression of emotions varies by cultures, situations, and individuals [e.g., (21)]. There are also lively discussions and research as to whether the basic emotion approach is the right way to characterize emotions in general [e.g., (16, 22, 23)]. Regardless of which theory best represents the nature of emotions, we were interested in using a theory that would be more appropriate for the study of emotions in social media, which, by its nature, relies on multiple types of emotional expression, not just facial (e.g., nonverbal and verbal); has less capacity to portray physiological responses (e.g., heart rate); and exists within particular historical and sociocultural contexts.

More recently, semantic space theory moves beyond and incorporates these rival emotions theories to describe dozens of emotions such as confusion, amusement, admiration, awe, sexual desire, and nostalgia (17–20). The creators of semantic space theory not only went beyond the existing paradigms to better understand the nature of emotions as potentially distinct clusters of emotional experiences, appraisals, behaviors, and influences but also acknowledged these clusters as simultaneously contextualized, variable, and recognizable within and across cultures (19). They brought advanced statistical techniques and multiple large corpora (e.g., of vocalization, nonverbal facial and body expression, and feelings evoked by videos and music) to ask whether there are distinct boundaries between emotions, how many clusters of emotions there are, and whether emotional experiences correspond to specific emotions or valence and arousal (19). They argued that a particular signal (e.g., widening eyes) need not map to a single emotional state but can still “carry informational value regarding emotional experience” [(19), p. 127]. Emotions in this theory exist within a larger, high-dimensional semantic space where blends can exist (e.g., anger/disgust); emotions can be of long and short duration; and emotions can be reflected and detected in a variety of experiences, expressions (facial, vocal, etc.), and physiology. In their visualizations of their findings, these emotions are distinct and, because they exist in a semantic space, may be closer to or farther from each other but have fuzzy boundaries (17–20). Their research identified, depending on the corpora, 24 to 28 distinct subjective emotional experiences and/or emotions with gradients between them (19). They found that emotions exist in high-dimensional spaces that are not sufficiently captured by valence and arousal (17): Valence/arousal and the six basic emotions each only explained roughly 30% of the variance in emotional behavior and experience (19). This finding suggests that neither of those competing approaches (core affect or basic emotions) fully captures the variety of human emotional expression. Thus, we wished to apply semantic space theory—a more comprehensive approach—to the question of emotions in social media. Adding to (and, in most cases, overlapping with) the semantic space approach, we included research from psychology that focuses on specific emotions, such as kama muta, or the heartwarming feeling that can be a reaction to cute or infant-like stimuli (24–27).

Because the six (seven, with the updated version) basic emotions are included within the various findings of semantic space theory (17), the goal of this study was to pit the theory of core affect against the recent semantic space approach in terms of what helps predict social media post sharing. Most research on emotions and social media sharing (compared to the psychology of emotions) examines valence, activation, and/or a small number of basic emotions. However, one cannot detect what one does not measure, which is why we used semantic space theory to suggest which emotions to study (among other emotions, see Materials and Methods). We used trained coders to annotate the full multimedia content of thousands of Facebook posts to compare models of valence (core affect theory), activation/arousal (also core affect theory), and more than 20 discrete emotions based on semantic space theory and research on individual emotions (26). We also (as another model to compare) clustered discrete emotions together using dimensionality reduction to test whether empirical parsimony could be useful and, for completeness, included a basic emotion model. While we had predictions for each model, the overarching aim was to determine which model was the best fit for the social media data. We note that, while we suggest potential explanations for some of these predictions, this study, as with most correlational/historic studies of social media, cannot make causal claims.

Social media sharing and emotion

For the valence model, we predicted that valence would increase sharing such that the greater the post’s positive or negative valence, the more the post is shared. This prediction is based on prior research, which has had mixed findings. For instance, in one sample, tweets with negative sentiment have been associated with more retweeting compared to tweets with positive sentiment, but, in others, the presence of sentiment in general, positive or negative, was associated with more retweeting (8). Similarly, tweets by British politicians with negative content were more likely to be retweeted compared to tweets with positive content, particularly if the tweet included a fear appeal (28). Using an experimental design, Roberts and colleagues (29) found that, using variations in news headlines, negative words increased click-throughs significantly, which they suggest is due to a negativity bias or general human preference for paying attention to negative stimuli. While click-throughs are a measure of interest rather than sharing, they are related to the overall construct of engagement with social media. Conversely, a meta-analysis of sharing on social media in health and crisis communication found a small positive effect for posts with any valence on sharing, but with positive posts significantly more likely to be shared than negative or simultaneously positive and negative posts (9). Other studies suggest that the findings for valence may be weak or not significant in all situations (30, 31). For instance, a study using lexical (word-based) methods to detect emotion in a large Twitter corpus found that neither emotion words nor moral words alone predicted sharing tweets but that moral-emotional words did predict sharing in their first study on gun control (7). In the paper’s second study on same-sex marriage, the authors found effects for both emotion words and moral-emotional words, and, in their third study on climate change, they found effects for moral, emotional, and moral-emotional words. In digging deeper, they found that whether negative or positive words predicted sharing varied depending on the topic (e.g., positive words predicted sharing for same-sex marriage but not climate change; both positive and negative words predicted gun control tweet reshares). The authors suggest that the causal factor might be a negativity bias in moral contagion, but, given that they focused on moral political topics, specifically, this mediator might not explain sharing for all types of posts. Even in the meta-analysis of sharing on social media, there were significant moderators for the effects of valence on social media sharing, including whether the study used human or machine annotation, suggesting [and echoing; (7)] that the effects for valence in general are small (9). Despite these contradictions, and indications that valence only has small effects, we nevertheless predicted that both types of valence could be associated with post sharing.

While these studies are generally correlational and do not test possible mediating factors, with the exception of Robertson and colleagues (29), other research suggests that negative moods might indicate problems that need to be focused on and/or fixed, whereas positive moods, if also activating, might encourage people to seek stimulation (32). Sharing positive content might also be done to make others also feel good and/or to make the sharer appreciated by others (33). These explanations could help explain social media post sharing: A negative post could indicate a problem that the sharer wants to inform others about, and a positive post might encourage engagement and help bond with friends.

For the activation/arousal model, we predicted that activation/arousal would be positively associated with social media sharing, regardless of whether the emotions involved are positive or negative (33, 34). In one study, inducing arousal (jogging versus sitting) had a significant positive effect on sharing a neutral news article (34). In a separate study, the more positive a New York Times stories was, the more it was shared; but, specific emotions with higher activation/arousal (awe, anger, and anxiety evoking stories) were related to more virality than a low activation/arousal emotion (sadness), regardless of valence (33). These studies are more clear-cut than those on valence, involving both experiments and relying on prior research that shows that physiological arousal readies people to different kinds of action and mobilization—in this case sharing information (33).

For the clustered, basic, and discrete emotion models, we drew on semantic space theory for our models of emotions, with the clustered model simply a data-driven test of whether parsimony would explain more than the full set of emotions. That noted, there exists research on specific emotions and information sharing. A prior study examined 14 emotions (e.g., anger, fear, guilt, love, trust, and surprise) based on theories of basic emotions using emotion words in hashtags of more than 4 million tweets (35). In addition to tweets with negative and positive hashtags being more likely to be shared compared to neutral emotions, joy, contempt, guilt, and distress were associated with more message sharing (35). In examining specific emotions using lexical methods, Brady and colleagues (7) found a negative effect on sharing for sadness, a low activation emotion, and no significant effects for disgust. Another study suggests that, among other findings, anger (and anticipation and trust) was associated with more shares, whereas surprise, disgust, and fear were associated with less sharing of rumors (36). These findings suggest that both the effects for degree of activation and particular emotions may depend on the topic under discussion.

Although not found in the 14-emotion study, anger and its cousin outrage are frequently studied emotions relevant to message sharing, representing both negative valence and high activation (13, 37–39). Anger can lead to heuristic processing, essentially reducing critical thinking and increasing reliance on stereotype, factors that are likely to lead to sharing of information conforming to a person’s views (37). Research on moral outrage, considered to combine anger and disgust, is thought to be particularly widespread in online environments (38), and anger at an opposing political party’s candidate predicts information sharing (40). Similarly, when researchers manipulated the valence of framing of true or false political news in a Romanian sample, negative framing of false news increased both anger and fear (the only negative emotions they tested), which then increased message virality (41). In another study, anger was positively associated with sharing for tweets about climate change (which was generally negative) but negatively associated with sharing for tweets about same-sex marriage [which generally had positive emotion words; (6)]. The Facebook algorithm allegedly boosted the visibility of messages where users had clicked the anger emoji reaction in the hopes of generating more user activity (42). Anger words were unrelated to clicking on headlines, however (29).

Anger is not the only discrete emotion that could be of interest although its relationship to heuristic processing could be an explanatory factor. In an interview study, participants who had familiarity with the content of a video reacted positively, with amusement or a cute-related emotion, and were then more likely willing to share the video (43). Although not the same as sharing posts, joy and fear words decreased, and sad words increased, the likelihood a person would click on a headline (29). At this time, other than anger, the different emotions have not been studied consistently enough to develop theory on each individual emotion; instead, we tested a broader array of emotions than is typically studied and, thus, compared theories at the high level: namely, core affect versus semantic space theories.

For the clustered emotion model, we tested the association of different specific and clustered emotions with post sharing. Semantic space emotion theory suggests that there may be overarching dimensions of emotions that can be mathematically detected, although there are slightly different patterns depending on how emotion is measured using facial expressions, vocal expressions, self-report, and so on (19, 20). Given that we measured emotion using trained coder annotation of multimedia posts and not facial or vocal expressions or self-report of feelings, we took a data-driven approach for our third model. Using data reduction techniques, we examined which clusters of discrete emotions tended to co-occur in these posts that were then likely to be related to post sharing. This model was exploratory, although we predicted that clusters that include anger, as well as joy, contempt, and/or guilt, might be more likely to be shared. For the basic emotion model, we included our covariates and, separately, the seven basic emotions (16). We expected findings similar to that noted previously: anger, and, possibly, sadness, contempt, and happiness, might be associated with post sharing.

Last, for the discrete emotion model, we examined the emotions in our corpus independently, but in a model together, such that each emotion was examined controlling for the others. This analysis included 23 emotions based on the broader psychology of emotions literature and semantic space theory (26). Given the unprecedented number of emotions in this study, this model is necessarily exploratory, albeit with the prediction that our findings should echo prior literature (e.g., joy, contempt, guilt, and anger associated with increased sharing). This discrete emotion approach may be fruitful, yielding findings above and beyond simple valence and activation/arousal.

Given the importance of social media sharing to political and social discourse, behavior, and attitudes offline and in general [e.g., (2, 4, 5, 7)], we focused on sociopolitical individual and group Facebook accounts (entities). We sampled social media from Poland and Lithuania, two countries where sociopolitical discourse was (is) contested internally and with Russia (4, 44–47). There are, therefore, security implications for social media discourse not just for those two countries, but regionally and globally. Using two countries and languages, we also strengthened cross-national generalizability. We chose Facebook because it was relatively popular in these countries during that time (48–49) and because it provided rich multimedia data that could offer a variety of emotional expression and content.

Our study advances the literature in several ways: We used an emotion annotation scheme designed for social media based on semantic space theory and other recent research on emotion [(26); e.g., (19, 25)]; we used Facebook as the platform, which is more commonly used in our target countries than other, well-studied platforms (49); and we used annotator assessments of all the emotions simultaneously for all the multimedia content of the post, which enables us to detect emotion combinations. Of note, our method enables us to detect a broader array of emotions than studies that are based on a small number of emotions or that use automated text-based or keyword-based annotation, which can miss emotional content both because of not including images, videos, etc., and may miss cultural and linguistic nuances. We compared the two theoretical frameworks in analyses of the five models. Valence and activation/arousal from core affect theory were modeled separately, and we analyzed discrete emotions, inspired by semantic space theory and other research on emotions, both independently and clustered.

RESULTS

Five negative binomial mixed-effects models, which take into account the skewed distribution of post sharing, were created to predict the number of shares a given post received by (i) valence, (ii) activation/arousal, (iii) scores derived from data-driven clustering using a principal components analysis (PCA), (iv) the individual magnitudes of the seven basic emotions described in basic emotion theory (16), and (v) the individual (unclustered) magnitudes of all 23 discrete emotions. Additional covariates were included to account for differences by country, the number of followers, and the amount of time the post was online. Random effects by user account and by topic were included to account for other idiosyncratic factors. We also tested for whether and how our specific emotion findings held after controlling for Facebook “angry” reactions for models 4 (basic emotions) and 5 [discrete emotions; (42)]. Additional details regarding these models and all of the variables used are provided in Materials and Methods and in the Supplementary Materials.

Across all of the models, there were no significant effects of country (Lithuania/Poland) in any model (all P values > 0.50); larger numbers of followers were associated with more sharing (all P values < 0.001); and the longer the post was online, the fewer shares it received (all P values < 0.001). These variables were statistically controlled for in the models. As noted, user account and topic were accounted for statistically by using random intercepts in the models.

With respect to the valence model, there was a significant interaction between the weighted valence score and the total post emotion magnitude (estimate = −0.001, z = −2.09, P = 0.04; see Fig. 1 and table S3). The effect for the overall emotion magnitude of a post was strongest for negative valenced posts, with the more strongly negative, the stronger the effect (i.e., more sharing). For posts with strong positive valence, the frequency/intensity of those emotions did little to affect sharing. In other words, posts with more frequent/intense negative emotions were shared the most, whereas posts with positive emotions, regardless of their frequency/intensity, were shared the least. These findings are consistent with some of the prior research (28).

Fig. 1. Interaction between weighted valence and total post emotion magnitude on the number of shares.

Note that, for illustrative purposes, weighted valence was divided into quintiles although it was treated as a continuous variable in the model. Shaded regions represent the 95% confidence interval. a.u., arbitrary units.

With respect to the activation/arousal model, there was no significant main effect of the weighted activation score nor an interaction between weighted activation/arousal and the total post emotion magnitude (see table S4). Higher total post emotion magnitude, however, was associated with a significantly higher number of shares; the more emotion (intensity/frequency) in general, the more post sharing occurred.

The clustered emotion PCA resulted in eight components, of which four had significant effects on sharing (see tables S5 and S6). The strongest effect, based on the model estimates, was driven by a component onto which loaded contempt, hate, and anger (estimate = 0.33, z = 10.80, P < 0.001): Higher scores on this component were associated with increased sharing. The next strongest effect was driven by a component onto which loaded sadness, fear, and empathic pain (estimate = 0.16, z = 6.01, P < 0.001): Higher scores on this component were also associated with increased sharing. After this was a component onto which loaded amusement, sexual attraction, and nostalgia (Est. = −0.11, z = −2.97, P < 0.01). For this component only, higher scores were associated with less sharing. The final, weakest effect was driven by a component onto which loaded confusion and wonder (Est. = 0.06, z = 2.03, P = 0.04), again with higher scores being associated with increased sharing.

For the seven basic emotion model, five of the seven emotions were significant predictors of sharing (see Fig. 2 and table S7): Higher levels of anger (estimate = 0.016, z = 7.31, P < 0.001), sadness (estimate = 0.008, z = 3.78, P < 0.001), fear (estimate = 0.009, z = 3.51, P < 0.001), and contempt (estimate = 0.012, z = 5.76, P < 0.001) were all associated with a significant increase in sharing, while a higher level of happiness (estimate = −0.007, z = −4.04, P < 0.001) was associated with a significant decrease in sharing (see Fig. 2). Disgust and surprise did not significantly predict sharing behaviors (table S7). We also tested this model controlling for the number of angry emoji reactions (table S8), as Facebook increased the visibility of messages when users had clicked this reaction during our sampled period (42). For this model, fear was no longer significant, but all remaining effects were the same.

Fig. 2. Significant effects of seven basic emotions on the number of shares, controlling for country, number of followers, and time up.

Note that each emotion is on its own scale to highlight the direction, rather than the strength, of the effect. Shaded regions represent the 95% confidence interval. n.s., not significant.

Last, the discrete emotion model, including each individual emotion as an independent variable, showed significant effects out of the total set of 23 (table S9). Higher levels of these emotions were associated with a greater number of shares: anger, contempt, cute/kama muta, wonder, pride, sadness, fear, and amusement. Sexual attraction and happiness both showed significant negative effects, such that higher levels for these emotions were associated with lower numbers of shares (see Fig. 3). We also tested this model controlling for the number of Facebook angry emoji reactions (table S10). For that model, fear was no longer associated with sharing, but love and admiration were now positively associated with sharing. Given the nature of these analyses, these emotions had effects even controlling for each other. Table 1 provides the estimates and P values for all the emotions in the discrete emotion and seven basic emotion models, both with and without the inclusion of the number of Facebook angry reactions as a covariate.

Fig. 3. Significant effects of discrete emotions on the number of shares, controlling for country, number of followers, and time up.

Note that each emotion is on its own scale to highlight the direction, rather than the strength, of the effect. Shaded regions represent the 95% confidence interval.

Table 1. Estimates and P values for each emotion in the discrete emotion and basic emotion models.

Table provides the estimates and P values (in parentheses) for each emotion in the discrete emotion and basic emotion models, both with and without the number of Facebook “angry” reactions included as a covariate. Asterisks indicate significance at P < 0.05.

Emotion	Basic emotions	Basic emotions with angry reactions	Discrete emotions	Discrete emotions with angry reactions
Anger	0.016 (<0.001*)	0.006 (<0.01*)	0.017 (<0.001*)	0.007 (<0.01*)
Hate	–	–	0.003 (0.27)	0.003 (0.35)
Contempt	0.012 (<0.001*)	0.008 (<0.001*)	0.01 (<0.001*)	0.007 (<0.001*)
Disgust	0.006 (0.69)	−0.009 (0.43)	0.005 (0.72)	−0.01 (0.36)
Embarrassment	–	–	−0.007 (0.44)	−0.008 (0.34)
Love	–	–	0.003 (0.35)	0.008 (0.02*)
Admiration	–	–	0.001 (0.55)	0.003 (0.04*)
Sexual attraction	–	–	−0.017 (<0.001*)	−0.014 (<0.001*)
Cute/kama muta	–	–	0.01 (0.03*)	0.013 (<0.01*)
Wonder	–	–	0.017 (<0.01*)	0.021 (<0.001*)
Pride	–	–	0.004 (0.01*)	0.005 (<0.01*)
Sadness	0.008 (<0.001*)	0.007 (<0.001*)	0.009 (<0.001*)	0.008 (<0.001*)
Nostalgia	–	–	−0.01 (0.24)	−0.007 (0.41)
Empathic pain	–	–	0.003 (0.38)	0.005 (0.15)
Gratitude	–	–	0.005 (0.09)	0.005 (0.08)
Envy	–	–	0.022 (0.43)	0.01 (0.69)
Fear	0.009 (<0.001*)	0.003 (0.21)	0.01 (<0.001*)	0.004 (0.09)
Relief	–	–	0.016 (0.19)	0.023 (0.05)
Confusion	–	–	0.002 (0.69)	−0.007 (0.2)
Surprise	0.004 (0.26)	<0.001 (0.98)	0.005 (0.15)	0.002 (0.58)
Happiness	−0.007 (<0.001*)	−0.004 (0.02*)	−0.009 (<0.001*)	−0.007 (<0.001*)
Excitement	–	–	−0.002 (0.37)	<0.001 (0.81)
Amusement	–	–	0.018 (<0.001*)	0.02 (<0.001*)

Among our five competing models, the models that emphasized individual emotions (the basic emotion and discrete emotion models) outperformed those that otherwise aggregated or clustered emotions along other dimensions (the valence, activation/arousal, and clustered emotion models) with the magnitude of the differences taken to be “strong evidence” (i.e., more than 10 points; (50); see Table 2). The basic emotion and discrete emotion models also competed with one another: While the discrete emotion model had the lowest Akaike information criterion (AIC) value, the basic emotion model had the lowest Bayesian information criterion (BIC) value. One reason for this discrepancy may lie in how each criterion accounts for the number of model parameters (i.e., the number of variables). For the AIC, the penalization is two times the number of parameters, while, for the BIC, the penalization is the number of parameters multiplied by the natural log of the number of observations. As such, when the number of observations is greater than 7, the BIC will always penalize models with more parameters more strongly than the AIC.

Table 2. AIC and BIC values for all models. Asterisks indicate the model with the lowest score for each criterion.

AIC, Akaike information criterion; BIC, Bayesian information criterion; a.u., arbitrary units.

Model	df	AIC (a.u.)	BIC (a.u.)
Valence	19	38,656.34	38,779.29
Activation/arousal	12	38,820.05	38,897.71
Clustered emotions	29	38,601.26	38,788.92
Basic emotions	16	38,422.44	38,525.98*
Discrete emotions	32	38,360.75*	38,567.83

Nonetheless, the results highlight the added benefit of examining many discrete emotions to explain sharing on social media. For example, the emotions that were significantly associated with more sharing in the discrete emotion model included both negative (anger) and positive (cute/kama muta) valence, as well as activating (amusement) and deactivating (sadness) emotions. While both amusement and happiness are positive valence, the former is associated with increased sharing and the latter with decreased sharing.

DISCUSSION

This study demonstrates that the semantic space theory of emotions is more informative and, compared to the valence, activation, and clustered models, more predictive with regard to social media sharing. In other words, examining emotions as discrete and numerous, rather than using either data-driven clusters or core affect dimensions (valence and activation/arousal), provides the greatest explanatory power. While more emotion, and more negative emotion, was associated with more post virality, the nuanced approach that examines different emotions separately was both statistically and practically more informative. If one only had the first two models (valence and activation/arousal), then one might assume that negative emotions are driving the effects of emotionality on sharing in general, but the semantic space models suggest that many different emotions contribute to sharing.

That noted, there is a distinct trade-off between explanatory power and model parsimony when examining larger sets of individual emotions, as inspired by semantic space theory. As discussed above, both the discrete emotion and seven basic emotion models show advantages over the valence, activation/arousal, and clustered emotion models but compete with each other depending on how strong the information criterion penalizes the number of model parameters (i.e., the number of variables). When the number of parameters is less strongly penalized as with the AIC, the discrete emotion model is the best fit; when the number of parameters is more strongly penalized as with the BIC, the seven basic emotion model is the best fit.

Nonetheless, it is important to note that not all of the seven basic emotions were significant predictors of sharing, and additional emotions surfaced as being strong predictors of sharing in the discrete emotion model. The emotions that were significant in the seven basic emotion model—anger, sadness, fear, happiness, and contempt—are also significant in the discrete emotion model and in the same directions. The discrete emotion model, however, also shows that other emotions, particularly positive emotions—sexual attraction, cute/kama muta, pride, and amusement—also contribute to sharing. Given that the basic emotion model is a subset of the discrete emotion model, a further argument in favor of the discrete emotion model is the fact that these additional emotions emerged as significant. Simplifying the discrete emotion model by removing nonsignificant terms would thus not result in the seven basic emotions. Therefore, we argue that the greater number of emotions represented in the discrete emotion model ultimately provides a more nuanced and informative picture of how emotional content might relate to sharing on social media, despite sacrificing some goodness of fit. In other words, one cannot find effects for what one does not measure.

As the social media environment is increasingly the public square, this research has implications for getting messages seen by more people. Rather than simply outraging social media users (increasing anger and contempt) in sociopolitical contexts, this study suggests that the public square can be a more positive place, as amusement, cute/kama muta, and wonder will also encourage message sharing.

No study is without limitations: By choosing Polish and Lithuanian entities posting around sociopolitical events, we do not cover the range of topics or emotions shared across global social media. Some emotions were rare in our corpus (e.g., envy, disgust, nostalgia, and relief), and prior literature warns that our findings might be context-dependent (9, 28). In addition, as noted, the effects of emotion on social media are not platform-agnostic. We know from news reporting that Facebook during at least some of the time the posts were created weighted the angry reaction emoji in what was seen on users’ feeds (42). When we tested the reported effects of the angry reaction emoji, we found that it had a large and significant effect, and also controlling for it changed which emotions were significantly related to sharing. Another possible limitation is the use of majority young adult female annotators; future research should determine whether there are gender or age differences in the accuracy of emotion annotation. That noted, by using cultural and linguistic natives who were present in-country when the events occurred, we avoid the invalidity of annotation that would occur due to cultural or linguistic mismatch [as other corpora have been criticized with having; (51)]. In addition, the fact that our method requires both intensive training to achieve reliability and separate annotations for the content of the post and the coders’ personal emotional responses helps coders distinguish between their own feelings and what the posts contain, minimizing personal bias.

Future work should repeat these analyses, also with in-country/in-language annotators using this or a broader array of emotions (e.g., including hope, curiosity, and boredom); in different countries and languages, around different kinds of events, and on additional topics [given previous findings suggesting differences by topic (6)]; and on platforms that may have different behind-the-scenes algorithms (e.g., YouTube), which might boost different kinds of content. In knowing which models are most predictive, future scholars could use them to develop theory, test hypotheses, and compare effect sizes between a broader array of emotions. While our sampled Facebook accounts included politicians and political parties, it also included many other sociopolitical entities. Across these different types of accounts, political stances were often either inconsistent or opaque to us. Future researchers interested in the effects of political affiliation could select only for a priori clear-cut political leanings of accounts, varied as is appropriate to the country under study. Similarly, other account-level factors such as personal/organizational variables (when relevant and identifiable for that account, e.g., ideology, gender or gender composition, size of organization, and occupation or type of business) could be selected and/or examined a priori rather than simply controlled for statistically.

If these findings replicate, then future research could also search for and test potential mediating factors. Our findings suggest that sharing is more complex than simple negativity bias, but future studies could include motives such as social identity, impression management, and need for affiliation (3) as well as emotion regulation, all of which might help explain sharing related to different positive emotions. Additional studies could also incorporate and extend research on moral-emotional emotions (7) by creating and validating a fully realized moral dimension annotation guide (not just a set of keywords) for use on multimedia content. With such a guide, annotators could examine the associations between moral and emotion annotation and examine their combined effects on sharing across a variety of topics.

This study is necessarily correlational, but future studies could be experimental. After informed consent, experimenters could use a simulated social media environment to show participants’ posts with different emotional content to determine specific causal relationships between particular emotions and post sharing. Future work could also delve deeper into the specific discrete emotions within this broader set that were related to sharing to better understand why those emotions in particular were relevant.

Nevertheless, our combination of theoretical and methodological rigor advances the field of emotions in social media. We apply modern, empirically supported emotion theory and culturally expert human annotation to multimedia posts. In other words, our work covers a broad array of emotional expression, both in terms of being based on rigorous theory with more emotions and in terms of assessing multimedia posts from two different countries and languages. Given the relevance of emotion to identifying and fighting misinformation (52), this study helps us gain a better understanding of the interplay between messages, psychological reactions, and sharing.

MATERIALS AND METHODS

Data collection and sampling

We collected Facebook posts from 330 Polish and 117 Lithuanian sociopolitical accounts (e.g., individual politicians, nongovernmental organizations, and media) from 2015 to 2020. Because that process resulted in too many posts for human annotation, we conducted a multistage purposive sampling around four sociopolitical events for each country: two elections, the first COVID-19 lockdown, and a country-specific event—a woman’s march in Poland and a political scandal in Lithuania. We chose social media accounts that were actively posting around each event and chose five Facebook posts per account around each event. In addition, to ensure that the collected content was sociopolitical in nature, we oversampled the Polish corpus using sociopolitical catchphrases popular on social media and the Lithuanian corpus for mentions of Russia. This sampling process resulted in us annotating a total of 1934 Facebook posts from 115 unique Lithuanian accounts and 3648 posts from 264 unique Polish accounts. The study followed the Institutional Review Board (IRB) guidelines: It was determined to be exempt by the University of Maryland IRB due to the public nature of the social media data. Upon opting to collect data on our annotators (e.g., demographics), the protocol was determined to still be exempt and then approved.

Emotion annotation

We opted to use human annotation for several reasons: (i) the primary automated tools are lexical, which would ignore the multimedia content of our corpus (more than 94% of our annotated corpus had some kind of multimedia content, e.g., image, video, emoji, and preview links); (ii) these lexicons are mostly developed in English, not for “low-resource” languages such as Lithuanian and Polish that we were interested in studying; (iii) there are validity issues with lexical ways of measuring emotion (53); and (iv) these lexicons are based on limited, outdated, contested, and/or invalidated emotion models [e.g., (15)].

Three teams of three or four Polish annotators (10 total) and one team of three Lithuanian annotators were extensively trained on the annotation guide over a week and then over four subsequent weeks iteratively, including heuristics for when to have consensus meetings for inter-annotator discrepancies (26). They annotated each post for the presence and intensity/magnitude of emotional content, including all multimedia content (text, images, video, audio, and emoji) on a scale of 0 (not present) to 100 (extremely intense and/or frequent in the post), referred to as the individual emotion magnitude (see fig. S1 for an example of annotation). Given that multiple emotions can coexist within the same social media post (35), each post was annotated for each of 23 emotions chosen for their relevance to social media: admiration, amusement, anger, confidence/pride, confusion, contempt, cute/kama muta, disgust, embarrassment (which included shame and guilt), empathic pain, envy (which included jealousy), excitement, fear (including nervousness, anxiety, and terror), gratitude, happiness (including joy and elation at the high end and contentment at the low intensity end), hate, love, nostalgia, relief, sadness, sexual attraction, surprise, and wonder (awe). This method was generally reliable, so long as those emotions are not extremely infrequent (26). We followed cross-cultural psychology methods in our codebook construction to ensure conceptual equivalence across Polish, Lithuanian, and English versions of the codebook even when direct translations did not exist (54): In addition to translating, we discussed every word and phrase as a multilingual group. In addition, we had annotators to judge separately the content of the post and their personal reactions to the post for each of the 23 (25 with the two other categories) emotions. We did not determine a consensus for emotional reactions, and we do not present the emotional reaction data here. This distinction helped coders to differentiate between the post and their own reactions and minimize bias (26).

We developed this annotation scheme over multiple iterations, testing with Twitter, Facebook, and YouTube for reliability, clarity of instructions, and the set of emotions (26). We chose this list of emotions by reviewing the literature on emotions and emotional categories [e.g., (15, 17, 55)] as well as psychological research that focuses on specific individual emotions [e.g., empathic pain; (56); awe/wonder; (57); and kama muta/reactions to cute content; (25, 27)]. We chose emotions that were both repeated across different category schemes (e.g., anger) and that might be particularly relevant to current social media (e.g., kama muta). We note that Ekman’s basic emotion category scheme defines emotions as involving distinctive signals but being brief and automatic (16). In analyzing social media, we drew on the broader definition of emotions used by semantic space theory, which was developed not just on immediate facial expressions but also on brief videos, vocalizations, and more (19). In social media, it is useful to identify emotions that are longer-lasting (e.g., love, admiration, and hate) and have context-specific and complex expressions, as well as to be able to identify emotion blends (26). Similarly to semantic space theory, we used an analogy to color (19) but, in our case, to guide annotators in generating 0 to 100 judgments based on the intensity or magnitude of emotion within a post: As with saturation or darkness of a color, light blue is a faint example of blue just as irritation is a lower-intensity version of anger, with wrath as the most intense. Our emotion annotation guide also captures blends by allowing for the simultaneous annotation of the 23 emotions, with later versions of the guide including additional emotions as relevant to research study (e.g., boredom).

Reliability results using two-way mixed effects, consistency, multiple-rater intraclass correlation coefficient [ICC(C,k)] for each emotion and annotator group ranged from 0.71 to 0.95, excluding emotions with fewer than 10 observed cases of that emotion for each annotator group. One exception was excitement (0.61) for the Lithuanian annotators (see table S1).

Topic annotation

We adapted the Comparative Agendas Project master codebook (58) to annotate our data for topics (see the Supplementary Materials), resulting in 237 low-level and 22 high-level codes. We trained the same Lithuanian and Polish annotators to annotate our dataset for topic. Annotation was performed at a post level, that is, annotators were instructed to assign one primary code for the whole post. They were also told that they can use a secondary code if needed, but the secondary code should be rarely used. Annotators first worked individually and met for consensus to discuss tranches of posts.

For the Polish coding, 83.5% of the posts had high-level topic agreement before consensus, as did 79.3% of the Lithuanian post coding. Of the 22 high-level categories (see fig. S2), the highest counts of the posts for the Polish corpus were first in the topic of government operations (e.g., political campaigns and personal political commentary) and then (in a decreasing order) in the topics of culture (e.g., arts, literary arts such as book releases, national holidays, and celebrity lifestyle), civil rights (e.g., gender discrimination and rights), health (e.g., pandemics), and law and crime (e.g., crime control, criminal and civil codes, and police). The majority of posts for the Lithuanian corpus were culture and then government operations, international affairs (e.g., issues related to the European Union, international terrorism, and a specific foreign country such as Russia or the United States), domestic commerce (e.g., banking, advertising, and tourism), and social welfare (e.g., assistance of different kinds, and emergency care). This variety suggests that our corpus covered topics beyond the sampled events.

Post shares and additional covariates

We measured shares by the number of Facebook post shares between the origin of a post and how many shares it had received as of data collection (2019–2020, via the CrowdTangle API). Given previous research (59), we controlled for the (log-normalized) number of followers, the country of origin (Poland and Lithuania), and the age of the post (log-normalized amount of time between data collection and post origin).

Valence and arousal assessments

For the valence and activation/arousal analyses, nine native Polish annotators and the third author (a Lithuanian native) independently rated each of the 23 emotions (not posts) on a scale of 0 to 10 for valence (negative to positive) and for activation/arousal, using definitions from the literature (13, 55). Reliability was high, with an ICC(C,k) of 0.98 for valence and of 0.90 for activation/arousal. We then averaged the annotators’ ratings to assign separate valence and activation/arousal scores for each of the 23 emotions (see Fig. 4). Because the models of clustered and discrete emotions used the number of shares per post as a dependent variable, to be comparable, the valence and activation models also required post-level values for valence and activation/arousal. Given that multiple emotions with different values could co-occur in a single post, weighted valence and weighted activation/arousal scores (a scale of 0 to 10) were calculated for each post by taking the weighted average of these scores based on the individual magnitudes of all emotions in a given post. We conducted a robustness test by having a multilingual annotator assess a subset of posts separately for global post-level valence (n = 95) and activation/arousal (n = 96) separately on the same scale of 0 to 10 (see the Supplementary Materials for details). Both weighted arousal (estimate = 0.20, t = 2.38, P = 0.02) and weighted valence (estimate = 0.45, t = 10.05, P < 0.001) were significant predictors of their post-level counterparts. This finding suggests that our valence and arousal algorithmic numbers would be a fair representation of post-level valence and arousal.

Fig. 4. Emotions by valence and activation/arousal aggregated across Polish and Lithuanian data.

Statistical analyses

Using the annotated data, five negative binomial mixed-effects models were created to predict the number of shares a given post received by (i) weighted valence, (ii) weighted activation/arousal, (iii) scores derived from data-driven clustering using a PCA, (iv) the individual (unclustered) magnitudes of the seven emotions described in basic emotion theory (16), and (v) the individual (unclustered) magnitudes of all 23 discrete emotions. The negative binomial distribution was chosen because it most accurately reflects the skewed count distribution of our data (see the Supplementary Materials). In models 1 and 2, both weighted valence and weighted activation/arousal were also allowed to statistically interact with total post emotion magnitude (the sum of the individual magnitudes of all emotions in a given post). For models 3, 4, and 5, the emotion scores are directly derived from or are the individual emotion magnitudes, respectively, so did not need to be tested for interaction effects. Given that the dependent variable, the number of shares, represents count data and follows a non-normal distribution, all of the models were specified to use a negative binomial distribution with a quadratic parameterization. All models also included various covariates (country, time since posting, and number of followers) as well as random intercepts by account and random intercepts by topic, which help account for idiosyncratic sharing behavior that might otherwise be driven by certain accounts and the different topics (see the Supplementary Materials for full descriptions of all variables). Model formulas and summaries are given in tables S3 through S10, with tables S8 and S10 also showing models controlling for Facebook angry reactions. The models (all non-nested) were compared using the AIC and BIC, where lower values represent better model fit and explanatory power (Table 2; see the Supplementary Materials for more information). Both the AIC and BIC penalize models with more degrees of freedom (i.e., more predictors), with the BIC enforcing a stronger penalization. While we describe the models and their significant effects in Results to highlight what each model is contributing, the main goal was to compare the models themselves to determine which set of predictors (based on the above theoretical positions) best explains sharing in this dataset.

Supplementary Materials

This PDF file includes:

Materials and methods

Supplementary Text

Figs. S1 to S3

Tables S1 to S11

References