Mining Social Media Data for Biomedical Signals and Health-Related Behavior

Abstract

Social media data have been increasingly used to study biomedical and health-related phenomena. From cohort-level discussions of a condition to population-level analyses of sentiment, social media have provided scientists with unprecedented amounts of data to study human behavior associated with a variety of health conditions and medical treatments. Here we review recent work in mining social media for biomedical, epidemiological, and social phenomena information relevant to the multilevel complexity of human health. We pay particular attention to topics where social media data analysis has shown the most progress, including pharmacovigilance and sentiment analysis, especially for mental health. We also discuss a variety of innovative uses of social media data for health-related applications as well as important limitations of social media data access and use.

1. INTRODUCTION

Humanity has crossed an important threshold in its ability to construct quantitative, large-scale characterizations of the networks of information exchanges and social interactions in human societies. Due to the widespread digitization of behavioral and medical data, the advent of social media, and the Internet’s infrastructure of large-scale knowledge storage and distribution, there has been a breakthrough in our ability to characterize human social interactions, behavioral patterns, and cognitive processes and their relationships with biomedicine and healthcare. For instance, electronic health records (EHRs) of entire cities can yield valuable insights on gender and age disparities in healthcare (1), and communication patterns on Twitter and Instagram can help us detect the spread of flu pandemics (2), warning signals of drug interactions (3), and depression (4).

Data science, together with artificial intelligence and complex networks and systems theory, has already enabled exciting developments in the social sciences, including the novel fields of computational social science and digital epidemiology (5, 6). Using social media and online data, researchers in these interdisciplinary fields are tackling human behavior and society in a large-scale quantitative manner not previously possible to study, for example, social protests (7, 8), fake news spread (9), and stock market prediction (10).

This approach also shows great promise in monitoring human health and disease given the newfound ability to measure the behavior of very large populations using individual self-reports (11). Indeed, population-level observation tools allow us to study collective human behavior (12-14) and, given the ability to obtain large amounts of real-world behavioral data, are expected to speed translational research in transformative ways (15) by monitoring of individual and population health (15-19). This promise has been substantiated by many recent studies. Google searches have been shown to correlate with dengue spread in tropical zones, for example (20). Although the accuracy of Google trends data alone has been problematic for epidemic flu modeling (21), such data add value in combination with other health data (22). Several studies have also shown that social media analysis is useful for: tracking and predicting disease outbreaks such as influenza (11, 23), cholera (24), Zika (25), and HIV (26); playing an important role in pharmacovigilance (3, 27-31); and measuring public sentiment and other signals associated public health issues such as depression (4, 32-34), human reproduction (35), vaccination rates (36, 37), and mental disorder stigma (13).

Social media data provide an increasingly detailed large-scale record of the behavior of a considerable fraction (about one-seventh) of the world’s population. Since 2017, 330 million people monthly have been active users of Twitter, making it one of the most populated global social networking platforms (38). Instagram currently has more than one billion monthly active users (39). It used to be the preferred social network among teens and young adults (ages 12–24), but since 2016 Instagram was surpassed by Snapchat in this demographic (40). Facebook, however, still has the overall majority of active users, with 2.45 billion users monthly (41) and a total of 2.8 billion across all of the company’s core products: Facebook, WhatsApp, Instagram, and Messenger (42).

Biomedical and public health researchers now have the ability to directly measure human behavior on social media, a promise emphasized by the National Institutes of Health, which consider this type of big data very relevant for biomedical research (43, 44). By “social media” we mean any user-generated content, including posts to sites such as Twitter and Facebook but also comments on disease-specific or health-related sites, forums, or chats. Most social media sites have been shown to be relevant for biomedical studies, including Twitter (15), Facebook (12), Flickr, Instagram (3), Reddit (45–47), and even Youtube (48, 49). Used together with other sources of data such as web search, mobility data, scientific publications, EHRs, and genome-wide studies, social media data help researchers build population- and individual-level observation tools that can speed translational research in fundamentally new ways.

Leveraging these kinds of data constitutes a novel opportunity to improve personalization in the face of multilevel human complexity in disease (50). For instance, new patient-stratification principles and unknown disease correlations and comorbidities can now be revealed (51). Moreover, social media data allow for a more direct measurement of the perspective of patients on disease, which is often different from that of physicians. Social media can help both patients and practitioners to understand and reduce this disconnect (52), which is known to hinder treatment adherence (53).

In summary, analysis of social media data enables more accurate “microscopes” of individual human behavior and decision making, as well as “macroscopes” for collective phenomena (5, 54). These micro- and macro-level observation tools can go beyond a descriptive understanding of biomedical phenomena in human populations by enabling quantitative measurement and prediction of various processes, as reviewed below. The ability to study humans as their own model organism is now a more reasonable prospect than ever before. Here we review recent work pertaining to the mining of social media for health-related information, that is, biomedical, epidemiological, or any social phenomena data of relevance to the multilevel complexity of human health (55). The review is structured as follows: Section 2 covers the use of social media for pharmacovigilance, including adverse drug reactions (ADRs) and drug-drug interactions (DDIs); Section 3 addresses the use of sentiment analysis tools to characterize individual and population behavior, especially mental health; Section 4 looks at the analysis of social media data for a wide variety of health-related applications; Section 5 considers limitations of the use of social media data; and Section 6 contains a conclusion and final remarks.

2. PHARMACOVIGILANCE

It is estimated that every year the United States alone spends up to $30.1 billion due to ADRs, with each individual reaction costing on average $2,262 (56). More than 30% of ADRs are caused by DDIs that can occur when patients take two or more drugs concurrently (polypharmacy). The DDI phenomenon is also a worldwide threat to public health (1, 57), especially with increased polypharmacy in aging populations.

Most ADR and DDI surveillance is still conducted by analysis of physician reports to regulatory agencies and by mining databases of those reports, such as the FDA (US Food and Drug Administration) Adverse Event Reporting System (FAERS) (58). However, clinical surveillance suffers from under reporting (59), which can be caused by clinicians failing to note adverse events or downgrading the severity of patients’ symptoms (60). For example, it has been well documented that depression and pain are underassessed by clinicians and underreported by patients, and therefore are under- or inappropriately managed, especially in specific cohorts such as athletes (61, 62). Even when clinicians are specifically trained or required to use screening tools for ADR, in practice these are done in a reactionary fashion at the time of a healthcare visit (59).

Such problems in reporting can be improved using new ways to study ADR and DDI phenomena provided by recently available large-scale online data about human behavior. Given the number of users, social media data are likely to allow for automated proactive identification of issues as they develop rather than after they occur and potentially become severe. Thus, analysis of social media data can identify underreported pathology associated with ADRs and further contribute to improvements in population health (Figure 1 shows a sample of social media posts containing drug and symptom mentions). For instance, it has been shown that the combination of clinical FAERS reporting and Internet search logs can improve the detection accuracy of ADRs by 19% (63), that discussions on Twitter related to glucocorticoid therapy reveal that insomnia and weight gain are more common adverse events than are reported in the UK regulator’s ADR database, and that more serious side effects are comparatively less discussed (52). Another study has compared patient reports of ADRs on social media (various discussion forums on health-related websites) with those of clinicians on EHRs for the case of the two drugs aspirin and atorvastatin (64). This study found that the most frequently reported ADR in EHRs matched the patients’ most frequently expressed concerns on social media. However, several less frequently reported reactions in EHRs were more prevalent on social media, with aspirin-induced hypoglycemia being discussed in social media only. The observed discrepancies and the increased accuracy and completeness of social media reports relative to those from regulator databases and EHRs have revealed that physicians and patients have different priorities (53). This suggests that social media data may be used to provide a more complete measurement of impact on quality of life (52), making it a useful complement to physician reporting.

Figure 1.

Selected sample of social media posts depicting known drug and symptom mentions, (a) Instagram photos depicting a variety of drugs. (b,c) Captions of Instagram posts. The two captions in panel b were posted two days apart by the same user, showing in the second post a possible side effect from a drug administration mentioned in the first post. (d) Twitter posts containing drugs known to be abused. (e) Epilepsy Foundation forum post and comments from users asking questions and sharing experiences over drug dosage (Keppra). For all examples, usernames, number of likes, and dates were omitted for privacy, and some content was modified for clarity and to maintain user anonymity. Terms of pharmacovigilance interest, including drug names, natural products, and symptoms, are highlighted in yellow using dictionaries developed for this problem (3, 65).

The use of social media for pharmacovigilance is recent, but it has been receiving increasing attention in the last few years. A review paper in 2015 found only 24 studies on the topic, almost evenly divided between manual and automated methods, and concluded that social media was likely useful for postmarketing drug surveillance (66). Another review paper from 2015 (29) found 22 studies and concluded that the utility of social media data analysis for biomedicine is hindered by the difficulty of comparing methods due to the scarcity of publicly available annotated data. This has led to a shared task workshop and the session Social Media Mining for Public Health Monitoring and Surveillance at the 2016 meeting of the Pacific Symposium on Biocomputing (15). Shared tasks have involved the automatic classification of posts mentioning ADRs, the extraction of related terms, and the normalization of standardized ADR lexicons (31). The conference session also attracted studies of social media data for a variety of health-related topics, including: tracking emotion (see Section 3) to detect disease outbreaks (67); pharmacovigilance, including dietary supplement safety (18), and ADR and DDI reports (3); and using machine learning (ML) to predict healthy behavior, such as diet success, using publicly shared fitness data from MyFitnessPal (68), smoking cessation from Twitter data (69), and overall well-being from volunteers’ Facebook data (70). Since this initial event, the shared task and workshop, currently named Social Media Mining for Health Applications, has been held annually and serves to bring together researchers interested in automatic methods for the collection, extraction, representation, analysis, and validation of social media data for health informatics (71, 72).

Before the community was able to analyze well-known social media sites such as Twitter and Facebook, most pharmacovigilance work on mining ADRs from social media had been focused on social interactions in specialized health forums and message boards (73-79). Schatz’s group was one of the first to pursue this angle by using network visualization, natural language processing (NLP), and sentiment analysis (see Section 3) to provide a qualitative ADR analysis of user comments on Yahoo Health Groups. They have shown that it is possible to visualize and monitor drugs and their ADRs in postmarketing (73), as well as to track patient sentiment regarding particular drugs over time (80).

Around the same time, Gonzalez’s group created an ADR-focused lexicon and manually annotated a corpus of comments in support of a rule-based, lexical matching system developed to analyze user comments on DailyStrength (https://www.dailystrength.org/; a health-focused site where users discuss personal experiences with drugs) and demonstrated that comments contain useful drug safety information (74). Later the group used association rule mining to automatically extract ADRs from user comments in the same platform (75), other supervised classifiers to predict if individual comments contained ADRs, and a probabilistic model to infer if the DailyStrength footprint of such predicted ADRs for a given drug were likely to indicate a public health red flag (79).

Subsequently, Benton et al. (76) used co-occurence statistics of drug-adverse effect pairs present in breast cancer message boards and compared them to drug labels of four different drugs. They found that 75–80% of these ADRs were documented on drug labels, while the rest were previously unidentified ADRs for the same drugs. Casting the extraction of (unreported) drug–event pairs in ADRs as a sequence labeling problem, Sampathkumar et al. (77) used a hidden Markov model on patient feedback data from Medications.com that had been automatically annotated using dictionaries of drug names, side effects, and interaction terms.

Several text mining and ML pipelines, as well as annotated corpora and lexica, were quickly developed for extraction and prediction of ADRs from various health forums and message boards. C. Yang et al. (81) used association mining and proportional reporting ratios to show that ADRs can be extracted from MedHelp (https://www.medhelp.org/) user comments; this study was conducted for a small set of five known ADRs (via FDA alerts) involving ten drugs. For the same platform, M. Yang et al. (82) used semisupervised text classification to filter comments likely to contain ADRs in support of an early warning pharmacovigilance system that they tested successfully, albeit with only three drugs associated with more than 500 discussion threads. Yates et al. (78) retrieved ADRs associated with breast cancer drugs by mining user reviews from askapatient.com, drugs.com, and drugratingz.com and produced the ADRTrace medical term synonym set.

Extraction and prediction of ADRs from social media is challenging, especially because of inconsistency in the language used to report ADRs by different social groups, settings, and medical conditions (71). Indeed, various types of evidence exist in scientific publications (e.g., in vitro, in vivo, clinical) and social media (e.g., short sentences on Twitter, long comments on Instagram) to report ADRs and DDIs. To deal with this problem, data scientists in the field use both manual and automatic methods. The former involve manual curation by experts for each context, leading to the development of context-specific lexica and corpora, such as scientific literature reporting pharmacokinetics (83, 84) or pharmacogenetics (85, 86) studies, tweets mentioning medication intake (87), or Instagram user timelines annotated with standardized drug names and symptoms (3). There is also a corpus for comparative pharmacovigilance comprising 1,000 tweets and 1,000 PubMed sentences, with entities such as drugs, diseases, and symptoms (88). Such corpora are very useful for training automatic methods to identify pharmacological relevance in both social media and scientific literature.

Automatic methods to deal with language inconsistency include automatic topic modeling and word embedding techniques that cluster similar terms according to their co-occurrence patterns with other terms (89), typically implemented with spectral methods such as singular value decomposition (SVD) (90). More recently, word embeddings using neural networks, such as Word2vec (91), have shown much promise in obtaining high-quality word similarity spaces for biomedical text (92) and drug lexicons for social media analysis (93). Interestingly, SVD provides a linear approximation of, and insight into, what neural networks do in each layer (94) and a fast method to train them (95).

An example of automatic methods to deal with language inconsistency in pharmacovigilance for social media is ADRMine (96). It uses conditional random fields—a supervised sequence labeling classifier—to extract ADR mentions from tweets. The performance of the system is greatly enhanced by preprocessing posts (from Twitter and DailyStrength) for term similarity features using unsupervised word embeddings obtained via deep learning. Similarly, Word2vec embedding has been shown to increase the performance of automatic estimation of ADR rates of ten popular psychiatric drugs from Twitter, Reddit, and Livejournal data (97), in comparison to the rates of ADRs for these drugs in the SIDER database (98). Interestingly, the lexicon derived by Word2vec leverages variants of ADR terms to deal with language inconsistency. A drawback of using deep learning methods, however, is the need for large training corpora, which limits the applicability to very commonly prescribed and discussed drugs and ADRs.

Most work using social media data for pharmcovigilance has focused on detecting signals for single drugs and their ADRs, although a few groups have studied DDI phenomena as well. Yang & Yang analyzed data from the patient discussion boards MedHelp, PatientsLikeMe, and DailyStrength and focused on 13 drugs and three DDI pairs (28). The study used association mining and DrugBank as a validation database (gold standard) with good results. From these datasets, the same group later built heterogeneous networks, where nodes represented such entities as “users,” “drugs,” or “ADR,” while edges signified “cause” or “treatment.” They went on to show that network motifs were effective in predicting DDIs for an expanded set of 23 drugs, using logistic regression as a link prediction classifier (99).

Soon after, Correia et al. (3) were the first to study DDI phenomena from all available posts on the popular social media site Instagram. The group focused on seven drugs known to treat depression, collected a large dataset of more than 5 million posts, and analyzed a population of almost 7 thousand users. Their study demonstrated the ability to analyze large cohorts of interest on popular social media sites. They also used a (heterogeneous) network science approach to produce a network of more than 600 drug, symptom, and natural product (NP) entities to monitor—via the web tool SyMPToM (Social Media Public healTh Monitoring; https://symptom.sice.indiana.edu/)—individuals and groups of patients, ADRs, DDIs, and conditions of interest. The top predicted links were validated against Drugbank and showed that the network approach allows for the identification and characterization of specific subcohorts (e.g., psoriasis patients and eating disorder groups) of relevance in the study of depression. Later on, the group expanded their work to include other epilepsy and opioid drugs, as well as analysis of Twitter data (65).

Recently, due to the opioid epidemic afflicting the United States, there has been an increased interest in using social media data to understand drug abuse (100). Several studies have analyzed licit (101) (chiefly alcohol), illicit (102) (e.g., cocaine and marijuana), and controlled substances (31, 103) (e.g., opioids) in diverse social media sites. Results are encouraging. For instance, analysis of Twitter data showed that geographical activity of posts mentioning prescription opioid misuse strongly correlates with official government estimates (104), and deep learning methods can be used to predict opiate relapse using Reddit data (105). However, an older study that considered both questionnaires and Facebook data on five behavioral categories—including smoking, drinking, and illicit drug use—reported no significant correlation between respondents’ Facebook profiles and illicit drug use (106). Analysis of such data on Internet health forums has also shown promise. Since these forums are often anonymous, open discussion about drug abuse may be more forthcoming. One study about the drug buprenorphine, a semisynthetic opioid effective in the treatment of opioid dependence, uncovered qualitative observations of public health interest such as increased discussion over time, perspectives on its utility, and reports of concomitant use with illicit drugs, which poses a significant health risk (107).

Social media data could also be useful in the study of the use, potential interactions, and effects of NPs and alternative medicines, including cannabis. Sales of NPs have nearly tripled over the past 20 years since passage of the Dietary Supplement Health and Education Act (108). Based on the general perception that “natural” means safe, the lay public often turn to NPs without discussing them with their healthcare practitioners (109). Consequently, patients frequently take NPs in conjunction with conventional medications, potentially triggering NP–drug interactions. The pharmacology of such products constitutes an array of ADRs and DDIs that have been very poorly explored by biomedical research (108). This is, thus, an arena where social media mining could provide important novel discoveries, early warnings, and insights, particularly with the possibility of studying large cohorts longitudinally (35). A preliminary study with NP and drug name dictionaries showed that it is possible to study their concomitant use longitudinally on Instagram and characterize associated symptoms with heterogeneous knowledge networks (3, 65).

3. CHARACTERIZING INDIVIDUAL AND COLLECTIVE PSYCHOLOGICAL WELL-BEING

The psychological and social well-being of individuals and populations is important, complex, and profoundly involved in shaping overall health-related phenomena. Scalable methodologies to gauge the changing mood of large populations from social media data—using NLP, sentiment analysis, ML, spectral methods, etc.—can help identify early warning indicators of lowered emotional resilience and potential health tipping points (e.g., the onset of mental disorders) for both epidemiological and precision health studies.

The brain is a complex system whose dynamics and function are shaped by the interactions of many components. Such systems can undergo critical transitions (CTs) (110), i.e., rapid and unexpected changes from one stable state to another, that are difficult to reverse. CTs provide a powerful framework with which to understand and model mental health and its relation to the use of pharmaceuticals and other substances. For instance, recent clinical longitudinal studies have provided compelling evidence that psychological CTs, or tipping points, do occur in human psychological mood states. In particular, they are observed in the development of clinical depression and are preceded by measures of critical slowing down, such as increased autocorrelation and variance, as well as other common antecedents, that yield useful early-warning indicators of pending CTs (110, 111).

Social media data are a natural source of high-resolution, large-scale, longitudinal, introspective, and behavioral data to study, monitor, and even potentially intervene before CTs occur, avoiding significant hysteresis to return the system to a desirable state. For each individual social media user we can infer several important social and ethnographic factors from their online parameters (e.g., their location, language, and sex), as well as important risk information from their statements with respect to health-risk behavior and addiction and from their friendship and “follow” ties. In particular, it is now possible to widely track the evolving psychological mood state of social media users over extended periods of time along several relevant psychological dimensions (35, 112, 113). Indeed, social media indicators have been shown to predict the onset of depression (4, 33, 34, 114). Putting one’s own feelings into words on Twitter—also known as affect labeling—can sharply reverse negative emotions, demonstrating the attenuating effects of online affect labeling on emotional dynamics and its possible use as a mood regulation strategy (14) (see Figure 2). Measuring individual and collective sentiment from social media enables the design of actionable intervention strategies to alert individuals and communities to prevent the onset of mental health issues and health risk behavior [e.g., sexual activity (35)], especially in underserved or stigmatized populations (13) (see Section 4).

Figure 2.

(a) An example tweet with its average ANEW (115) scores for arousal, dominance, and valence dimensions. Only words found in the ANEW dictionary were matched to their score. (b) A mood histogram time series showing the per-day distribution of ANEW valence scores for a cohort of Twitter users who self-reported being diagnosed with depression (116). (c) A mean-centered time series of ANEW valence scores for a cohort of Twitter users who stated that they were having a strong emotion on Twitter. Scores are shown for 1-min increments, smoothed by a 10-min rolling average, used to study (14) the effects of affect labeling on Twitter, i.e., the act of putting one’s feeling into words, in this case by stating “I feel ” in a tweet followed by a set of words that denote a strong emotion. Time t = 0 h (red dashed line) is the time at which the affect labeling tweet was posted for each person in the cohort. (d) Average LIWC (117) functional word count of the Facebook posts of a subject from a cohort of patients who died of SUDEP whose behavior on Facebook was studied after their death. This young patient, like several others in the cohort, showed an increase in functional words before SUDEP. Functional words are pronouns, prepositions, articles, conjunctions, auxiliary verbs, and a few other categories understood to indicate emotional states and other individual differences. Abbreviations: 50p, 50th percentile; ANEW, Affective Norms for English Words; CI, confidence interval; LIWC, Linguistic Inquiry and Word Count; SUDEP, sudden unexpected death in epilepsy.

The term “sentiment analysis” refers to a set of computational techniques that are used to measure the opinions, sentiments, evaluations, appraisals, attitudes, and emotions that people express in natural language. This sentiment can be about entities such as products, services, organizations, individuals, issues, events, topics, and their respective attributes, but may also include self-referential elements (117, 118). Sentiment analysis is also broadly defined to include the computational treatment of opinion, mood, and subjectivity in text (119). The earliest studies of online sentiment relied on explicit user-defined features such as labels, ratings, reviews, etc. that were recorded as metadata to the text (118, 119). However, those features are not available for most online texts, including social media posts where health-related indicators need to be inferred from unstructured, unlabeled text. Indeed, the fundamental assumption of sentiment analysis is that individual- and population-level emotions are observable from unstructured written text.

Different methodological approaches have therefore been developed to extract sentiment indicators from text. Some methods use NLP and rely on the detection of word constructs (n-grams) in text to extract sentiment indicators with respect to an entity (120). Other techniques classify text into positive or negative mood classes using ML algorithms applied to annotated training sets, such as support vector machines (119) or naïve Bayes classifiers (112). Frequently, however, very good results are obtained with lexicon matching (119, 121, 122), a method that uses word lists (lexicons or dictionaries) of terms that are preannotated with sentiment values assigned by human subjects. Lexicons of sentiment-annotated terms are obtained via a variety of methods such as expert curation and consensus, population surveys, and automatic feature construction and selection in classification tasks (115, 118, 119, 121, 123-126). This approach is particularly useful when reliability over individual text segments is less important than scalability over large-scale datasets, as is the case for social media data.

Many sentiment lexicons focus on a single dimension of measured affect, such as negative or positive valence (roughly, happiness). Such lexicons include The General Inquirer (127), ANEW (Affective Norms for English Words) (115, 121) along with its several extensions and translations (128), GPOMS (Google Profile of Mood States) (10, 126), LabMT (124), SentiWordNet (122), LIWC (Lingustic Inquiry and Word Count) (117), VADER (Valence-Aware Dictionary and Sentiment Reasoner) (129), and OpinionFinder (130). See the sidebar titled Sentiment Analysis Tools for more details about these tools and the categories or dimensions of sentiment that they aim to capture. Many more sentiment analysis tools exist, with extensive reviews found in References 131 and 132.

These text analysis approaches, combined with large-scale social media data, have enabled the study of temporal patterns in the mood of populations at societal and global levels (35, 138). This includes studies of the changing features of language over time and geography (121, 125).

Because collective mood estimations are derived from tweets from large and diverse populations, the resulting distributions of sentiment values can contain distinct and informative components. However, many analyses of collective mood rely on average or median sentiment over time, which obscures this important information. Spectral methods such as SVD (90) have been effective in (a) removing the base sentiment contribution attributable to regular language use and (b) extracting sentiment components associated with specific phenomena of interest, e.g., moods correlated with increased interest in sex (35) or depression (116). These so-called eigenmoods are components that explain a significant proportion of the variation of sentiment in time series data instead of the average distribution of sentiment values that reflect prevailing language use. As such, they allow for more fine-grained assessments of individual- and population-level emotions associated with health behaviors of interest (116).

The different emotional dimensions of each sentiment analysis tool have been used for specific problems relevant to health and well-being. The authors of LIWC demonstrated how its various indicators are useful to study the relation between language and a wide array of psychological problems (134). Underlying psychological states were shown to be revealed by various LIWC indicators, including the increased use of first-person singular pronouns to describe pain or trauma, the use of verb tenses to describe the immediacy of an experience, the use of first-person plural pronouns to denote higher social status, and prepositions and conjunctions used as proxies for thought complexity, among other examples, all of which enable the measurement of individual differences. For instance, textual features from the speech of student self-introductions measured by LIWC and analyzed with principal component analysis, were shown to be good predictors of overall academic performance. In particular, the use of commas, quotes, and negative affect were positively correlated with final performance, while the use of the present tense, first-person singular, and words from home, eating, and drinking categories were negatively correlated (139). LIWC has also been useful in classifying positive versus negative affect of dream reports (140) and completed versus noncompleted suicide attempts from suicide notes (141), as well as in measuring mood shifts and trends in large populations—e.g., feelings of sadness, anxiety, and anger—during extreme events like the September 11 World Trade Center attacks (142) or hurricanes (143). Diurnal and seasonal rhythms measured from Twitter data were found to be correlated with positive and negative sentiment as measured by LIWC. These measures thus revealed a change from positive to negative sentiment in the morning, as well as increased positive sentiment in days with more hours of daylight (125). Similarly, an analysis of more than 800 million Twitter posts for circadian mood variations further decomposed negative mood into anger, sadness, and fatigue, finding that fatigue follows an inverse pattern to the known circadian variation of concentrations of plasma cortisol—a hormone known to affect mood (144).

Many other sentiment analysis tools make use of lexicons. The ANEW lexicon (115, 128) consists of thousands of English words that have been rated by human subjects on three dimensions: valence, arousal, and dominance. This allows text sentiment to be analyzed along distinct emotional dimensions. ANEW was used to show that the happiness of blogs steadily increased from 2005 to 2009, exhibiting a surprising increase in happiness with blogger age, but a decrease with distance from the Earth’s equator (121). Twitter happiness was also shown to follow a cycle that peaks on the weekend and bottoms out mid-week (121). The equally extensive LabMT lexicon was used to demonstrate that sentiment measurements are robust to tuning parameters and the removal of neutral terms (124). Indeed, a comparison of different sentiment analysis tools and their performance on several corpora, including LabMT, ANEW, and LIWC, found that these lexical tools tend to agree on positive and negative terms, but with notable differences in performance (131). Some lexicons are dedicated to specific application areas, e.g., subjective states (130), whereas others are geared toward general applicability. In general, lexical tools were found to perform well only if the sentiment lexicon covers a large enough portion of the word frequency in a given text and its terms are scored on a continuous scale (132).

Social media data are also useful when sentiment analysis is applied to measure and address public health problems. Qualitative content analysis of sentiment (not using automatic sentiment analysis tools) on websites and discussion forums such as RateMDs.com revealed a positivity bias in reviews of doctors (145) and that positive reviews are associated with surgeons who have a high volume of procedures (146). Similar qualitative content analysis applied to Twitter content found mostly positive views of marijuana (147) with self-reports of personal use increasing when marijuana was legalized in two states (148). Most early sentiment studies of the relevance of social media for public health studies are based on qualitative, manual analysis, but there has been increased interest in large-scale, automatic studies. Using a custom sentiment analysis tool based on text classification (trained on annotated samples), Salath & Khandelwal (36) studied dispositions toward flu vaccination on Twitter. They found that information flows more often between users who share the same sentiments and that most communities are dominated by either positive or negative sentiments toward a novel vaccine (homophily) (36). Unfortunately for public health campaigns, they also found that negative sentiment toward vaccines spreads more easily than positive sentiment in social networks (37).

Choudhury et al. (4) have shown that sentiment analysis tools like ANEW and LIWC are useful for analyzing the sentiment of tweets related to depression by building a large crowd-sourced corpus of tweets from individuals diagnosed with clinical depression (based on a standard psychometric instrument). They also introduced a social media depression index to characterize levels of depression in populations and demonstrated that its predictions correlate with geographic, demographic, and seasonal patterns of depression reported by the Centers for Disease Control and Prevention (CDC). In addition to increased negative affect, onset of depression is also found to be correlated with a decrease in social activity, stronger clustering of social interactions, heightened relational and medicinal concerns, and greater expression of religious involvement (34).

Sentiment analysis of social media data was shown to help differentiate people on Twitter with posttraumatic stress disorder, depression, bipolar disorder, and seasonal affective disorder from control groups (149). This work also identified language and sentiment variables associated with each of the conditions. A similar study found that it is possible to identify linguistic and sentiment markers of schizophrenia on Twitter (150). Given that CTs in mental disease are likely associated with mood changes over time that can be captured by statistical parameters, like autocorrelation and variance (114), the multidimensional, large-scale data that can be extracted from social media, including sentiment, are likely to be of much use in the years to come.

ML methods such as deep learning have been used to accurately classify social media posts according to associated mental conditions (32). This has raised the possibility of characterizing a range of mental health conditions. The approach is still relatively new and most findings are preliminary (151), but it has stimulated an important discussion on the ethics of generating predictions about underlying conditions while also respecting health-related information privacy concerns.

4. OTHER PROMISING APPLICATIONS

Social media data have been used to study a wide range of other health-related problems and have yielded promising outcomes, especially when combined with other data sources. In disaster and crisis informatics, when combined with physical sensor data, signals from social media have been useful in forecasting next-day smog-related health hazards (152). When used effectively by credible sources like emergency response teams, social media have also been used to mitigate community anxiety and the propagation of misinformation and rumors during and after environmental disasters (153).

In epidemiology, social media data have been useful in predicting disease outbreaks such as influenza (11, 23), cholera (24), and Zika (25). In the 2015–2016 Latin American outbreak of Zika, McGough et al. (25) used a dataset that combined Google searches and Twitter data to produce predictions of weekly suspected cases up to three weeks in advance of official publication. Such predictions have often been made based on correlations with the use of certain language, such as keywords or even emojis (154), or made indirectly through the use sentiment analysis tools (118). For instance, it has been shown that general Twitter sentiment about vaccines correlates with CDC estimates of vaccination rates by region (36) (see Section 3). Another study has shown that higher rates of tweets containing future-oriented language (e.g., “will,” “gonna”) are predictive of counties with lower HIV prevalence (26), demonstrating that social media may provide an inexpensive, real-time surveillance of disease outbreaks.

Social media research has also shown promise in efforts to combat stigma, offering a unique means by which to improve outcomes, benefiting healthcare providers and the public alike (155-157). Anti-stigma advocates and government organizations already have well-developed presences on Internet discussion boards and on social media for dozens of health conditions, including obesity/body issues (158) and HIV (159), which help raise awareness of health-related issues, including organ donations (160). Anti-stigma efforts around epilepsy, for example, include TalkAboutIt.org, a collaboration between actor Greg Grunberg and the Epilepsy Foundation (161, 162), and Out of the Shadows, a joint international project among the World Health Organization, the International League Against Epilepsy, and the International Bureau for Epilepsy. Efforts generally center around education about the disease, increasing awareness of epilepsy as a treatable brain disorder, and raising public acceptability of epilepsy. While these efforts do not utilize data science per se, engagement with social media platforms allows for data collection for future analysis of stigma in health.

There is not much data on the efficacy and long-term success of such anti-stigma efforts for epilepsy (163) or mental health disorders (164), although what exists has provided important insights. In their review, Patel et al. (165) have shown the benefits of social media anti-stigma efforts, with 48% of studies indicating positive results, 45% reporting results that were undefined or neutral, and 7% reporting potentially harmful effects. Researchers accessing Twitter have found 41% of epilepsy-related tweets to be derogatory (166). An analysis of the top ten epilepsy-related videos on Youtube has revealed that real-life or lived-experience videos garner the most hits, comments, and empathetic scores but provide little information. Videos with important health information, in contrast, have received only neutral or negative empathy scores (15 5). As contributing factors, concerns about privacy and the reactions of others limit respondents’ willingness to access and engage with content on a website (167). In one of the only network-based studies conducted so far, Silenzio et al. (168) found by mapping social network interactions that some individuals on social media may be more important to the spread of anti-stigma interventions. Additional research in this area is clearly needed.

The study of health-related issues around human sexuality can also be improved by analysis of web search, social media discourse, and health forum data, especially on those platforms that provide anonymity such as Reddit (45). For instance, web search and Twitter data have been instrumental in clarifying competing hypotheses about the cyclic sexual and reproductive behavior of human populations. Analysis of global data has suggested that rather than an evolutionary adaptation to the annual solar cycle, observed birth cycles are likely a cultural phenomenon, since characteristic emotions around major cultural and religious celebrations (measured by sentiment analysis) correlate with interest in sex [measured by Google search data and birth records (35)].

On the medical side of human reproduction issues, pregnant women have frequently turned to the Internet and social media for reassurance on the normalcy of their pregnancies and to gather information on normal pregnancy symptoms, pregnancy complications, birth, and labor (169). For first-time mothers in particular, social media platforms have appeared to be the preferred mechanisms for obtaining important information during the antepartum and postpartum periods (170). Posting status updates and photos on social media appears to have provided pregnant women with a sense of connection with their peers, as well as with their own unborn babies (169, 171). Considering, in addition, the numbers of legal and illegal drug users on social media, as described above, social media platforms appear to be untapped sources of large-scale data on underreported population-level risk for neonatal and related conditions, such as neonatal abstinence syndrome. Social media signals may be effective resources to model the pharmacological, phenotypical, and psychosocial markers associated with drug use during pregnancy, and may lead to better early-problem warnings and prevention strategies.

Other measures that are known to correlate with health outcomes have also been investigated. For instance, social media deviations in diurnal rhythms, mobility patterns, and communication styles across regions have been included in a model that produces an accurate reconstruction of regional unemployment incidence (172). Additionally, the potential to use social media to predict severe health outcomes in epilepsy is preliminary but promising. Sudden unexpected death in epilepsy (SUDEP), for example, remains a leading cause of death in people with epilepsy. A small study of the Facebook timelines of people who died in this way was conducted to identify potential behavioral signs preceding SUDEP and has suggested that prior to dying a majority of the subjects wrote more text than they had previously on social media (see Figure 2).

5. LIMITATIONS

Social media can yield useful healthcare information, but there are inherent limitations to its use for biomedical applications. On the positive side, because analysis takes place after the data are recorded, social media analysis in general avoids experimenter and social conformity bias. Social media data constitute a type of real-world data (173) that allow for very large-scale population samples that surpass those of traditional social science and clinical trial approaches by several orders of magnitude. Indeed, Twitter offers strong opportunities for academic research given its public nature, real-time communication, and user population that approaches significant pluralities of the world’s population. This is also the case for other social media platforms such as Facebook, Instagram, and Reddit. However, social media data frequently lack demographic indicators and ground truth, possibly resulting in biased or poorly representative samples—particularly when compared to the precisely defined inclusion and exclusion criteria of randomized controlled trials (RCTs) (173). In this section we provide a short overview of the literature related to the challenges of deriving valid and reliable indicators of human behavior from social media data and how these challenges can be mitigated.

In spite of scale, social media data generally entail self-selected samples, since subjects are free to choose when to participate and what content to submit. This bias is compounded by a mix of access restrictions imposed by social media platforms (174). As a result, researchers are prone to use so-called convenience samples, i.e., social media datasets that are, due to standardization efforts, more widespread, accessible, and convenient to use, although potentially not representative of the wider population. Combined, these biases may lead to samples that do not validly or completely represent human behavior and diversity

Social media content may also be subject to lexical bias (123) that could cause sentiment data to overrepresent positive sentiment. In addition, platform-specific factors may alter user behavior (174, 175) and lead to bias in subsequent data analysis. Indeed, users may be encouraged to engage in profile and reputation management by establishing different online personas to highlight their individuality and qualities that are perceived as desirable (176).

Privacy issues and algorithmic bias may also lead to mischaracterization of human factors. The behavior of most social media users is profoundly shaped by interface designs and, increasingly, algorithmic factors, e.g., the use of ML services for recommendations of social relations and relevant content. Nonhuman participants such as bots are, furthermore, widespread in some social media sites, e.g., Twitter. Moreover, exogenous events such as polarized elections may trigger individual and global sentiment changes, discourse polarization, and temporary deviations from baseline social linkage dynamics. The particular social-economic-political context in which social media data are recorded therefore plays an important role in analysis. Given these potential population biases, mining social media for healthcare information relevant to the broader human population requires a careful consideration of the multilevel complexity of human health (55), in which social and behavioral contexts play a critical role (177).

Perhaps one of the most important issues with social media mining is the difficulty of establishing sample validity and precise inclusion and exclusion criteria. Primarily, two sources of bias impact harvested social media data: sampling bias and algorithm bias. Sampling bias means that researchers cannot treat sampled social media data (e.g., a sample of 1% of tweets) as a truly representative and random sampling of the human population. This affects efforts to build valid cohorts and make generalizations from analyses (174). Samples cannot be balanced because of the lack of ground truth with respect to user demographics. Furthermore, the demographics of social media sites can vary broadly. In a survey of social media usage among American adults, 43% of women said they have used Instagram at least once, while for men this number was only 31%. Similarly, Hispanics appear underrepresented on LinkedIn—only 16% said they have used the platform as compared to 24% of Whites and 28% of Blacks. At the same time, Hispanics appear to be the largest demographic on WhatsApp—42% as compared to 24% of Whites and 13% of Blacks (178).

Sampling bias can be accentuated when subcohorts of social media users are used to draw geographical inferences, e.g., when particular key terms are used to infer location. Such subsamples may vary considerably in the degree to which they represent an unbiased sample. Future research using social media data must benefit from the large-scale nature of this real-world data, while specifying more precise inclusion and exclusion criteria, as used in RCTs, to avoid sample biases (173). Getting to that point requires the ability to stratify social media user cohorts using more fine-tuned ML, as well as via greater collaboration with and openness from social media platform providers. It is encouraging that Twitter data have been shown to match census and mobile phone data in geographical grids down to a square-kilometer resolution (179). Indeed, ML methods can be used on Twitter data to automatically track the incidence of health states in a population (180). Moreover, user demographics such as age and gender can be estimated from user content with reasonable accuracy (14).

In addition to sample bias, it is important to be aware of algorithmic biases that result from interface design, policies, and incentives associated with social media platforms. Since company revenues are tied to targeted advertisement, social media algorithms are tailored for navigation retention and profile building. These algorithms are highly dynamic, proprietary, and secret, all of which have consequences for research reproducibility (175). Most researchers, like users, are largely unaware of how platforms filter timelines and other user information (181). Therefore, greater openness on the part of social media companies, perhaps encouraged or mandated by public policy, is needed to increase the utility of social media data for biomedicine.

In addition to sample and algorithmic biases, sentiment analysis can be manipulated by third parties through the injection of tweets (174), i.e., the deliberate insertion of tweets containing specific words known to affect sentiment tools, e.g., to boost general sentiment during a political debate. These efforts can be difficult to detect and mitigate since they affect the sample a priori, before a researcher can apply efforts to unbias their sample and address sample validity. Indeed the extraction of emotional and social indicators from social media is fraught with difficulty. Users may indirectly disclose mood states, sentiment, health behavior, and diet, but rarely do so explicitly (14). Social media users furthermore favor an idiomatic style and vernacular that is difficult to analyze with traditional NLP tools and supervised ML algorithms. Applications of the latter are hampered by the lack of vetted ground truth datasets and the highly dynamic nature of underlying emotional processes. Additional difficulties in analyzing social media discourse include subjective opinion, the use of sarcasm (particularly toward the effects of specific drugs), and the polarity of a sentiment-laden word or phrase in context (119, 129). For instance, social media users may use the term “prozac” in a variety of idiosyncratic ways, but not necessarily because they are actually administering the drug.

Users revealing sensitive personal information about others and information pertinent to their social relationships raises serious privacy concerns. Indeed, data from eight to nine social media relations of an individual are sufficient to predict that individuals characteristics just as well as their own information (182). In other words, privacy concerns are not just a matter of what users reveal about themselves but also a matter of what their social relations (unwittingly) reveal about them. Some users are aware of this phenomenon, which lowers their motivation and willingness to participate in studies using social media data (167).

Another limitation, finally, is the danger of overfitting in subsequent analysis. Because of data availability and privacy issues, information on specific cohorts is derived from indicators that are in turn derived from the content they generate. This will favor certain content and cohorts, possibly leading to models that overfit the data and generalize poorly (22, 183).

6. CONCLUSION

The studies reviewed in Section 2 show that social media users discuss a wide variety of personal and medical issues on social media platforms, e.g., their medical conditions, prognoses, medications, treatments, and quality of life, including improvements and adverse effects they experience from pharmacological treatments (3, 27, 52, 184). This collective discourse in turn can be monitored for early warnings of potential ADRs and to identify and characterize underreported, population-level pathology associated with therapies and DDIs that are most relevant to patients (3, 15, 27, 29). The new data-enabled modes of pharmacovigilance that social media afford are likely to be particularly relevant for patient-centered management and prevention of chronic diseases (185), such as epilepsy (186) and inflamatory diseases (52), which continue to be the chief healthcare problem in the United States (187). The inclusion of signals from, and engagement with, social media in patient-centered personal health library services that can store, recommend, and display individualized content to users is expected to significantly improve for chronic disease self-management (185), which is known to significantly lower disease severity and the number of unhealthy days and improve quality of life (188).

It is clear that disease prevention is increasingly becoming a matter of mitigating disease risk factors caused by an individual’s lifestyle and decision making, which are subject to a range of cognitive, emotional, and social factors that have until now been difficult to assess with sufficient accuracy and scale. An understanding of the emotional and social factors that contribute to the emergence of public health issues is crucial for efficient mitigation strategies. As we describe in Section 3, there is already a substantial body of literature characterizing psychological well-being, especially by measuring individual and collective sentiment and other social interactions online. These methods have been particularly effective when used in combination with other sources of health and human behavior data, from physical sensors, mobility patterns, EHRs, and more precise physiological data.

The methodologies we cover also fall in the area of computational social science, which is presently focused on establishing the methodological framework to monitor societal phenomena from large-scale social media data—the abovementioned social macroscopes. For this methodology to be relevant in the prevention of disease and improvement of public health, researchers need to move from descriptive inductive modes of analysis to explanatory models with predictions and testable hypotheses. In particular, researchers need to establish social media not just as a tool for observation, but also as the foundation for explanatory models of the generative factors in health behavior and outcomes, of the type that computational and complexity sciences are already producing, e.g., in molecular and organismal biology (189).

There is reason to be optimistic about our ability to reach such predictive explanatory models since we know from psychological research that emotions play a significant role in human decision making (190). Behavioral finance in particular, for example, has provided evidence that financial decisions are significantly driven by emotion and mood (191). Hence it is reasonable to assume that online mood and sentiment, as well as all social media analysis we review, may be used to predict health behaviors and can therefore be used to predict individual and societal health outcomes.

The literature reviewed also points to a newfound ability to use social media data for improved well-being of small, specific cohorts and even individuals using precise characterizations and interventions. These may include pharmacological warnings, patient-centered management of chronic disease, and mental disorder assistance. For instance, donated timelines from individuals at risk of suicide can help ML models recognize early warning symptoms of depression and suicide (192).

Despite the proven importance to the specific goal of improving human health, social media data have been increasingly difficult to collect. Only a few social media data sources remain open for scientists. Many previously accessible sites are now almost entirely sealed from researchers, which is surprising given that the data are generated by and for its users, not the platforms, which mostly serve a mediating function. These limitations explain why most of the work reported has focused on Twitter, which remains open for data analysis. Nonetheless, other social networks have been shown to be useful for biomedicine, including Facebook (12), Instagram (3), Reddit (45-47), and even Youtube (48, 49).

It is possible that government policies may be leveraged to ensure accessibility to these important data sources, which could be considered a public good to be regulated much like publicly funded scientific publication data (193). This would help improve the sample and algorithmic limitations discussed in Section 5, allowing this large-scale, real-world data to better identify health factors that more expensive clinical trials cannot due to their smaller scale and cost. We intend for our review to contribute to establishing the importance of social media data for biomedical research and demonstrates the need to make such data more accessible in general to scientific research.

SENTIMENT ANALYSIS TOOLS.

The General Inquirer (127) is a tool to organize non-numerical data and to tag words in a text across various categories. The system started as a general-purpose tool with a dictionary of categories for the 3,000 most common English words and a few hundred words of interest to behavioral scientists. It has since grown to include the Harvard IV-4 and Lasswell content analysis dictionaries and has 198 categories.

ANEW (Affective Norms for English Words) includes ratings from 1 to 9 for 1,034 words along three mood dimensions: valence (unhappy to happy), arousal (calm to excited), and dominance (controlled to in control). These ratings are based on a nine-point, Likert-like scale and were collected from surveys given to psychology undergraduates (115). ANEW is used as a basis for several new dictionaries, including an extended version with nearly 14,000 words (128) and translations to several languages.

GPOMS (Google Profile of Mood States) is an extension of POMS (Profile of Mood States), a questionnaire of self-reported Likert scale questions measuring six underlying dimensions of mood: tension or anxiety; depression or dejection; anger or hostility; vigor or activity; fatigue or inertia; and confusion or bewilderment (133). GPOMS is used to translate this questionnaire into a dictionary suitable for sentiment analysis of large-scale social media data. Using word co-occurrences in Google’s’ n-gram corpus, GPOMS extends the original 72 POMS terms to a dictionary of 964 terms that correspond to moods across six categories: calm, alert, sure, vital, kind, and happy (126). LabMT is a dictionary used by Amazon’s Mechanical Turk to send out ANEW-like surveys that rank thousands of words on a nine-point scale from sad to happy and collects at least 50 ratings per word. Initially, LabMT comprised 10,222 English words found by merging the 5,000 most used words in each of four corpora: Google Books, Twitter, music lyrics, and The New York Times (124). This has since been extended to include 10 languages with about 10,000 words each collected across 24 corpora (123).

LIWC (Linguistic Inquiry and Word Count; pronounced “Luke”) is a text analysis tool that has been actively supported and widely used since its first public release in 2001 (117, 134). LIWC was developed by several judges through a well-documented procedure that included external validity with psychological studies. The latest version, LIWC2015, has dictionaries containing nearly 6,400 words and produces outputs across about 90 categories including emotion and the use of pronouns, articles, cognitive processes, time focus, personal concerns, and informal language (117).

SentiWordNet (122) is a dictionary with word scores along three dimensions: positive, negative, and objective (neutral). It is built by automatically annotating sets of synonymous words (synsets) from WordNet (135), with additional steps using semisupervised classifiers and a random walk.

VADER (Valence-Aware Dictionary and Sentiment Reasoner) is a tool for measuring positive or negative text sentiment beyond simple lexicon matching (129). It searches for specific words in a sentence and modifies the associated dictionary-based sentiment scores based on simple rules, such as the presence of exclamations or negations. OpinionFinder (130) is a processing pipeline that tokenizes a document and then uses a series of classifiers trained on different corpora to find subjective statements and speech events. It identifies opinion sources and expressions of sentiment and classifies expressions as positive or negative.

In addition to these tools, other researchers suggest modifications of sentiment scores based on context by using compositional rules to modify sentiment scores from sentence parse trees (136, 137). Extensive reviews of sentiment analysis tools can be found in References 131 and 132.