Ethical issues of collecting, storing, and analyzing geo-referenced tweets for mental health research

Abstract

Spatial approaches to epidemiological research with big social media data provide tremendous opportunities to study the relationship between the socio-ecological context where these data are generated and health indicators of interest. Such research poses a number of ethical challenges, particularly in relation to issues such as privacy, informed consent, data security, and storage. While these issues have received considerable attention by researchers in relation to research for physical health purposes in the past 10 years, there have been few efforts to consider the ethical challenges of conducting mental health research, particularly with geo-referenced social media data. The aim of this article is to identify strengths and limitations of current recommendations to address the specific ethical issues of geo-referenced tweets for mental health research. We contribute to the ongoing debate on the ethical implications of big data research and also provide recommendations to researchers and stakeholders alike on how to tackle them, with a specific focus on the use of geo-referenced data for mental health research purposes. With increasing awareness of data privacy and confidentiality issues (even for non-spatial social media data) it becomes crucial to establish professional standards of conduct so that compliance with ethical standards of conducting research with health-related social media data can be prioritized and easily assessed.

Introduction

Digital epidemiology for mental health research

To date, most epidemiological studies that have involved mining large social media data have focused on the prediction of infectious disease outbreaks, ¹ but there is great potential to provide information on emerging trends in a wide range of both communicable and non-communicable conditions. More recently, researchers have started harvesting social media platforms to retrieve and analyze data related to mental health.^2,3 Social media research aimed at identifying mental health disorders is based on the analysis of naturally occurring exchanges among online users. In this sense, a specific area of data science has focused on Natural Language Processing (NLP) and Machine Learning (ML) methodologies to analyze, detect, and predict symptoms of mental disorders through social media text, ⁴ images, and emoticons, ⁵ as well as social interactions (e.g. through mapping friendship connections). ⁶ In text mining, sentiment analysis (i.e. the classification of a given text as having positive, neutral, or negative emotional valence) is the most popular tool for understanding emotion expression. For example, Coppersmith and colleagues have measured four different mental health disorders (depression, bipolar disorder, post-traumatic stress disorder, and seasonal affective disorder) by applying the Linguistic Inquiry Word Count (LIWC) tool and open-vocabulary analysis in automatically derived samples of Twitter data. ⁷ The study was based on previously observed differences between depressed and control groups via LIWC on Twitter. Depressed users tended to use first person pronouns, negative emotion words, and words expressing anger more frequently. ⁸ Other NLP tools that have been used for research include OpinionFinder, ⁹ SentiStrength,^10,11 and Affective Norms for English Words. ¹² In addition, EMOTIVE and Stresscapes go beyond sentiments and identify basic emotions and stress, respectively.^13,14 Studies showed that more in-depth and fine-grained affective measures/“signals” have some likely predictive value. In particular, research has showed how extending the LIWC variables with emotion variables (using EMOTIVE) improves the predictive models’ performance (on the same kind of Twitter datasets). One study showed how emotions over time have predictive value in classifying likelihood of depression (building on Coppersmith’s approach), ¹⁵ while another closely replicated Coppersmith’s study on predicting the four mental health conditions, showing how emotion variables can improve predictive performance across these. ¹⁶ Such methods have been developed both for monitoring population mental health trends and patterns, and identifying risk factors for individuals. ¹⁷

While more than 50% of the global population now use social media, with highest relative growth of social media usage occurring in low- and middle-income countries in recent years,^18,19 much of research to date using social media for the purpose of mental health has leveraged data from Twitter. Twitter data feature several characteristics that make them particularly appropriate for mental health research. First, the Twitter API is more openly accessible, and tweets typically contain valuable meta-data (such as geo-referenced data) that can provide insight into public sentiments, opinion, and behaviour. ²⁰ Second, Twitter has a strong hashtag culture that makes it easy to find, sort, and collect data. ² Third, with more than 500 million tweets per day and 126 million daily active users, of which 80% use mobile devices, Twitter research can benefit from a large volume of public real-time, sometimes geo-tagged contents.^21,22 Finally, communication on Twitter appears to mirror real world offline trends, for example, performance of movies at the box office, ²³ disaster events, ²⁴ political events/elections, ²⁵ which suggests reliability/validity of the data relative to actual human behaviors, interactions, and events. Specifically, in relation to mental health, studies suggest that this platform is a feasible avenue for investigating mental health-related phenomena. Users seem to have a shared understanding that Twitter allows them to communicate freely about their mental health issues and to receive appropriate support from others with similar experiences. ²⁶ However, studies have also found a substantial portion of communication on Twitter about mental illness is negative or stigmatizing,^27,28 while others have found that mental health tweets can be used to challenge stigma or share information. ²⁹

Adding the spatial dimension to digital epidemiology

Although person, time, and space represent the three core elements of epidemiology, ³⁰ most digital epidemiological studies targeting mental health are location agnostic, missing the opportunity to integrate the spatial dimension into the analysis of mental health phenomena through social media data analysis. ³¹ Geo-social or geo-referenced social media data, that is, social media data that include geographic information, have the potential to foster our understanding of the relationship between the socio-ecological context where data are generated (i.e. place-based social determinants of health) and the given mental health indicators. ³² Furthermore, collecting geographic information on the concentration of specific emotions holds the promise of identifying areas of possible mental health distress to inform timely and targeted prevention and intervention strategies. ²⁴ Only few studies analyzed emotional responses of Twitter users in the context of human-made or large-scale natural disasters over space and time. For example, Gruebner and colleagues investigated the feasibility of a space-time syndromic surveillance system for the emotional consequences of mass traumatic events based on Twitter data generated in the context of the terrorist attacks in Paris in 2015. ³³ They also used geo-referenced Twitter data to extract negative emotions indicating discomfort in New York City (NYC) before, during, and after Superstorm Sandy in 2012, and to assess geographic variation in emotions over time,^34–36 also in the COVID-19 context. ³⁷ Others investigated emotional acute and chronic stress as expressed in geo-referenced, in-situ social media language and their relationship with urban environments. ¹⁴ Han and Wang classified Weibo (a popular social media platform in China that functions in a similar way to Twitter) texts into six topics and nine sentiments related to the 2018 Shouguang flood and analyzed their spatial-temporal characteristics. ³⁸

The collection, storage, and analysis of large datasets of social media data that involve tracking the location of individuals over time and assessing health-related outcomes in relation to shifting socio-ecological settings raise not only significant technical and methodological challenges but also ethical questions, particularly in relation to issues of privacy, informed consent, and data protection. ³⁹ Yet studies using social media data rarely mention ethical concerns and frequently assume that, since the content they are interested in is public, its collection and use does not require ethical review. ⁴⁰ In addition, recommendations aimed at providing practical advice on how to ethically collect, store, and analyze social media data for health-related research purposes are scant.

Therefore, the aim of this article is to review recommendations on how to address the ethical issues of collecting, storing, and analyzing social media data, and highlight their strengths and limitations to address the specific ethical issues of geo-referenced tweets for mental health research.

Strengths and weaknesses of current recommendations addressing the ethical issues of social media research

To identify ways to address the ethical implications of conducting research with (geo-referenced) social media data,^41,42 we purposely identified ethical recommendations for conducting research with social media data following the approach defined in “Internet Research Ethics” of the Stanford Encyclopedia of Philosophy. ⁴³ In the past two decades, an increasing number of research scholars, various professional associations, and non-profit scholarly and scientific agencies have explored ethical concerns of conducting research with Internet and social media data, and drafted statements or guidelines, or addendum to their extant professional standards. For the purposes of this article, we focused on ethical guidelines for Internet research drafted by scholarly associations, as they are likely to have wider recognition among researchers involved in collecting, storing, and analyzing geo-referenced tweets for mental health research. The Stanford Encyclopedia of Philosophy identifies four main associations that drafted guidelines of such type: the Board of Scientific Affairs of the American Psychological Association, which established an Advisory Group on Conducting Research on the Internet in 2001; the American Counseling Association, which revised its Code of Ethics in 2005; the Association of Internet Researchers (AoIR) Ethics Working Group Guidelines; and the National Committee for Research Ethics in the Social Sciences and the Humanities (NESH Norway). From this list, we excluded the guidelines developed by the Norwegian National Committee for Research Ethics as they may have only limited, local recognition, and those proposed by the American Counseling Association, as they constituted a minor addition to their more general Code of Ethics, which does not address the ethics of Internet or social media research as its main focus. Therefore, we selected the “Ethics Recommendations for Internet-mediated Research,” developed by the British Psychological Society (BPS), ⁴⁴ and “Internet Research: Ethical Recommendations 3.0,” developed by the AoIR. ⁴⁵ However, the two documents do not make specific reference to location tracking in social media research. The BPS suggests that the main ethics principles outlined in the Society’s Code of Human Research Ethics, namely respect for autonomy, privacy, and dignity of research participants, scientific integrity, social responsibility, and maximizing benefits and minimizing harm, should also apply to internet-mediated research. ⁴⁴ Furthermore, for each principle, the recommendations highlight issues that may need special consideration in the context of internet-mediated research, including: the public-private domain distinction online; confidentiality and security of online data; procedures for obtaining valid consent; procedures for ensuring withdrawal rights and debriefing; levels of researcher control; and implications for scientific value and potential harm.^42,43

The ethical concerns addressed by the two documents may be summarized into four main issues: (1) users’ privacy expectations and informed consent, (2) data security, (3) data confidentiality, and (4) social responsibility.

Privacy expectations

The recommendations state that collecting data without informed consent from a public online domain is justifiable when people could reasonably expect to be observed by others. ⁴⁴ However, research shows that participants are not always aware that their contributions on social media websites are potentially accessible and that they may have context-specific intentions and expectations regarding how their content is used and who uses it. ⁴⁶ In the case of Twitter, considering that approximately half of all tweets contain personal information about the user generating the content, ⁴⁷ retrieving, analyzing, and storing such data can pose potential threats to users’ privacy and represent a source of rights violation, especially for those who are unaware of the public nature of their information and its harvesting for research purposes. Accessing such information may expose users to different sources of harm (e.g. posting tweets disclosing information about vacations, consuming drugs or alcohol, engaging in reckless behaviors, or having a stigmatizing disease that can put users at risk of harm from their family or other personal relationships, their employers, or insurance companies). ⁴⁸ However, as the chance of violations of anonymity and confidentiality increases, arguments that valid consent is not necessary are weakened. ⁴⁴ Furthermore, the recommendations recognize that participants may provide their consent without having been properly informed of the relevant issues. According to the AoIR, a way of trying to mitigate risks for participants is by reserving the acquisition of informed consent to the dissemination stage of a project, that is, by asking for informed consent from specific subjects for the publication of a quote or other data that might make them and their personal information identifiable. Finally, the guidelines do not recognize that users can lose control over their ability to remove their posts from social media (e.g. by closing an account, or deleting old posts) if this information is retained in a research dataset. For example, a user may withdraw her consent for a piece of information to be public, but cannot remove it from a dataset where this information has already been included, simply because the user is unaware the dataset exists and cannot contact the data owner. This could potentially have major implications for informed consent and for users’ withdrawal from a research study when their preferences over the extent to which “to be public” have changed. To address this issue, it would be ideal to establish a system to regularly assess whether the Twitter users involved in the dataset have changed their privacy settings or limited audience to which the data were made available, particularly before analyses are conducted to answer a new research question. However, this solution may be challenging (especially with regards to resourcing such a continuous task) and careful anonymization might be the only option to mitigate privacy concerns. For example, while a process of running a light-weight rehydration ¹ to establish whether Twitter still returns content for specific tweet IDs could be implemented, it can be time consuming with large datasets. If a new research question/data sharing request is made to a repository, a fairly common practice is that it would be up to the researcher team interested in the dataset to rehydrate the data. Platform companies should also provide more efficient ways for researchers to be able to re-query validity of consent (especially for large datasets).

Data security

To maximize data security, the AoIR recommends applying technological solutions such as encryption, grounding such recommendation in the general principle that online activities should be mediated by an IT Security team with appropriate special responsibility and training. ⁴⁵

Data confidentiality

The BPS recognizes risks relating to the levels of researcher control over confidentiality of data, particularly during the data collection and storage process, highlighting the need for researchers to recognize that it is not possible to uphold complete confidentiality of participants’ personal information gathered online because the networks are not in the control of the researcher.^44,45 Researchers need to think about practices to properly inform participants about the ways their data are stored and transferred, particularly where risks are higher. This could be done by providing information through the platform where data are generated by the users. Furthermore, participants should be informed about the possibilities for breaches of confidentiality through the use of search engines (e.g. by copying and pasting the text portion of a social media post in a web search engine such as Google, it is possible to trace the user who generated this content online) and the accumulation of data from multiple sources.^44,45 The recommendations highlight, that issues of confidentiality breaches may increase the risk of harm for participants, but do not mention how data sharing should occur and what the possible consequences of a sub-optimal handling of the data from a methodological point of view could be. To further reduce the risk of causing privacy harm or social disruptions by third parties who might be granted access to the data for research purposes, we recommend that researchers interested in using the data seek ethical approval for any secondary use of the data. In addition, we recommend that researchers share only agreed portions of the original, full dataset with third parties. For instance, researchers can be provided with a dataset containing the geo-location of tweets and the timestamp and a limited number of variables, so that identification of the users is further limited. When deciding which variables to share, special consideration should be given to different types of variables: for example, content-based emotion or sentiment variables are unlikely to be very revealing with regards to identity, while other tweet/account specific metadata (such as number of followers/followees, account creation date) might be much more dangerous. This would ensure that the project’s intrusion on users’ privacy by accessing their information is justified, and respect for their privacy expectations is maximized and balanced against the progress of the methodology and the promotion of public good interests. ⁴⁹

Social responsibility

The BPS raises several key points in relation to the issue of social responsibility. It recommends considering the extent to which proposed research study procedures and dissemination practices might disrupt/harm specific social groups, and maintaining respect for social structures, as intrusions from researchers into spaces that their users consider private may be perceived as “invasive, unwelcome, and socially irresponsible.” ⁴⁴ Since data extracted and enriched with sentiment analysis might identify users, locate them, or link information that can become a potential source of stigmatization (by, for instance, identifying them as having a mental condition), we recommend researchers to carefully assess whether the Twitter users involved in the dataset intended to make their information public in the light of their privacy settings or limited audience to which the data were made available. Therefore, we recommend researchers to only “rehydrate” those tweets that are available via the Twitter API. This procedure effectively provides for a recent assessment of whether consent had been withdrawn by the user.

Discussion

In this article, we mapped two widely recognized sets of recommendations in the context of analyzing big social media data for public health research. We found that three additional ethical considerations may apply to studies using geo-referenced data for mental health research that the recommendations do not specifically address: (1) an increased risk of identification; (2) increased risk of stigma; and (3) consequences of suboptimal quality of the analysis.

Increased risk of identification

Geo-referenced data offer the possibility to localize and therefore more accurately monitor people over space and time. Studies have shown that, while people generally geotag in uncommon places, consciously and intentionally, during or soon after being in a given place, many Twitter users do so inadvertently, or they geotag more precisely than they think. ⁵⁰ Studies suggest that participants might identify researchers as intrusive in online communities, ⁵¹ and indicate a preference that researchers only use information marked as “public” by the author.^7,52,53 Other studies found that only few users were aware that their tweets could be extracted and analyzed by researchers, with most feeling that this should not be done without their consent. ⁵⁴ A very similar finding is from an exploratory survey with over 100 UK-based (mostly student) respondents. ⁵² Specifically, participants claimed to be generally aware of some analytics occurring, but were unsure as to what data about them was being analyzed and unclear about the actual reach of information they reveal online—its visibility to third parties. ⁵⁵ To address this issue, we recommend following Rivers and Lewis six “TACTICs” for the ethical use of Twitter data that highlight transparency of the research process, protection of the anonymity of authors, respect for authors’ control efforts, focused collection, respect for the context where data are generated and collaboration with an IRB (Institutional Review Board). ⁵⁶ In addition, to ensure transparency on what data is being collected and for which purposes, a registry should be made publicly available that summarizes all research studies using Twitter geo-reference data, and that offers contact information so that questions can be forwarded to PIs and their research teams. This could also be important for clearly stating the research objectives of different studies.

Increased risk of stigma

Due to the stigma associated with mental illness, special considerations may apply to the collection and analysis of geo-referenced big data for mental health research.^57,58 There are ethical implications to using social media data posted by people who may have shared their information while being exposed, for example, during a large-scale disaster or disease outbreak. Therefore, they may not necessarily recognize that they share potentially sensitive data that can be collected and analyzed, either for scientific activities, commercial purposes, or first aid assistance. ⁴⁶ In addition, when mental health issues have been identified through geo-referenced social media data and prompt intervention is warranted, those targeted by the intervention may incur in stigma-related harm for being identified as mentally ill. ⁵⁹ Finally, while Twitter users may have (knowingly or unknowingly) consented to their information being public, they are unlikely to have consented for their information to be analyzed for mental health purposes. It may be a source of distress for some users to know that their posts were collected and analyzed to explore potential presence of mental health symptoms, or more alarmingly, to “label” them with a specific mental illness.

Consequences of suboptimal quality of the analysis

A third issue is the aftermath of the scientific analysis of the geo-referenced data, when its quality is suboptimal. For example, a major issue in social media big data research is that few studies are driven by a priori hypothesis testing, and instead are exploratory, which raises concerns about data dredging. If the scientific work is done poorly (e.g. the study is not rigorously designed, or the analytic techniques are not adequately described for the given purpose), wrong beliefs may come out of the research and may have direct impacts on public health measures (e.g. use of social media for diagnosis of mental health conditions risks labeling individuals who might not actually seek clinical consultation), or the stigmas associated to given populations. Since analyzing big data may lead to the creation of new data, ⁶⁰ this should occur in a context that meets the highest methodological and data protection standards. A data sharing and confidentiality agreement is a formal contract that documents what data are being shared between the project principal investigators and third parties and how the data can be accessed and analyzed. It specifies the intended use of the data, any constraints on the use of the data, and how to ensure data confidentiality and security. However, issues of methodological rigor are not contemplated in such an agreement. Therefore, we recommend including considerations of methodological robustness in the data sharing and confidentiality agreement.

Limitations and conclusions

Some limitations of this perspective article are worth noting. We excluded the regulations and policies and the terms of service (including privacy statements) of the social media used in the research (i.e. Twitter) as well as the scientific literature available on the topic of the ethical, legal, and social implications of big data and social media research. This is because the scope of this article was to target professional, practical recommendations, rather than theoretical frameworks. Therefore, our search is not exhaustive, and may not capture all practical recommendations from professional societies. We also excluded the legal aspects of collecting, storing, and analyzing geo-referenced social media data, which is also an important area that requires increasing attention and emphasis for informing recommendations.

As the field continues to evolve rapidly, with new digital platforms emerging and new applications of digital data, there is a need to revisit existing guidelines on a regular basis to ensure that the concerns highlighted here are adequately addressed and modified accordingly to new risks that are identified. Furthermore, as most of the world’s social media users are located in low- and middle-income countries, scholars involved in the analysis of social media data for mental health purposes in these settings need to reflect on the implications of their work in light of the paucity of regulations, limited role of professional societies governing how social media data is used, relatively fewer efforts aimed at exploring social media communication in different languages and diverse cultural contexts, and significant deficits in providing even basic mental health services. ⁶¹ Here we provided a selection of practical recommendations to researchers interested in the use of geo-referenced social media data for analyzing mental health outcomes, by highlighting strengths and weaknesses of current recommendations.