Abstract
Background
Twitter is an indicator of real-world performance, thus, is an appropriate arena to assess the social consideration and attitudes toward psychosis.
Objective
The aim of this study was to perform a mixed-methods study of the content and key metrics of tweets referring to psychosis in comparison with tweets referring to control diseases (breast cancer, diabetes, Alzheimer, and human immunodeficiency virus).
Methods
Each tweet’s content was rated as nonmedical (NM: testimonies, health care products, solidarity or awareness and misuse) or medical (M: included a reference to the illness’s diagnosis, treatment, prognosis, or prevention). NM tweets were classified as positive or pejorative. We assessed the appropriateness of the medical content. The number of retweets generated and the potential reach and impact of the hashtags analyzed was also investigated.
Results
We analyzed a total of 15,443 tweets: 8055 classified as NM and 7287 as M. Psychosis-related tweets (PRT) had a significantly higher frequency of misuse 33.3% (212/636) vs 1.15% (853/7419; P<.001) and pejorative content 36.2% (231/636) vs 11.33% (840/7419; P<.001). The medical content of the PRT showed the highest scientific appropriateness 100% (391/391) vs 93.66% (6030/6439; P<.001) and had a higher frequency of content about disease prevention. The potential reach and impact of the tweets related to psychosis were low, but they had a high retweet-to-tweet ratio.
Conclusions
We show a reduced number and a different pattern of contents in tweets about psychosis compared with control diseases. PRT showed a predominance of nonmedical content with increased frequencies of misuse and pejorative tone. However, the medical content of PRT showed high scientific appropriateness aimed toward prevention.
Keywords: social stigma, social media, psychosis, breast cancer, HIV, dementia, public opinion, diabetes
Introduction
Psychotic disorders are among the world’s leading causes of disability [1,2]. The estimated lifetime rate of suffering any psychotic disorder is 2 to 3% [3]. The societal and economic burden of schizophrenia is very high [4]. Schizophrenia is associated with at least 10 to 15 years of potential life lost, with no indication of a decline in this trend [5].
Despite many decades of research, the treatment of psychotic disorders remains only partially effective, and their etiology is not fully understood [6,7]. Currently, patients are encouraged to take an active role in the development of an active and meaningful life while growing beyond the misfortune of mental illness [8,9]. The traditional clinical and societal view of schizophrenia is of a debilitating and deteriorating disorder, with a poor outcome [10]. There is evidence of persisting stigma about mental illness that leads to negative stereotyping and to discriminatory behavior toward people with schizophrenia [11]. Stigma may cause affected patients to experience rejection and to feel shame about their condition, reducing their self-esteem and limiting their opportunities [12-14].
Background
In recent years, the internet and social media have become pivotal instruments for sharing knowledge [15]. Accordingly, the internet has radically modified how most people communicate, share, and seek out information regarding health and medical conditions [16,17]. Twitter, one of the most popular and widely used platforms of social media, is currently considered an effective channel of communication [18]. Different players in health and medicine have realized Twitter’s potential for acquiring and distributing medical information [19]. Furthermore, American mainstream media outlets and the general public demonstrate a preferential interest for psychiatric disorders on Twitter [20]. A third of patients with schizophrenia use social networking sites, including Twitter, at least daily [21]. The analysis of distributed tweets is increasingly appreciated in health research [22]. The utilization of Twitter data has enabled researchers to study health-related attitudes toward behaviors and diseases, predict the incidence of both communicable and noncommunicable diseases, or have an insight of patients’ medical experience [23-25]. The utilization of online data for health care purposes has led to the development of an emerging field defined as infodemiology [26].
Moreover, the analysis of tweets about psychiatric disorders is a recent relevant area of study for understanding the sentiments of society, patients, and health players [27-33]. Concerning results have been reported about the trivialization, stigmatization, and mockery of schizophrenia and other psychiatric disorders by Twitter users [34-39]. The areas of medical and nonmedical interest of Twitter users about psychosis spectrum disorders have not been established. The reach and impact of psychosis-related tweets (PRT) remain unknown.
Objectives
The aims of this multidisciplinary study were to investigate the medical knowledge and social consideration of Twitter users toward psychosis in comparison with 2 prevalent causes of death worldwide (breast cancer and diabetes mellitus), a relevant cause of severe neurocognitive impairment (Alzheimer disease) and a socially relevant disease (human immunodeficiency virus, HIV infection), as well as their areas of medical and nonmedical interest. In addition, we investigated the potential impact and reach derived from tweets and retweets of each condition.
Methods
Research Strategy
In this observational quantitative and qualitative study, we focused on searching for tweets that referred to psychosis over a period of 8 consecutive days in 2018. As controls, we studied in parallel the tweets related to breast cancer, diabetes mellitus, Alzheimer disease, and HIV infection. In this study, we focused on tweets with the following hashtags: #psychosis, #psychotic, #schizophrenic, #schizophrenia, #diabetes, #diabetic, #breastcancer, #hiv, and #alzheimer [40]. Content was limited to English-language tweets. Data collection spanned from Monday, February 26 to Monday, March 5, 2018. This period of time has at least 2 months of separation from any major international awareness month for any of the diseases we studied and was selected to avoid potential bias in the type of disease-related tweets.
Search Tool and Data Collection
In this study, we used the Twitter Firehose data stream, which is managed by Gnip and allows access to 100% of all public tweets that match a set of “search” criteria (query) [41]. In our study, the search criteria were the previously mentioned hashtags. Tweet Binder, the search engine we employed, uses automatic machine-learning text analysis algorithms, as well as node.js and the PHP language, which enables an analysis of tweets in the json format (used by Gnip).
Content Analysis Process
All of the collected tweets were classified using qualitative content analysis methods as a systematic method for making inferences from the text to summarize the content of communication [42]. In this study, we used a codebook specifically created by the members of the research team. All of the team’s members who qualitatively analyzed the content were medical doctors specialized in psychiatry, medical oncology, internal medicine, immunology, or endocrinology, with clinical practice in university hospitals. The analysis strategy included a series of steps. First, to achieve reliability, raters reviewed an initial subset of 100 tweets to apply initial classifications of each category. Differences in categorization and other discrepancies between the evaluators were discussed until a consensus was reached, and classification criteria were adapted to reflect the initial rating experience. Second, researchers grouped by pairs, independent and blinded, rated a second training set of 300 tweets using the improved codebook. The obtained reliability was higher than 90% for tweet content analysis, and a final version of the codebook was established. Third, all tweets were analyzed separately by 2 blinded researchers. If discrepancies in the classification of a tweet occurred between both raters (less than 10% of the cases), the whole group of researchers analyzed the tweet’s content and reached a final decision by a consensus of at least two-thirds of the research team. Tweets that included unclassifiable content were excluded.
Each tweet, depending on the content, was rated as medical or nonmedical. Medical tweets included a reference to the illness and its diagnosis, treatment, prognosis, or prevention. We also assessed if the content was medically appropriate or inappropriate according to the current medical knowledge. Nonmedical tweets were classified into 4 categories: (1) patient, family, or caregiver testimony; (2) information about medical health providers and scientific meetings; (3) solidarity, support campaigns, and advocacy; and (4) misuse. Nonmedical tweets were also classified as positive or pejorative, depending on the tone of the tweet. Classification criteria and examples of tweets by category are shown in Multimedia Appendix 1.
Measuring Influence on Twitter: Retweets and Hashtags’ Reach and Impact
We analyzed the number of retweets generated by each tweet as an indicator of the user interest in a given topic [43-45]. We also measured the potential reach and impact of the hashtags analyzed. Impact is a numerical value representing the potential views a tweet may receive. To calculate impact, we multiplied, for each user who contributed to the hashtag, the number of followers by the number of tweets posted, and finally, we added this number for all such users. Reach is a numerical value measuring the potential audience of the hashtag (how many people could have seen it). To calculate reach, we measured the number of followers of each user who contributed to the hashtags and added them all together. We collected the 10 hashtags most frequently associated with the hashtags we studied.
Ethical Considerations
This study received the approval of the University of Navarra Research Ethics Committee and was compliant with the research ethics principles of the Declaration of Helsinki (7th revision, 2013). However, this study did not directly involve human subjects nor include any intervention but instead used publicly available tweets. Nevertheless, we have taken care to not reveal any username and to avoid citing the tweets that could reveal it.
Statistical Analysis
A descriptive study of the sample was performed, describing the variables by their absolute and relative frequencies. The percentages found were compared using the chi-square test. The mean numbers of retweets per original tweet about the different diseases were compared by analysis of variance. The Tamhane test was performed for a posteriori comparison between diseases.
Results
Increased Pejorative Sentiment and Misuse Content in Psychosis-Related Tweets
The number of tweets generated about Alzheimer disease and psychosis were lower than that of diabetes, HIV infection, or breast cancer (Table 1). Of the total of 15,443 tweets analyzed, 101 were excluded according to the criteria of the study. Thus, 15,342 tweets were classified into 2 categories according to their medical or nonmedical content, and the frequencies of both categories between the different diseases were significantly different (P<.001; Table 1). The percentage of PRT with nonmedical content was higher than those in the groups related to diabetes, HIV infection, or Alzheimer disease and lower than that in the breast cancer group. The percentage of tweets with medical content was higher among those related to HIV infection and diabetes than those related to the other diseases analyzed.
Table 1.
Number and content of tweets about psychosis and control diseases. Percentages (%) were calculated with respect to the total number of tweets generated about the 5 diseases. Number of tweets with nonmedical and medical contents generated about the diseases. Percentages (%) were calculated with respect to the total number of tweets generated about each disease.
Medical condition | Tweets generated, n (%) | Contenta | |
Nonmedical, n (%) | Medical, n (%) | ||
Psychosis | 1029 (6.66) | 636 (61.81) | 393 (38.19) |
Breast cancer | 3703 (23.98) | 2434 (65.98) | 1255 (34.02) |
Diabetes | 6467 (41.88) | 3115 (48.65) | 3288 (51.35) |
Alzheimer | 930 (6.02) | 506 (54.64) | 420 (45.36) |
HIV | 3314 (21.46) | 1364 (41.40) | 1931 (58.60) |
Total | 15,443 (100) | 8055 (52.50) | 7287 (47.50) |
aTest chi-square; P<.001. 101 tweets nonclassifiable (99.35% analyzed).
Interestingly, different patterns of distribution of the nonmedical tweets among the 4 categorizes of contents were found between the diseases (P<.001; Figure 1). Remarkably, in PRT, the category with highest frequency of tweets was misuse, which was significantly higher than that found in the control diseases 33.3% (212/636) vs 1.15% (853/7419); (P<.001). In contrast, misuse was absent or minimal in breast cancer, diabetes, and HIV infection. The frequency of PRT with misuse content was 12 times higher than in those related to HIV infection. The frequency of tweets with solidarity and advocacy content related to HIV infection was the highest. The frequency of tweets with content about medical health providers and scientific meetings was lower in those related to psychosis, breast cancer, and HIV infection compared with Alzheimer disease and diabetes.
Figure 1.
Different percentages (%) of tweets with nonmedical and medical content generated about psychosis and control diseases. Percentages (%) were calculated with respect to the total number of tweets generated about each disease.
We analyzed the tone of the 8055 nonmedical tweets (Table 2). The frequencies of positive and nonpositive tweet contents were significantly different between the different diseases (P<.001). The frequency of PRT with positive content was significantly lower than what was found in the control diseases 63.7% (405/636) vs 88.67% (6522/7354; P<.001). The percentage of pejorative tweets related to psychosis 36.3% (231/636) doubled that of breast cancer 15.0% (365/2424) and diabetes 12.75% (677/3115) and was 5 times higher than that of Alzheimer 7.6% (38/506) and HIV 2.72% (37/1364). In psychosis and in the control diseases, the frequency of tweets with positive content was significantly lower in the misuse category than those found in the other 3 categories (P<.001).
Table 2.
Number of tweets with nonmedical, positive-tone content about psychosis and control diseases. Percentages (%) were calculated with respect to the total number of nonmedical contents tweets generated about each disease. Number of tweets with nonmedical, positive-tone content in the testimonies, medical health providers, solidarity/advocacy, or misuse categories generated about the diseases. Percentages (%) were calculated with respect to the total number of tweets generated about each category and disease.
Medical condition | Nonmedical content positivea, n (%) | Positive sentimentb | ||||
Personal/family, n (%) | Commercial/professional, n (%) | Solidarity/advocacy, n (%) | Missuse, n (%) | P value | ||
Psychosis | 405 (63.7) | 112 (80.58) | 105 (88.24) | 144 (87.27) | 44 (20.75) | <.001 |
Breast cancer | 2070(85.05) | 741 (80.81) | 501 (91.26) | 828 (85.54) | 0 (0) | <.001 |
Diabetes | 2703 (87.25) | 713 (74.89) | 1236 (97.86) | 746 (86.04) | 8 (50) | <.001 |
Alzheimer | 457 (92.40) | 94 (100) | 164 (98.20) | 194 (97.98) | 5 (13.89) | <.001 |
HIV | 1287 (97.28) | 129 (100) | 262 (100) | 818 (99.63) | 2 (6.45) | <.001 |
Total | 6927 (86.70) | 1789 (80.19) | 2268 (96.10) | 2730 (90.43) | 59 (20) | <.001 |
aTest chi-square; P<.001. 65 tweets not classifiable.
b85 tweets not classifiable.
The Medical Content of Psychosis-Related Tweets Showed High Scientific Appropriateness Geared Toward Prevention
We investigated the scientific appropriateness and areas of interest of the 7287 tweets with medical content, and we excluded 8 tweets according to the analysis criteria. The frequency of appropriateness between the 5 diseases was significantly different (P<.001; Table 3). According to the scientific evaluation, the content of the 391 PRT analyzed was correct. This frequency of scientific appropriateness was higher than that found in the 4 control diseases 100% (391/391) vs 93.66% (6030/6439; P<.001). The scientific appropriateness found in the diabetes- and HIV infection–related tweets was higher than that found in breast cancer and Alzheimer disease.
Table 3.
Number of tweets with medically appropriate content about psychosis and control diseases. Percentages (%) were calculated with respect to the total number of tweets generated with medical content about each disease. Number of tweets with medically appropriate content about diagnosis, treatment, prognosis, and prevention generated in the different diseases. Percentages (%) were calculated with respect to the total number of tweets generated about each different medical content and disease.
Medical condition | Medical content accuracya, N (%) | Scientific accuracyb | ||||
Diagnosis, N (%) | Treatment, N (%) | Prognostic, N (%) | Prevention, N (%) | P value | ||
Psychosis | 391 (100) | 28 (100) | 91 (100) | 2 (100) | 264 (100) | —c |
Breast cancer | 1034 (82.52) | 285 (84.07) | 400 (84.39) | 99 (84.62) | 250 (77.40) | .05 |
Diabetes | 3126 (95.57) | 627 (100.00) | 895 (88.61) | 85 (97.70) | 1519 (98.19) | <.001 |
Alzheimer | 374 (89.05) | 23 (95.83) | 60 (61.22) | 4 (66.67) | 287 (98.29) | <.001 |
HIV | 1905 (98.76) | 685 (98) | 258 (96.99) | 2 (100) | 958 (99.58) | .004 |
Total | 6830 (94.03) | 1648 (95.98) | 1704 (87.88) | 192 (89.72) | 3278 (96.75) | <.001 |
aTest chi-square; P<.001. 23 Tweets not classifiable.
b8 tweets not classifiable.
cIt is not possible to calculate the p value because in Psychosis the four categories (Diagnosis, Treatment, Prognosis and Prevention) had the same value (100).
Furthermore, the tweets were classified according to the area of interest of the medical content: diagnosis, prognosis, treatment, and prevention (Figure 1). We found a different pattern of distribution of the 4 categories of medical content between tweets related to psychosis and tweets related to the control diseases (P<.001). Interestingly, the frequency of tweets with content about disease prevention were higher in those related to psychosis and Alzheimer disease than in those related to diabetes, HIV infection, and breast cancer. Opposite results were observed in diagnosis-related tweets. Tweets with treatment content were higher in breast cancer 37.84% (475/1255). As shown in Table 3, the lowest frequencies of scientifically appropriate medical content were found in tweets related to treatment and prognosis of Alzheimer disease.
Psychosis-Related Tweets Showed High Frequency of Retweets
We measured the number of retweets generated about each disease (Table 4). We found that the retweet-to-tweet ratio, and thus, the probability of being retweeted for the PRT, was significantly higher than that found for the control diseases. We did not find significant differences (P=.49) in the frequency of retweets between those with pejorative or positive tone related to the different health conditions analyzed. We did not find significant differences in the frequency of retweets between those with misuse content and the rest of the nonmedical tweets (P=.08).
Table 4.
Potential impact, potential reach, and number of retweets generated by psychosis- and control disease–related tweets.
Medical condition | Potential impact | Potential reach | Contributors, n | Followers per contributor, n | Retweets per original tweet, mean (SE) | P valuea |
Psychosis | 7,738,305 | 5,360,995 | 1155 | 19,409 | 0.23 (1.22) | Refb |
Breast cancer | 62,348,473 | 20,930,244 | 3161 | 6621 | 0.03 (0.29) | <.001 |
Diabetes | 92,770,714 | 46,143,068 | 5087 | 9071 | 0.11 (0.01) | .002 |
Alzheimer | 10,019,729 | 7,118,104 | 1105 | 6442 | 0.04 (0.32) | <.001 |
HIV | 101,643,088 | 52,072,034 | 7308 | 11,029 | 0.08 (0.59) | .02 |
aAnalysis of variance; P<.001. Numbers are Tamhane test between psychosis and each disease.
bRef: reference category.
Psychosis-Related Tweets Showed Limited Reach and Impact
As shown in Table 4, we found that the potential impact and reach (7,738,305 and 5,360,995, respectively) of PRT were less than those of breast cancer (62,348,473 and 20,930,244, respectively), diabetes (92,770,714 and 46,143,068, respectively), Alzheimer disease (10,019,729 and 7,118,104, respectively), and HIV infection (101,643,088 and 52,072,034, respectively). Finally, we analyzed the 10 hashtags most frequently associated with the hashtags of the different diseases analyzed. We found that the hashtags most frequently associated with #psychosis were psychosis, mentalhealth, schizophrenia, cannabis, bipolar, depression, mental illness, schoolshooting, ptsd, and wpatc18. In the case of the control diseases, the hashtags most frequently associated with #breastcancer were cancer, bcsm, mastectomy, breastreconstruction, BreastCancerAwareness, bccww, health, blog, and chemo; the hashtags most frequently associated with #diabetes were health, obesity, t1d, cancer, diet, diabetic, insulin, t2d, and type1diabetes; the hashtags most frequently associated with #Alzheimer were dementia, health, brain, care, memory, caregiver, aging, science, and alzheimers; and the hashtags most frequently associated with #HIV were aids, PrEP, stigma, USA, health, Philippines, std, vaccine, and tuberculosis.
Discussion
Principal Findings
In this work, we investigated all the tweets generated about psychosis during 8 consecutive days in the winter of 2018. As controls, we studied 2 prevalent causes of death worldwide (breast cancer and diabetes mellitus), a relevant cause of severe neurocognitive impairment (Alzheimer disease), and a socially relevant disease (HIV infection) [46]. We found a different pattern of content in tweets about psychosis with respect to those related to control diseases. PRT showed a predominance of nonmedical content with increased frequency of misuse and pejorative tone with respect to the control diseases. However, the medical content of PRT showed high scientific appropriateness geared toward prevention. The potential reach and impact of the tweets related to psychosis were low but showed a high retweet-to-tweet ratio.
The search tool utilized for data collection allows access to 100% of all public tweets. Thus, the conclusions were obtained from the results measured the total population of tweets, and they are not deduced from the analysis of a reduced sample (previous health-related studies utilizing Twitter have generally focused on the analysis of a 1% sample of the total number of tweets available). To our knowledge, this is the first study that analyzed all tweets about psychosis in particular in a defined period of time.
Our data show a differential pattern of information and opinions expressed in the contents of the PRT in comparison with those relating to the different control diseases. The majority of PRT with nonmedical content were focused on misuse, with a small proportion expressing solidarity. This bias observed in the content of PRT was further supported by the finding that more than a third of these nonmedical tweets had pejorative content about the disease and/or patients. Unfortunately, psychosis is still employed as an insult in a relevant proportion of tweets. Our findings about the elevated misuse and pejorative tone toward psychosis on Twitter are consistent with previous studies that analyzed schizophrenia in selected samples of tweets [35-38]. A recent study found that the terms psychosis/psychotic are associated with a significantly higher number of tweets with negative contents than schizophrenia/schizophrenic [34].
The relevance of this evident and extended misuse and pejorative content found in PRT is supported by the comparison of these results with those found in the investigated control diseases. Misuse in psychosis was 4 times greater than in Alzheimer disease and was marginal in breast cancer, diabetes, and HIV infection. The frequency of pejorative psychosis tweets was 5 times greater than the frequencies found in tweets related to Alzheimer disease and HIV infection and doubled those of breast cancer and diabetes. The fact that #schoolshooting was among the most frequently associated hashtags with #psychosis also reflects the negative and incorrect stereotyping of psychosis by a relevant number of Twitter users. All together, these twitter data show that psychosis patients are targets of negative ideas, feelings or judgments by twitter users, demonstrating the persistence of social stigma for psychiatric diseases, in general, and psychosis, in particular [47-50]. The bad social habit of using “schizophrenia” or “psychosis” to refer to a “madness” of some kind might have an impact on these negative results found in PRT [51]. Social stigma has major adverse effects on the lives of people with mental health conditions [52]. Stigma has also been common in portrayals of physical conditions. HIV infection has been a paradigmatic example of an organic condition suffering not only social stigma but also stigma from health care providers and professionals [53,54]. Interestingly, our results show a marginal frequency of HIV-related tweets with misuse and negativity content. We found that fewer than 3% of the HIV-related tweets had stigmatizing content, and this low number was 6 times lower than the frequency recently described [35]. This reduction may be explained by different factors, including the size and the selection criteria of the sample and the temporal gap between the 2 studies. These results support the notion of evolution in the social attitudes about diseases.
The cause of stereotypes about psychosis in Twitter is multifactorial. It may reflect the persistence of social negative stereotyping and stigmatizing attitudes toward people with psychosis [11]. Furthermore, the use of Twitter for the distribution of health care information carries some risks that are even more pronounced in the field of mental health: high rate of misinformation, sources of questionable reliability, overwhelmingly high volumes of information available, and concerns about professionalism [55]. Due to the small number of characters required, tweets are often brief and must omit key information and may lead to fruitless discussions [56]. In this context, terms such as “psychotic” or “mentally ill” can be used to disparage or ridicule someone, thus spread social stigma to the social network. Furthermore, the massive and immediate response to nonexpert opinions or news related with mental disorders can convert Twitter on an “echo chamber of ideas,” representing shared opinions rather than balanced facts because of the ease of quoting or retweeting. Furthermore, other players may lead to the production of stigma in Twitter. The public rely on the media as a key source of information about mental illness turns the news media in a strong influence on public discourse and attitudes about mental health issues as well as to medical decisions, health service utilization, or consumption of antidepressants [57,58]. Selecting the topics they cover and highlighting certain aspects, they contribute to the creation of stereotypes. It has been shown that news stories referring to mental illness frequently emphasize on violence, although rates of violence among those with mental illnesses are very low, and people with schizophrenia are more likely to be victims of violence rather than perpetrators [59,60].
Twitter appears to be a relevant communication tool for distributing and acquiring medical information. Thus, it was relevant to investigate the scientific accuracy and the areas of interest of the medical content on Twitter pertaining to psychosis and the control disorders. Interestingly, the scientific content of all the PRT was surprisingly correct, and the rate was higher than those found in the control diseases. This accuracy of tweets containing medical content starkly contrasts the high rates of misuse and negative tone in the nonmedical tweets about psychosis. These data suggest Twitter is also used to communicate medical content about psychosis by users with correct medical information. The areas of interest of the medical information varied between psychosis and control diseases. Interestingly, the frequency of tweets with content about disease prevention was markedly higher in those related to psychosis or Alzheimer disease than in those of diabetes, HIV infection, and breast cancer. Different reasons may support this differential pattern of medical interest. The social perception of limited effectiveness for the medical treatment of psychosis and Alzheimer disease might support the special interest in preventive strategies. The chronic and severe impact of patients with both diseases on their families and caregivers might also explain the high interest in the prevention of these disorders [10,61,62]. The absence of established analytical or image criteria for the diagnosis of psychosis might also contribute to the limited interest in the diagnosis of the disease.
Finally, we investigated the interest and diffusion of the tweets generated about psychosis. In the period of time analyzed, the frequency of retweets generated by PRT was higher than that found in the control diseases. This parameter is considered an indicator of the user interest in the topic of each tweet [43-45]. However, the metrics of the tweets related to psychosis were small compared with those of the control diseases. The impact of the PRT measured as the potential views that the tweet may receive was clearly lower than that of the control diseases. Similarly, poor diffusion of the PRT was found when we calculated the potential audience or reach of the PRT. These metrics are used for the quantification of the diffusion of tweets and potential influence in society. Thus, our data imply that the stigmatization of and limited social support for psychosis are reflected in the low impact and reach of the PRT.
Strengths and Limitations
Although this study improves and expands previous research on the communication of psychosis in a popular and widely used form of social media, there are still some limitations. The rating process had an inherent degree of subjectivity because of differences in the perceived context and emotional tone of some tweets. This was made particularly evident by words that had dual meanings. There was also a degree of selection bias, as stigmatizing and trivializing tweets were more likely to be lacking in context and/or grammatical correctness, rendering them less likely to be considered for analysis. We minimized the effects of these issues through our robust rating criteria and binary rating system, which were chosen for the analysis performed by expert clinicians in the medical fields analyzed. To achieve maximum reliability, the qualitative analysis of the disease-related tweets required a manual input that was time-consuming and required expert involvement. We followed what we considered a gold-standard qualitative analysis strategy.
Conclusions
Twitter is a tool for developing interventions and strategies of information aimed at modifying health-related social and individual behaviors [63]. Our results support the dynamic and potentially positive evolution of the social stigmatization of health disorders, as can be observed in HIV infection and cancer. Thus, it is possible to expect a reduction of psychosis stigmatization. Moreover, mixed-methods research on Twitter and other social media may be a relevant strategy for measuring the effectiveness of the strategies and actions established for overcoming the social psychosis stigma [64]. A proactive sensitization by professionals, scientific societies, patient associations, and other social agents to use the promising platform of social media communication is needed. Furthermore, the respect and supportive in social media communication content may also impact a patient’s life and treatment. Psychosis patients often use social media [21], and despite the anonymity of Twitter, many users identify themselves as patients [27,32].
Although stigmatization is significantly decreasing and societal consideration is improving in other disorders, such as breast cancer and HIV, the stigma regarding psychosis is not decreasing. On one hand, psychosis is used as hate-speech on Twitter, but on the other hand, Twitter is used as a beacon of medically accurate information for the disorder. Therefore, Twitter may be a great tool for antistigma campaigns and promotion of healthy habits.
Acknowledgments
This work was partially supported by grants from the Fondo de Investigación de la Seguridad Social, Instituto de Salud Carlos III (PI18/01726), Spain, and the Programa de Actividades de I+D de la Comunidad de Madrid en Biomedicina (B2017/BMD-3804), Madrid, Spain.
Abbreviations
- PRT
psychosis-related tweets
Classification criteria and examples of tweets by category.
Footnotes
Conflicts of Interest: None declared.
References
- 1.Hjorthøj C, Stürup AE, McGrath J, Nordentoft M. Years of potential life lost and life expectancy in schizophrenia: a systematic review and meta-analysis. Lancet Psychiatry. 2017 Dec;4(4):295–301. doi: 10.1016/S2215-0366(17)30078-0. [DOI] [PubMed] [Google Scholar]
- 2.Walker ER, McGee RE, Druss BG. Mortality in mental disorders and global disease burden implications: a systematic review and meta-analysis. JAMA Psychiatry. 2015 Apr;72(4):334–41. doi: 10.1001/jamapsychiatry.2014.2502. http://europepmc.org/abstract/MED/25671328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Owen MJ, Sawa A, Mortensen PB. Schizophrenia. Lancet. 2016 Jul 2;388(10039):86–97. doi: 10.1016/S0140-6736(15)01121-6. http://europepmc.org/abstract/MED/26777917. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Cloutier M, Sanon AM, Guerin A, Nitulescu R, Ramanakumar AV, Kamat S. The economic burden of Schizophrenia in the United States in 2013. J Clin Psychiatry. 2016:764. doi: 10.4088/JCP.15m10278. http://www.psychiatrist.com/jcp/article/pages/2016/v77n06/v77n0613.aspx. [DOI] [PubMed] [Google Scholar]
- 5.Westman J, Eriksson SV, Gissler M, Hällgren J, Prieto ML, Bobo WV, Frye MA, Erlinge D, Alfredsson L, Ösby U. Increased cardiovascular mortality in people with schizophrenia: a 24-year national register study. Epidemiol Psychiatr Sci. 2018 Dec;27(5):519–27. doi: 10.1017/S2045796017000166. http://europepmc.org/abstract/MED/28580898. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lieberman J, First M. Psychotic disorders. Ropper AH. 2018 Jul 19;:270. doi: 10.1056/NEJMra1801490. http://www.nejm.org/doi/10.1056/NEJMra1801490. [DOI] [PubMed] [Google Scholar]
- 7.Radua J, Ramella-Cravaro V, Ioannidis JPA, Reichenberg A, Phiphopthatsanee N, Amir T, Yenn Thoo H, Oliver D, Davies C, Morgan C, McGuire P, Murray RM, Fusar-Poli P. What causes psychosis? An umbrella review of risk and protective factors. World Psychiatry. 2018 Feb;17(1):49–66. doi: 10.1002/wps.20490. doi: 10.1002/wps.20490. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Gask L, Coventry P. Person-centred mental health care: the challenge of implementation. Epidemiol Psychiatr Sci. 2012 Jun;21(2):139–44. doi: 10.1017/S2045796012000078. [DOI] [PubMed] [Google Scholar]
- 9.Fleischhacker WW, Arango C, Arteel P, Barnes TRE, Carpenter W, Duckworth K, Galderisi S, Halpern L, Knapp M, Marder SR, Moller M, Sartorius N, Woodruff P. Schizophrenia--time to commit to policy change. Schizophr Bull. 2014 Apr;40(Suppl 3):S165–94. doi: 10.1093/schbul/sbu006. http://europepmc.org/abstract/MED/24778411. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Mittendorfer-Rutz E, Rahman S, Tanskanen A, Majak M, Mehtälä J, Hoti F, Jedenius E, Enkusson D, Leval A, Sermon J, Taipale H, Tiihonen J. Burden for parents of patients with schizophrenia-a nationwide comparative study of parents of offspring with rheumatoid arthritis, multiple sclerosis, epilepsy, and healthy controls. Schizophr Bull. 2018 Sep 3;:1–10. doi: 10.1093/schbul/sby130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Imhoff R. Zeroing in on the effect of the schizophrenia label on stigmatizing attitudes: a large-scale study. Schizophr Bull. 2016 Mar;42(2):456–63. doi: 10.1093/schbul/sbv137. http://europepmc.org/abstract/MED/26409222. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Henderson C, Corker E, Lewis-Holmes E, Hamilton S, Flach C, Rose D, Williams P, Pinfold V, Thornicroft G. England's time to change antistigma campaign: one-year outcomes of service user-rated experiences of discrimination. Psychiatr Serv. 2012;63(5):451–7. doi: 10.1176/appi.ps.201100422. [DOI] [PubMed] [Google Scholar]
- 13.Carpiniello B, Pinna F. The reciprocal relationship between suicidality and stigma. Front Psychiatry. 2017;8:35. doi: 10.3389/fpsyt.2017.00035. doi: 10.3389/fpsyt.2017.00035. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Picco L, Abdin E, Pang S, Vaingankar JA, Jeyagurunathan A, Chong SA, Subramaniam M. Association between recognition and help-seeking preferences and stigma towards people with mental illness. Epidemiol Psychiatr Sci. 2018 Dec;27(1):84–93. doi: 10.1017/S2045796016000998. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.McGowan BS, Wasko M, Vartabedian BS, Miller RS, Freiherr DD, Abdolrasulnia M. Understanding the factors that influence the adoption and meaningful use of social media by physicians to share medical information. J Med Internet Res. 2012;14(5):e117. doi: 10.2196/jmir.2138. http://www.jmir.org/2012/5/e117/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Lee K, Hoti K, Hughes J, Emmerton L. Dr Google and the consumer: a qualitative study exploring the navigational needs and online health information-seeking behaviors of consumers with chronic health conditions. J Med Internet Res. 2014 Dec 2;16(12):e262. doi: 10.2196/jmir.3706. http://www.jmir.org/2014/12/e262/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Hesse BW, Moser RP, Rutten LJ. Surveys of physicians and electronic health information. N Engl J Med. 2010 Mar 4;362(9):859–60. doi: 10.1056/NEJMc0909595. [DOI] [PubMed] [Google Scholar]
- 18.Topf JM, Hiremath S. Social media, medicine and the modern journal club. Int Rev Psychiatry. 2015 Apr;27(2):147–54. doi: 10.3109/09540261.2014.998991. [DOI] [PubMed] [Google Scholar]
- 19.Huang S, Martin LJ, Yeh CH, Chin A, Murray H, Sanderson WB, Mohindra R, Chan TM, Thoma B. The effect of an infographic promotion on research dissemination and readership: a randomized controlled trial. CJEM. 2018 Dec;20(6):826–33. doi: 10.1017/cem.2018.436. [DOI] [PubMed] [Google Scholar]
- 20.Alvarez-Mon MA, Asunsolo Del Barco A, Lahera G, Quintero J, Ferre F, Pereira-Sanchez V, Ortuño F, Alvarez-Mon M. Increasing interest of mass communication media and the general public in the distribution of tweets about mental disorders: observational study. J Med Internet Res. 2018 Dec 28;20(5):e205. doi: 10.2196/jmir.9582. http://www.jmir.org/2018/5/e205/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Miller BJ, Stewart A, Schrimsher J, Peeples D, Buckley PF. How connected are people with schizophrenia? Cell phone, computer, email, and social media use. Psychiatry Res. 2015 Feb 28;225(3):458–63. doi: 10.1016/j.psychres.2014.11.067. [DOI] [PubMed] [Google Scholar]
- 22.Wakamiya S, Morita M, Kano Y, Ohkuma T, Aramaki E. Tweet classification toward Twitter-based disease surveillance: new data, methods, and evaluations. J Med Internet Res. 2019 Feb 20;21(2):e12783. doi: 10.2196/12783. http://www.jmir.org/2019/2/e12783/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Wakamiya S, Matsune S, Okubo K, Aramaki E. Causal relationships among pollen counts, tweet numbers, and patient numbers for seasonal allergic rhinitis surveillance: retrospective analysis. J Med Internet Res. 2019 Feb 20;21(2):e10450. doi: 10.2196/10450. http://www.jmir.org/2019/2/e10450/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Hendriks H, van den Putte B, Gebhardt W, Moreno M. Social drinking on social media: content analysis of the social aspects of alcohol-related posts on facebook and instagram. J Med Internet Res. 2018 Dec 22;20(6):e226. doi: 10.2196/jmir.9355. http://www.jmir.org/2018/6/e226/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Kolliakou A, Ball M, Derczynski L, Chandran D, Gkotsis G, Deluca P, Jackson R, Shetty H, Stewart R. Novel psychoactive substances: an investigation of temporal trends in social media and electronic health records. Eur Psychiatry. 2016 Dec;38:15–21. doi: 10.1016/j.eurpsy.2016.05.006. [DOI] [PubMed] [Google Scholar]
- 26.Eysenbach G. Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the internet. J Med Internet Res. 2009 Mar 27;11(1):e11. doi: 10.2196/jmir.1157. http://www.jmir.org/2009/1/e11/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Hswen Y, Naslund JA, Brownstein JS, Hawkins JB. Online communication about depression and anxiety among twitter users with schizophrenia: preliminary findings to inform a digital phenotype using social media. Psychiatr Q. 2018 Dec;89(3):569–80. doi: 10.1007/s11126-017-9559-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Thackeray R, Burton S, Giraud-Carrier C, Rollins S, Draper C. Using Twitter for breast cancer prevention: an analysis of breast cancer awareness month. BMC Cancer. 2013 Oct 29;13:508. doi: 10.1186/1471-2407-13-508. https://bmccancer.biomedcentral.com/articles/10.1186/1471-2407-13-508. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Berry N, Lobban F, Belousov M, Emsley R, Nenadic G, Bucci S. #WhyWeTweetMH: understanding why people use Twitter to discuss mental health problems. J Med Internet Res. 2017 Dec 5;19(4):e107. doi: 10.2196/jmir.6173. http://www.jmir.org/2017/4/e107/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Arseniev-Koehler A, Lee H, McCormick T, Moreno MA. #Proana: pro-eating disorder socialization on Twitter. J Adolesc Health. 2016 Dec;58(6):659–64. doi: 10.1016/j.jadohealth.2016.02.012. [DOI] [PubMed] [Google Scholar]
- 31.Sutton J, Vos SC, Olson MK, Woods C, Cohen E, Gibson CB, Phillips NE, Studts JL, Eberth JM, Butts CT. Lung cancer messages on Twitter: content analysis and evaluation. J Am Coll Radiol. 2018 Dec;15(1 Pt B):210–7. doi: 10.1016/j.jacr.2017.09.043. [DOI] [PubMed] [Google Scholar]
- 32.Lachmar E, Wittenborn A, Bogen K, McCauley H. #MyDepressionLooksLike: examining public discourse about depression on twitter. JMIR Ment Health. 2017 Oct 18;4(4):e43. doi: 10.2196/mental.8141. http://mental.jmir.org/2017/4/e43/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Lee JL, DeCamp M, Dredze M, Chisolm MS, Berger ZD. What are health-related users tweeting? A qualitative content analysis of health-related users and their messages on Twitter. J Med Internet Res. 2014 Oct 15;16(10):e237. doi: 10.2196/jmir.3765. http://www.jmir.org/2014/10/e237/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Passerello G, Hazelwood J, Lawrie S. Using Twitter to assess attitudes to schizophrenia and psychosis. BJPsych Bull. 2019 Feb 20;:1–9. doi: 10.1192/bjb.2018.115. https://www.cambridge.org/core/product/identifier/S2056469418001158/type/journal_article. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Robinson P, Turk D, Jilka S, Cella M. Measuring attitudes towards mental health using social media: investigating stigma and trivialisation. Soc Psychiatry Psychiatr Epidemiol. 2019 Jan;54(1):51–8. doi: 10.1007/s00127-018-1571-5. http://europepmc.org/abstract/MED/30069754. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Reavley N, Pilkington P. Use of Twitter to monitor attitudes toward depression and schizophrenia: an exploratory study. PeerJ. 2014;2:e647. doi: 10.7717/peerj.647. doi: 10.7717/peerj.647. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Athanasopoulou C, Sakellari E. 'Schizophrenia' on Twitter: content analysis of Greek language tweets. Stud Health Technol Inform. 2016;226:271–4. [PubMed] [Google Scholar]
- 38.Joseph AJ, Tandon N, Yang LH, Duckworth K, Torous J, Seidman LJ. #Schizophrenia: use and misuse on Twitter. Schizophr Res. 2015;165:5. doi: 10.1016/j.schres.2015.04.009. [DOI] [PubMed] [Google Scholar]
- 39.Lydecker J, Cotter E, Palmberg A, Simpson C, Kwitowski M, White K, Mazzeo SE. Does this Tweet make me look fat? A content analysis of weight stigma on Twitter. Eat Weight Disord. 2016 Jun;21(2):229–35. doi: 10.1007/s40519-016-0272-x. [DOI] [PubMed] [Google Scholar]
- 40.Chiang AL, Vartabedian B, Spiegel B. Harnessing the hashtag: a standard approach to GI dialogue on social media. Am J Gastroenterol. 2016 Aug;111(8):1082–4. doi: 10.1038/ajg.2016.259. [DOI] [PubMed] [Google Scholar]
- 41.Joseph K, Landwehr P, Carley K. Two 1%s Don't Make a Whole: Comparing Simultaneous Samples from Twitter's Streaming API. International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation; March 31-April 3, 2014; Washington, DC, USA. Switzerland: Springer, Cham; 2014. pp. 75–83. http://link.springer.com/10.1007/978-3-319-05579-4_10. [Google Scholar]
- 42.Hsieh H, Shannon S. Three approaches to qualitative content analysis. Qual Health Res. 2005 Nov;15(9):1277–88. doi: 10.1177/1049732305276687. [DOI] [PubMed] [Google Scholar]
- 43.Tsugawa S, Kito K. Retweets as a predictor of relationships among users on social media. PLoS One. 2017;12(1):e0170279. doi: 10.1371/journal.pone.0170279. http://dx.plos.org/10.1371/journal.pone.0170279. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Fung IC, Jackson AM, Mullican LA, Blankenship EB, Goff ME, Guinn AJ, Saroha N, Tse ZT. Contents, followers, and retweets of the Centers for Disease Control and Prevention's Office of Advanced Molecular Detection (@CDC_AMD) Twitter profile: cross-sectional study. JMIR Public Health Surveill. 2018 Apr 2;4(2):e33. doi: 10.2196/publichealth.8737. http://publichealth.jmir.org/2018/2/e33/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Suh B, Hong L, Pirolli P, Chi E. Want to be Retweeted? Large Scale Analytics on Factors Impacting Retweet in Twitter Network. 2010 IEEE Second International Conference on Social Computing; August 20-22, 2010; Minneapolis, MN, USA. 2010. pp. 177–84. http://ieeexplore.ieee.org/document/5590452/ [Google Scholar]
- 46.Ayers JW, Althouse BM, Dredze M, Leas EC, Noar SM. News and internet searches about human immunodeficiency virus after Charlie Sheen's disclosure. JAMA Intern Med. 2016 Apr;176(4):552–4. doi: 10.1001/jamainternmed.2016.0003. [DOI] [PubMed] [Google Scholar]
- 47.Hayhow B, Gaillard F, Velakoulis D, Walterfang M. Delayed diagnosis of multiple sclerosis in a patient with schizoaffective disorder: a case of 'diagnostic overshadowing'. Aust N Z J Psychiatry. 2015 Feb;49(2):186–7. doi: 10.1177/0004867414551067. [DOI] [PubMed] [Google Scholar]
- 48.Corrigan P, Druss B, Perlick D. The impact of mental illness stigma on seeking and participating in mental health care. Psychol Sci Public Interest. 2014 Oct;15(2):37–70. doi: 10.1177/1529100614531398. [DOI] [PubMed] [Google Scholar]
- 49.Webber M, Corker E, Hamilton S, Weeks C, Pinfold V, Rose D, Thornicroft G, Henderson C. Discrimination against people with severe mental illness and their access to social capital: findings from the Viewpoint survey. Epidemiol Psychiatr Sci. 2014 Jun;23(2):155–65. doi: 10.1017/S2045796013000243. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Lasalvia A, Zoppei S, Van Bortel T, Bonetto C, Cristofalo D, Wahlbeck K, Bacle SV, Van Audenhove C, van Weeghel J, Reneses B, Germanavicius A, Economou M, Lanfredi M, Ando S, Sartorius N, Lopez-Ibor JJ, Thornicroft G, ASPEN/INDIGO Study Group Global pattern of experienced and anticipated discrimination reported by people with major depressive disorder: a cross-sectional survey. Lancet. 2013 Jan 5;381(9860):55–62. doi: 10.1016/S0140-6736(12)61379-8. [DOI] [PubMed] [Google Scholar]
- 51.Schomerus G, Matschinger H, Angermeyer MC. Familiarity with mental illness and approval of structural discrimination against psychiatric patients in Germany. J Nerv Ment Dis. 2007 Jan;195(1):89–92. doi: 10.1097/01.nmd.0000252314.45371.92. [DOI] [PubMed] [Google Scholar]
- 52.Krajewski C, Burazeri G, Brand H. Self-stigma, perceived discrimination and empowerment among people with a mental illness in six countries: pan European stigma study. Psychiatry Res. 2013 Dec 30;210(3):1136–46. doi: 10.1016/j.psychres.2013.08.013. [DOI] [PubMed] [Google Scholar]
- 53.Roeloffs C, Sherbourne C, Unützer J, Fink A, Tang L, Wells KB. Stigma and depression among primary care patients. Gen Hosp Psychiatry. 2003;25(5):311–5. doi: 10.1016/s0163-8343(03)00066-5. [DOI] [PubMed] [Google Scholar]
- 54.Greene K, Banerjee S. Disease-related stigma: comparing predictors of AIDS and cancer stigma. J Homosex. 2006;50(4):185–209. doi: 10.1300/J082v50n04_08. [DOI] [PubMed] [Google Scholar]
- 55.Pershad Y, Hangge PT, Albadawi H, Oklu R. Social medicine: Twitter in healthcare. J Clin Med. 2018 May 28;7(6):1–9. doi: 10.3390/jcm7060121. http://www.mdpi.com/resolver?pii=jcm7060121. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Widmer RJ, Engler NB, Geske JB, Klarich KW, Timimi FK. An academic healthcare twitter account: the Mayo Clinic experience. Cyberpsychol Behav Soc Netw. 2016 Jun;19(6):360–6. doi: 10.1089/cyber.2015.0607. [DOI] [PubMed] [Google Scholar]
- 57.Ross A, Morgan A, Jorm A, Reavley N. A systematic review of the impact of media reports of severe mental illness on stigma and discrimination, and interventions that aim to mitigate any adverse impact. Soc Psychiatry Psychiatr Epidemiol. 2019 Jan;54(1):11–31. doi: 10.1007/s00127-018-1608-9. [DOI] [PubMed] [Google Scholar]
- 58.Green Lauridsen M, Kälvemark Sporrong S. How does media coverage effect the consumption of antidepressants? A study of the media coverage of antidepressants in Danish online newspapers 2010-2011. Res Social Adm Pharm. 2018 Dec;14(7):638–44. doi: 10.1016/j.sapharm.2017.07.011. [DOI] [PubMed] [Google Scholar]
- 59.McGinty E, Kennedy-Hendricks A, Choksy S, Barry C. Trends in news media coverage of mental illness in the United States: 1995–2014. Health Aff (Millwood) 2016 Dec 1;35(6):1121–9. doi: 10.1377/hlthaff.2016.0011. http://europepmc.org/abstract/MED/27269031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Caqueo-Urízar A, Boyer L, Urzúa A, Williams D. Self-stigma in patients with schizophrenia: a multicentric study from three Latin-America countries. Soc Psychiatry Psychiatr Epidemiol. 2019 Feb 26;:1–5. doi: 10.1007/s00127-019-01671-4. [DOI] [PubMed] [Google Scholar]
- 61.Gupta S, Isherwood G, Jones K, Van Impe K. Assessing health status in informal schizophrenia caregivers compared with health status in non-caregivers and caregivers of other conditions. BMC Psychiatry. 2015 Jul 21;15:162. doi: 10.1186/s12888-015-0547-1. https://bmcpsychiatry.biomedcentral.com/articles/10.1186/s12888-015-0547-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Daley S, Murray J, Farina N, Page TE, Brown A, Basset T, Livingston G, Bowling A, Knapp M, Banerjee S. Understanding the quality of life of family carers of people with dementia: development of a new conceptual framework. Int J Geriatr Psychiatry. 2019 Dec;34(1):79–86. doi: 10.1002/gps.4990. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Booth RG, Allen BN, Bray Jenkyn KM, Li L, Shariff SZ. Youth mental health services utilization rates after a large-scale social media campaign: population-based interrupted time-series analysis. JMIR Ment Health. 2018 Apr 6;5(2):e27. doi: 10.2196/mental.8808. http://mental.jmir.org/2018/2/e27/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Saha K, Weber I, Birnbaum M, de Choudhury M. Characterizing awareness of schizophrenia among Facebook users by leveraging Facebook advertisement estimates. J Med Internet Res. 2017 Dec 8;19(5):e156. doi: 10.2196/jmir.6815. http://www.jmir.org/2017/5/e156/ [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Classification criteria and examples of tweets by category.