The times, they are a-changin’: tracking shifts in mental health signals from early phase to later phase of the COVID-19 pandemic in Australia

Abstract

Introduction

Widespread problems of psychological distress have been observed in many countries following the outbreak of COVID-19, including Australia. What is lacking from current scholarship is a national-scale assessment that tracks the shifts in mental health during the pandemic timeline and across geographic contexts.

Methods

Drawing on 244 406 geotagged tweets in Australia from 1 January 2020 to 31 May 2021, we employed machine learning and spatial mapping techniques to classify, measure and map changes in the Australian public’s mental health signals, and track their change across the different phases of the pandemic in eight Australian capital cities.

Results

Australians’ mental health signals, quantified by sentiment scores, have a shift from pessimistic (early pandemic) to optimistic (middle pandemic), reflected by a 174.1% (95% CI 154.8 to 194.5) increase in sentiment scores. However, the signals progressively recessed towards a more pessimistic outlook (later pandemic) with a decrease in sentiment scores by 48.8% (95% CI 34.7 to 64.9). Such changes in mental health signals vary across capital cities.

Conclusion

We set out a novel empirical framework using social media to systematically classify, measure, map and track the mental health of a nation. Our approach is designed in a manner that can readily be augmented into an ongoing monitoring capacity and extended to other nations. Tracking locales where people are displaying elevated levels of pessimistic mental health signals provide important information for the smart deployment of finite mental health services. This is especially critical in a time of crisis during which resources are stretched beyond normal bounds.

Summary box.

What is already known about this subject?

• Supplementary to survey-based assessments dominantly used in mental health research, social media data (eg, Twitter) provide a powerful source of information to track the signals of mental status individually and the patterns of mental health signals of aggregated populations.

• Current survey-based studies are subject to limited sampling sizes or lacking locational information to map the spatial pattern of mental health, while social media data-based studies primarily focus on the early stage of the pandemic with case studies dominantly in the USA.

• There is a pressing need to extend the mental health assessment to a national scale and to a longer timeline of the pandemic to reflect broad trends of mental health, and to be implemented in Australia where mental health issues in the COVID-19 are less explored.

Summary box.

What are the new findings?

• We find that the mental health signals of the Australian public who used Twitter changed from being pessimistic in the early phase of the pandemic (before 11 March 2020) to be optimistic during the first 3 months of the pandemic and furthermore optimistic at the middle phase of the pandemic (11 March 2020–25 March 2021), followed by a progressive recession towards a more pessimistic outlook in the later phase of the pandemic (after 26 March 2021).

• More specifically, the feeling of fear accounted for the largest proportion of emotion during the pandemic, which was mixed with joy, anticipation and trust accounting for relatively larger proportions of emotion than sadness and anger.

• Important geographic differences of public mental health signals by the capital city were also observed. In the later phase of the pandemic, public mental health signals decreased toward pessimistic (particularly fear and anger) most apparent in Darwin, followed by Adelaide and Sydney.

• To the best of our knowledge, the current study represents the first nationwide investigation using social media data with a large spatial and temporal coverage to unveil broad trends in the public’s mental health signals from the early phase to later phase of the pandemic in Australia.

What do the new findings imply?

• We set out a novel empirical framework to systematically classify, measure and map mental health signals of a nation, through which the role of public health policy and mental health services in face of the pandemic can be assessed.

• The utility of social media data enables us to reveal broad trends in public mental health signals to supplement traditional survey-based approaches.

• In the spirit of reproducibility, we share our methodological approach such that our empirical framework that will be readily augmented into a monitoring capacity and extended to other nations.

• Our study also provides the policy implications on delivering digital/online mental health programmes and services to people in need, and education on vaccine safety to diminish the public’s concern and reluctance to vaccination.

Introduction

Globally, the outbreak of the COVID-19 pandemic has caused profound social and economic impacts and threatens to create a mental health crisis.1 The public’s negative sentiment (eg, depression, fear, sadness and anxiety) toward COVID-19 has been observed in studies in the USA,2 UK,3 Australia4 and China,5 alongside a number of European nations.6 Furthermore, the elevated need for mental health services has been reported, although the increasing prevalence of vaccination may act to lower the negativity toward COVID-19.7 As the United Nations’ policy brief COVID-19 and the need for action on mental health,8 it concluded that an increased level of mental health crises in the era of COVID-19 is a priority worth a prompt response urgently planned by each country. Australia, as the largest developed country in the Southern Hemisphere, has invested substantial efforts to control the virus spread, although has been facing widespread problems of psychological distress that threatened the public’s mental health especially at the initial stage of the pandemic, according to the COLLATE (COvid-19 and you: mentaL heaLth in AusTralia now survEy) project.4 However, what remains unknown is a nationwide shift of mental health along the different phases of the pandemic and across geographic contexts.

Through a systematic search of the Web of Science, PubMed, medRxiv, SSRN for articles published in English and preprints of articles on COVID-19 and mental health-related research, we found out that the current research in mental health in the COVID-19 context has largely employed survey-based assessments,9 reporting the increase of person’s vulnerability to experiencing psychological distress in the early pandemic.4 10–13 However, these studies primarily focus on the early stage of the pandemic and are subject to data drawbacks (eg, under-representativeness and limited data coverage) without the capability to reflect the universal trend of public mental health along the full timeline of the pandemic. Alternatively, social media data (eg, Twitter) provides a potential powerful source of information for mental health researchers on quantifying the psychological reaction of a given population to a certain phenomenon. The use of both language and social expressions readily observable in the qualitative content of social media data are telling indicators of mental health, providing important insights into the signals of sentiment and emotion individually to reflect the patterns of mental signals of aggregated population.14 These studies driven by social media data have used advanced modelling techniques (eg, machine learning algorithms) to investigate the public’s mental health signals toward home schooling,15 social restriction policies16 and vaccination.17 However, they were limited to the early stage of the pandemic and with case studies dominantly in the USA. There is a pressing need to extend the mental health assessment to a national scale and to a longer timeline of the pandemic to reflect broad trends of mental health, and to be implemented in Australia where mental health studies in the COVID-19 are less explored.

To address knowledge deficits, this study aims to classify, measure and map changes in the public’s mental health signals in Australia through 244 406 geotagged tweets (ie, tweets with location information, hereinafter termed as geotweets) in Australia between the period 1 January 2020 and 31 May 2021. Employing machine learning and spatial analytic techniques, we address three questions: (1) ‘to what extent do the public’s mental health signals change along the pandemic timeline?’, (2) ‘what are the topics or keywords discussed by the public potentially associated with the change of mental health signals?’ and (3) ‘to what extent do the public’s mental health signals vary across capital cities?’. In doing so, this study delineates the locales where people with elevated levels of pessimistic mental health signals concentrate and provides important information through which the allocation of finite mental health facilities and services can be deployed. Our study contributes the first investigation of mental health associated with COVID-19 in Australia, offering a machine learning-based empirical framework to unveil broad trends of the public’s mental health signals shifting alongside the pandemic and goes some way to enrich the requisite evidence necessary for place-based mental health policy and planning.

Methods

Data retrieval and post-processing

We use the Twitter academic full track application programming interface (AFT-API) to search and retrieve geotweets in Australia. Compared with the normal Twitter API which returns 1% of the total tweets for data privacy purposes, AFT-API enables us to fully retrieve tweets with predefined queries and improve the data coverage and representativeness.18 We defined the searching terms as ‘pandemic, epidemic, virus, COVID-19*, coronavirus, corona, and vaccin*’; the search timespan was defined as from 1 January 2020 to 31 May 2021 with the country defined as ‘AU’ (Australia). Consequently, 244 406 geotweets posted by 24 296 Twitter users were retrieved from the total+860 million tweets in Australia. The statistical summary of geotweets across the capital cities and pandemic phases is provided in online supplemental appendix table A2. In order to investigate the change of mental health signals over time, we further divided the analytical timeline into four phases: phase 1 was the early phase of the pandemic from 1 January 2020 to 10 March 2020—1 day before the announcement of the global pandemic by WHO on 11 March 202019; phase 2 was the first wave of the pandemic in Australia defined by Wang et al 20 from 11 March 2020 to 15 June 2020 during which COVID-19 cases spread out to each of the states in Australia in tandem with a series of social restriction policies intensively implemented by governments at different levels; phase 3 was the second wave of the pandemic20 from 16 June 2020 to 25 March 2021 during which the virus resurged dominantly in the State of Victoria; and phase 4 was the later phase of the pandemic featured by the implementation of mass vaccination, starting from 26 March 2021 when the COVID-19 vaccine, Pfizer, was first approved in Australia21 to 31 May 2021 when this study was conducted. The individual-level geotweets were then aggregated by day, week, phase and by capital city in the modelling process at the later stage.

Machine learning models and mapping methods

We commence with a sentiment analysis by the Valence Aware Dictionary for sEntiment Reasoning (VADER), a machine learning model (online supplemental appendix section 2.1) to estimate sentiment compound scores (SCS) ranging between −1 (extremely negative) and +1 (extremely positive).22 Such SCS were further reclassified to represent three general trends of mental health signals based on the threshold of 0.5 and −0.5 as the 75% and 25% quantiles: optimistic (SCS >0.5), neutral (−0.5<SCS<0.5) or pessimistic (SCS <−0.5). It was followed by an emotion analysis by the National Research Council Canada Lexicon model (online supplemental appendix section 2.2) to detail the optimistic and pessimistic mental health signals to eight types of emotions,23 including joy, trust, anticipation and surprise as optimistic signals as well as fear, anger, sadness and disgust as pessimistic signals. Each type of emotion was measured as a percentage indicating the proportion of geotweets with such emotion accounted for the total geotweets. Then, SCS and emotion percentage were analysed on daily and weekly basis at the national level. We then employed a word cloud mapping24 (online supplemental appendix section 2.3) to visualise the popular keywords discussed by the public based on the frequency of keywords and compared these keywords over four phases to speculate the episodic events potentially associated with the change of the public’s mental health signals over time.

In order to reveal the spatial variation of mental health signals, we located geotweets by their geographic coordinates (online supplemental appendix section 1). Given 82.2% of geotweets were located within the Great Metropolitan Areas (GMA) of state/territory capital cities where 67.7% of Australians inhabit,25 our spatial analysis focused on the eight GMA of capital cities, including Greater Sydney, Melbourne, Brisbane, Perth, Adelaide, Darwin, Hobart and Canberra (ie, Australian Capital Territory). We employed a kernel density mapping approach26 (online supplemental appendix section 2.4) to reveal the hotspot of optimistic signals (where optimistic tweets were concentrated) and the coldspot of pessimistic signals (where pessimistic tweets were concentrated). We then compared the eight types of emotion at the capital city level over four phases to unveil the change of mental health signals along the whole timeline of the pandemic.

Patient and public involvement

Not applicable

Results

Temporal change of mental health signals from the early phase to later phase of the pandemic

Figure 1 reveals the temporal change of mental health signals reflected by the sentiment score and eight types of emotion along the timeline of the pandemic. The sentiment score in the early phase of pandemic (mean=−0.043, 95% CI −0.052 to –0.034, online supplemental appendix table A3) increased by 174.1% to (mean=0.058, 95% CI 0.055 to 0.062) in the first wave of the pandemic and further increased by 9.4% in the second wave (mean=0.064, 95% CI 0.061 to 0.066), indicating that the public’s mental health signals changed from being relatively pessimistic in the first wave of the pandemic to being more optimistic during the pandemic. It is possibly due to that the implementation of policy interventions effectively controlling the virus spread, which were backed by decreasing numbers of confirmed COVID-19 cases alongside the introduction of social security payments (eg, the JobKeeper scheme)27 that together were implemented from 30 March onward. The government’s effective action on controlling virus transmission and supporting business/employed persons affected by the significant economic impact of the COVID-19 can be seen as to have boosted morale and improved positive sentiment. However, the sentiment score in the later phase of the pandemic has a sharp decrease by 48.8% (95% CI 34.7 to 64.9) from 26 March 2021 to 11 April 2021 (figure 1A), possibly associated with the concern and redundancy of vaccination reflected by the later word cloud mapping (figure 2), for example, the side effects from AstraZeneca.28 When we further detail the optimistic and pessimistic mental health signals into eight types of emotion (figure 1A), fear accounts for the largest proportion of emotion (21.52%, 95% CI 20.82 to 22.27, online supplemental appendix table A4) in the early phase of the pandemic, although its proportion decreases from the early phase to the first wave of the pandemic meanwhile the proportions of optimistic signals (ie, trust, anticipation and joy) increase. During the pandemic, fear still accounts for the largest proportion (18.57% in phase 2 and 19.21% in phase 3), followed by trust, anticipation, sadness, joy, anger, surprise and disgust (online supplemental appendix table A4). In the later phase of the pandemic, fear has an obvious increase to the peak of 24.24% on 21 May 2021 and all other types of emotion have no substantial changes over time.

Figure 1.

Temporal change of the public’s (A) sentiment score and (B) emotion by type over four phases. Note: data before 11 January 2020 were excluded given the number of geotweets on these days was less than 30 per day.

Figure 2.

Keywords potentially related to optimistic and pessimistic mental health signals over four phases.

In the word cloud maps (figure 2) with word frequencies detailed in online supplemental appendix table A5, optimistic mental health signals emerge in the keywords of ‘great, help, love, thank, need, good, time, well, best and australia’ in the early phase of the pandemic. Among them, the size of the keywords ‘thank, good, great, need and love’ grows in the first wave of the pandemic, indicating the rising popularity of such words being discussed and are potentially associated with the increase of optimistic mental signals in the first wave of the pandemic as depicted in figure 1. Optimistic keywords in the later phases include ‘support, love, good, thank, great, vaccine and need’ and ‘vaccine’ become increasingly frequent toward the later phase of the pandemic. In contrast, pessimistic mental health signals in the early phase of the pandemic emerge in the keywords of ‘china, panic, crisis, kill, spread, bad, outbreak, infect, death, die and fear’ as well as ‘toilet paper’, which potentially associates with the panic shopping in February 2020 and early March 2020 (Wang et al).20 Among these keywords, the size of ‘die, bad, kill, crisis and death’ increases in the first wave of the pandemic with the newly emerging keywords such as ‘lockdown’ and ‘trump’. In the second wave of the pandemic, the size of ‘trump, kill, die and death’ has no substantial change, with emerging ‘victoria state’, where the second wave of the pandemic mainly attacked. Pessimistic mental health signals in the later phase of the pandemic are dominantly associated with the ‘vaccine and vaccinate’, followed by ‘fail, india, die and lockdown’, indicating the mass vaccination that started after March 2021 in Australia may intertwine with the pessimistic feeling, potentially explained by the safety concern and redundancy toward COVID-19 vaccines.28 It is worth noting that some words (eg, ‘lockdown, quarantine’) appear frequently in both optimistic geotweets (eg, ‘I support lockdown orders’ and ‘Quarantine makes our community safe and protected’) and pessimistic geotweets (eg, ‘The continuous lockdown sucks!’ and ‘Quarantine makes me frustrated’), given they are intensively discussed.

Spatial variation of mental health signals in Australian capital cities

The spatial variations of mental health signals are distinct across capital cities (figure 3) with the hotspots of optimistic signals (the concentration of optimistic geotweets) shown as red spots, whereas coldspots (the concentration of pessimistic geotweets) are shown as blue spots. Some hotspots of optimistic signals are observed in the inner city of Darwin (sentiment score=0.61, online supplemental appendix table A6), Perth (sentiment score=0.91), Adelaide (sentiment score=0.87), Melbourne (sentiment score=0.92) and Canberra (sentiment score=0.93), while hotspots appear in western suburbs of Brisbane (eg, Chapel Hill (sentiment score=0.97)) and western and eastern suburbs of Sydney (eg, Springwood (sentiment score=0.98)). In the contrast, the coldspots of pessimistic signals are concentrated in Galston–Laughtondale (sentiment score=−0.67) and Berowra–Brooklyn–Cowan (sentiment score=−0.71) northwest to Sydney inner city, in Brookfield, Kenmore Hills (−0.85), west to the Brisbane inner city, in and around Darwin airport (sentiment score=−0.61), and in Greenway (sentiment score=−0.54) of Canberra, whereas such coldspots are more dispersedly distributed in Melbourne, Perth, Adelaide and Hobart.

Figure 3.

Kernel density estimates of optimistic and pessimistic mental health signals in eight capital cities.

The change of mental health signals over four phases is further examined in online supplemental appendix tables A6,7. In the early phase of the pandemic, pessimistic mental health signals are most apparently observed in Adelaide (sentiment score=−0.11, 95% CI −0.19 to –0.03, online supplemental appendix table A6), followed by Melbourne (sentiment score=−0.08, 95% CI −0.13 to –0.03) and Brisbane (sentiment score=−0.05, 95% CI −0.11 to 0.01). The public’s mental health signals in the first wave of the pandemic have been improved in all capital cities with the increased sentiment score most apparently in Perth (218.8%, online supplemental appendix table A7) and least apparently in Hobart (34.8%). From the first wave to the second wave of the pandemic, the mental health signals have been decreased most apparently in Darwin by −111.5%, followed by Adelaide (−76.1%) and Sydney (−16.2%), whereas the increase appears most apparently in Canberra by 33.7%, followed by Perth (31.9%) and Melbourne (12.2%). From the second wave of the pandemic to the later phase of the pandemic, the increase of mental health signals only remains in Perth (increase by 104.1%) and Hobart (36.3%).

The changes of emotion by type in each capital city are compared across four phases (figure 4; online supplemental appendix tables A8–12). From the early phase to the first wave of the pandemic, there are substantial increases of joy, anticipation and trust in Sydney (2.91% for joy, 1.91% for anticipation and 2.10% for trust; online supplemental appendix table A12), Melbourne (2.37% for joy, 0.79% for anticipation and 2.25% for trust), Brisbane (1.74% for joy, 2.37% for anticipation and 3.37% for trust), Perth (1.67% for joy, 0.75% for anticipation and 1.71% for trust) and Adelaide (1.95% for joy, 2.35% for anticipation and 3.01% for trust). Meanwhile, there are significant decreases of fear and disgust in Sydney (−2.14% for fear and −2.88% for disgust), Melbourne (−2.56% for fear and −2.19% for disgust), Brisbane (−2.30% for fear and −3.72% for disgust), Perth (−1.88% for fear and −2.09% for disgust) and Adelaide (−1.31% for fear and −4.22% for disgust). In contrast, the decrease of joy, anticipation and trust and the increase of fear and anger are observed in all capital cities (except Hobart) from the first wave to the second wave of the pandemic. From the second wave of the pandemic to the later phase of the pandemic, it is harder to generalise the changing pattern of optimistic and pessimistic mental health signals, given the mixed fluctuations of emotional types across capital cities.

Figure 4.

Change of emotion by type over four phases in eight capital cities.

Discussion

This study represents the first nationwide assessment of the public’s mental health signals in Australia. We find that the public’s mental health signals shifted from being relatively pessimistic in the early phase of the pandemic before March 2020 to being optimistic in the first wave of the pandemic and further more optimistic in the second wave of the pandemic, although a progressive recession of mental health signals was observed in the later phase of the pandemic after the mass COVID-19 vaccination started after March 2021, possibly associated with the concern and reluctance to vaccination. More specifically, the feeling of fear accounted for the largest proportion of emotion during the pandemic, mixed with joy, anticipation and trust. Compared with the early phase of the pandemic, the public’s mental health signals in the first wave of the pandemic increased toward optimistic (in particular joy, anticipation and trust) in all capital cities, most apparently in Perth and least apparently in Hobart. In the second wave of the pandemic, the public’s mental health signals decreased toward pessimistic (in particular fear and anger) most apparently in Darwin, followed by Adelaide and Sydney. Moreover, pessimistic mental health signals resurged slightly in most capital cities (except Perth and Hobart) in the later phase of the pandemic possibly due to the public concern and reluctance against vaccination.

A number of survey-based studies in Australia have attempted to evaluate the public’s mental health signals toward COVID-19, indicating that mental problems were widespread in the early stage of the pandemic (ie, increased psychological distress, health anxiety and contamination fears).4 11–14 It is in line with our findings to some degree that the public’s mental health signals tend to be pessimistic in the early phase before March 2020. The feeling of fear remains at a relatively higher level in the early phase and the first wave of the pandemic, coupled with the anxiety and sadness potentially associated with panic shopping, infection and life loss. What we add to the literature is the finding that the public’s mental health signals toward COVID-19 at the national level have been changed from being relatively pessimistic in the early phase to being optimistic in the first wave of the pandemic and further more optimistic in the later phases. This finding may reflect that the prompt actions and efforts made by governments to control virus spread and to build up public confidence and positivity toward the pandemic,27 releasing their worries and mental stresses to some degree.

With the advantage of social media data containing location information, we further contribute a spatial investigation of mental health signals across eight Australian capital cities by delineating the locales highly subject to pessimistic mental health signals. It is surprising to see that people in Melbourne where the second wave of the pandemic dominantly attacked have not experienced a decreased level of mental health signals, possibly to further speculate that the adverse effect of COVID-19 on the public’s mental health tends to diminish progressively to be marginal after multiple resurges that are relatively minor in scales and effects. In addition, pessimistic mental health signals emerge slightly in most capital cities (except Perth and Hobart) in the later phase of the pandemic when the mass vaccination started, possibly associated with public concern and reluctance to COVID-19 vaccines.28

Public health implications

We set out a novel empirical framework to systematically classify, measure and map the mental health signals of a nation, through which the role of public health policy and mental health services in face of the pandemic can be assessed. The provision of mental health services and the implementation of mental health policies need to adjust at different phases of the pandemic. Digital health platforms and diverse channels (eg, text messaging, mobile health applications, telehealth and telemedicine) to deliver mental health services need to be incorporated to guide through the public’s mental status.7 The key contribution of our study is for the delineation of when and where people are displaying higher levels of pessimistic mental health signals provides important information through which the allocation of finite mental health facilities and services can be deployed. Government and health authorities can use our empirical framework, supported by our methodological workflow shareable to the public (online supplemental appendix sections 2.5), to long-term track the public’s mental health in and beyond the pandemic period and to develop strategies and guidelines on mental health in face of future public health emergencies.

In response to the Australian’s vaccine hesitancy potentially associated with the resurge of pessimistic mental health signals in the later phase of the pandemic after 26 March 2021, healthcare providers are suggested to design and deliver more effective vaccine campaigns in (1) addressing the concerns of the side effects of approved vaccines,29 (2) encouraging the public to get vaccinated through multiple channels, including social media platforms (eg, Twitter), and (3) delivering educational information about vaccination in the vaccine-pessimistic locales identified in our study.

The results of this study should be interpreted in light of several important limitations. First is that Twitter users may not fully reflect the characteristics of the general population as noted by many studies.30 The elderly and those who have limited access to digital devices and social media are underrepresented in this study. Second, we need to acknowledge that sentiment and emotion detected by Twitter data are based on direct responses from Twitter users. Whether users’ genuine thoughts comply with what they express in tweets (ie, the trustworthiness and credibility) deserves further investigation. Third, given the complexity of human emotion, efforts are highly encouraged to extend from the eight types of emotion in our study to more types. Finally, we only analyse tweets written in English as English is the most spoken language in Australia. Future works need to consider incorporating multilinguistic tweets to improve representativeness when investigating non-English residents in Australia or other countries beyond. We call for future research to extend our empirical framework to investigate mental health of broader populations by using datasets with a higher representativeness (eg, mobile signal data) and containing multilanguages to render our findings comparable with observations in other geographic contexts. Our findings can also be complemented and calibrated by longitudinal survey data collected at various spatial scales where further endeavours can be made.

In a summary, our study unveils broad trends in the public’s mental health signals in Australia and across Australian capital cities throughout the different phases of the pandemic. More specifically, it offers a novel empirical framework to classify, measure, map and track the mental health signals of a nation in a manner that can readily be augmented into an ongoing monitoring capacity and extended to other nations through accessing our open-source analytical framework. Tracking locales where people are displaying elevated levels of pessimistic mental health signals provide important information for the smart deployment of finite mental health services. These new insights provide evidence for guiding public health policy and directing mental health services in the COVID-19 era and beyond.

Data availability statement

Data are available upon reasonable request. Twitter data at the aggregated level can be shared through direct requests to the author. The analytical workflow embedding all models used in this study can be accessed via 10.6084/m9.figshare.14985153. Users can run it via an open-source software, KNIME (https://www.knime.com/).

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study does not involve human participants.