Skip to main content
NCBI home page
Wellcome Open Research logoLink to Wellcome Open Research
. 2020 Jun 10;5:82. Originally published 2020 Apr 30. [Version 2] doi: 10.12688/wellcomeopenres.15870.2

Mapping population mental health concerns related to COVID-19 and the consequences of physical distancing: a Google trends analysis

Duleeka Knipe 1,a, Hannah Evans 2, Amanda Marchant 2, David Gunnell 1,3,#, Ann John 2,#
PMCID: PMC7331103  PMID: 32671230

Version Changes

Revised. Amendments from Version 1

To address the comments and suggestions made by the reviewers we have: 1.Conducted an additional sensitivity analysis to check whether the conclusions of the study would have been altered using an averaged dataset and added text to the methods and results sections 2.Added additional limitations to our study in the limitations section 3.Clarified in the methods how search topics were identified to be included in the analysis, how they were entered into the Google query, and why we did not include the search terms in different languages.

Abstract

Background: The 2020 Coronavirus pandemic is a major international public health challenge.  Governments have taken public health protection measures to reduce the spread of the virus through non-pharmalogical measures. The impact of the pandemic and the public health response on individual and population mental health is unknown. 

Methods: We used Google Trends data (1 Jan 2020 -  30 Mar 2020) to investigate the impact of the pandemic and government measures to curb it on people’s concerns, as indexed by changes in search frequency for topics indicating mental distress, social and economic stressors and mental health treatment-seeking. We explored the changes of key topics in Google trends in Italy, Spain, USA, UK, and Worldwide in relation to sentinel events during the pandemic.

Results: Globally there appears to be significant concerns over the financial and work-related consequences of the pandemic, with some evidence that levels of fear are rising. Conversely relative searching for topics related to depression and suicide fell after the pandemic was announced, with some evidence that searches for the latter have risen recently. Concerns over education and access to medication appear to be particular social stressors. Whilst searches for face-to-face treatments have declined, those for self-care have risen.

Conclusions: Monitoring Google trends shows promise as a means of tracking changing public concerns. In weeks to come it may enable policy makers to assess the impact of their interventions including those aiming to limit negative consequences, such as government funded financial safety nets.

Keywords: COVID-19, Suicide, Mental health, Pandemic, Economic, Depression, Anxiety, Coronavirus

The 2020 COVID-19 pandemic is the largest global public health challenge of this century, with over 200,000 deaths and almost three million people infected 1. In the absence of effective vaccines or treatments, governments worldwide are trying to contain the disease using public health measures including physical distancing and self-isolation. There are concerns that the pandemic and the public health response may adversely affect population mental health e.g. through job loss, debt and social isolation 2, 3. However, the general public’s reactions to this crisis and the impacts of these measures are unknown. A better understanding of concerns during this emergency may help us respond better to community need.

Google Trends is a publicly available data source of real-time internet search patterns, and has been used previously for health surveillance and research 4, 5. We aimed to use Google trends data to investigate the impact of the pandemic on peoples’ concerns and mental health, as indexed by changes in search frequency for terms indicating mental distress, social and economic stressors arising as a result of government measures to curb the epidemic and treatment seeking.

Methods

Data source

We used Google Trends for this analysis. We used the checklist previously suggested for documentation and development of our Google Trends analysis 6.

Google does not provide information on absolute numbers of searches. Instead it provides an indexed value to display search activity for a given term/topic at a specific period, time and area. This value is scaled from 0 to 100 with 100 representing the maximum searching activity for a particular term/topic in a given time period/area with search volumes for days/weeks/months given relative to this. Periods with very low search volumes are identified as zero activity

We identified key dates in relation to the pandemic from a number of sources ( Table 1). The date of the first death in each country was identified through news articles and as such are dates of first report, rather than date the death occurred. The date of first 10 deaths for each country was sourced from the World Health Organisation (WHO) daily situation reports. These dates may be subject to change where COVID-19 is identified post-mortem. The government enforced lockdown dates were also identified through news articles. In the USA where different states govern and enforce rules independently, the date for the first state-wide lock down was used, in this instance for California. WHO declared a global pandemic on 12 March 2020.

Table 1. Dates used in analysis of Google Trends.

Worldwide Italy Spain UK USA
Date of first death 09/01/2020 7 21/02/2020 8 13/02/2020 9 05/03/2020 10 28/02/2020 11
First 10 deaths 22/01/2020 12 26/02/2020 13 09/03/2020 14 14/03/2020 15 06/03/2020 16
First lockdown 09/03/2020 17 14/03/2020 18 23/03/2020 19 19/03/2020 20
Open in a new tab

Search strategy

Searches were carried out on 2 Apr 2020 to include all data available on that day (data available up until 30 Mar 2020) and included all query categories. We grouped the searches into four main themes: levels of mental distress, social stressors, economic stressors and treatment seeking. These were examined in Italy, Spain, USA, UK, and Worldwide. The countries were selected to represent locations with the largest numbers of COVID-19 deaths (Italy, Spain) or those predicted to experience large numbers of deaths but which were, at the time of analysis, earlier on the epidemic curve to investigate whether events occurring elsewhere in the world impacted on local concerns (UK, USA). We examined searches for the period between 1 Jan 2020 and 30 Mar 2020 to assess the immediate impact of the pandemic on population concerns. Google provides data during this period by day (90 data points for each country). In addition to this, we examined searches for the period 1 Jan 2019–30 Mar 2020 to assess whether any seasonal patterns in searching accounted for any of the changes seen in Jan–March 2020. Google provides search data summarised by week for this time period (65 data points for each country).

We explored the changes in trends of key topics in Google Trends. Topics are a group of related terms that share the same concept in any language. The use of topics in Google Trends negates the need for translating search terms. The chosen topics are outlined in Table 2 grouped into general themes. To indicate what search terms would be included in each topic, we looked at the top related search terms for each included topic ( Table 3). It is important to note that for some topics there are overlapping related search terms (e.g. the topics depression and anxiety both include the “anxiety” search term). Google trends allows a maximum of 5 topics to be displayed in a single graph. We selected five topics which showed stability of trends (as a marker of frequency) and which provided a meaningful addition to the broad themes we were exploring. We identified topics to be included in this analysis based on mental distress markers which we hypothesised would be associated with the pandemic, and stressors which were being discussed in the media, Government and WHO policy briefings, and preliminary review of concerns in relation to suicide 2 at the time of this analysis (March 2020). We also drew upon the experiences of the lead author (DK) who was acting as a UK community support volunteer during the outbreak, to select topics to explore under the themes of economic and social stressors. The topics explored under the treatment seeking theme were selected based on popular methods of treatment seeking. Previous analyses have highlighted that different data are provided for search terms entered with and without quotation marks in Google Trends 6, 21. As we are exploring search topics in Google Trends, as opposed to terms, the use of quotation marks is not applicable. A list of topics explored but not included are also summarised in Table 2 – the topics excluded either had very low indexed search values (i.e. close to 0) or were unstable over time. The simultaneous download of multiple topics allows for comparison of terms relative to each other.

Table 2. General themes and topic areas searched.

Mental distress Economic Stressors Social stressors Treatment seeking
Included in main search Depression Eviction Pharmacy Cognitive behavioural therapy
Anxiety Mortgage loan Education Self-care
Suicide Unemployment Abuse Counselling
Fear Food bank Alcohol drink Crisis hotline
Loneliness Welfare Divorce Mindfulness
Topics explored but not
included		 Suicidal ideation Job Argument Self-treatment
Stress Debt Domestic abuse Self-help
Mental health Universal credit Substance misuse Meditation
Benefits Legal separation
Bills Neglect
Job search Social care
Recession Child abuse
Domestic violence
Home schooling
Alcohol
Open in a new tab

Table 3. Topics included and the top 3 related searches in Google.

Included Topic Related search terms *
Depression Depression, depressed, anxiety
Anxiety Anxiety, anxious, angst
Suicide Suicide, suicidal, commit suicide
Fear Fear, scared, afraid
Loneliness Lonely, loneliness, lonely song
Eviction Eviction, evicted, evict
Mortgage loan Mortgage, mortgage calculator, home loan
Unemployment Welfare, social welfare, payment
Food bank Food bank, pantry, food pantry
Welfare Unemployment, unemployment insurance, unemployment rate
Pharmacy Pharmacy, Pharmacy near me, all-night drugstore
Education Education, school education, child education
Abuse Abuse, child abuse, abusive
Alcohol drink Alcohol, liquor, alcoholic
Divorce Divorce, divorced, how to divorce
Cognitive behavioural therapy CBT, cognitive, cognitive therapy
Self-care Self care, self care quotes, how to self care
Counselling Counselling, counsellor, counselling guidance
Crisis hotline Hotline, suicide, suicide hotline
Mindfulness Mindfulness, mindful, meditation
Open in a new tab

*Based on worldwide searches regardless of language and spelling errors were ignored. CBT – cognitive behavioural therapy

Previous analysis of Google Trends data has highlighted that slightly different relative search values are provided by Google for the same search (with the same parameters) on different days 21. This has been highlighted to be a problem when data for earlier years are downloaded and for smaller countries. As a sensitivity analysis we generated an averaged dataset for the topics included in this analysis. The averaged dataset was created by re-running the same Google Trends search query with the same parameters for time period (i.e. 1 Jan 2020 – 30 Mar 2020), location (e.g. USA), and each set of topics (e.g. mental distress topics) on 7 different days. We took the average value for each search topic (20 topics) on each of the 90 data points for all 5 settings, as it was recorded on each of the 7 separate days. We then correlated that 7-day average with values for the 90 days (1 Jan 2020–30 Mar 2020) used in the main analysis (which used values as they stood when the data were extracted on 2 Apr 2020). We checked to see whether the use of the averaged dataset would have altered our overall conclusions.

Data management and analysis

We provide graphical presentations of Google Trends data by country and themes explored for each time period (1 Jan 2019 – 30 Mar 2020, and 1 Jan 2020 – 1 Mar 2020). Key dates related to the pandemic are marked on graphs for the 1 Jan 2020 – 30 Mar 2020 period. In addition to the sensitivity analysis outlined in the previous section , we also removed the single topic that dominated some themes to investigate the remaining topics. STATA 16 was used to manage data and for creating graphs.

Results

We explored Google trends data for two time periods for each of the selected countries and worldwide for the selected themes. Trends in searches for specific topics worldwide and by country for 1 Jan 2019 to 30 Mar 2020 showed no evidence to suggest that changes during the current pandemic are similar to trends observed at the same time in the previous year (data not shown but available as underlying data 22).

Focusing on trends between 1 Jan 2020 to 30 Mar 2020, there appears to be growing world-wide concerns over the financial and work-related consequences of the pandemic, with some evidence that levels of fear are rising ( Figure 1). Conversely searching for topics related to depression and suicide fell after the pandemic was declared, with some evidence that searches for the latter have risen recently. Concerns over education and access to medication (pharmacy, Figure 1) appear to be particular social stressors. Whilst searches for face-to-face treatments (i.e. cognitive behavioural therapy (CBT), counselling) have declined, those for self-care have risen. We observed a “sawtooth pattern” in trends reflecting weekday versus weekend searching.

Figure 1. Trends in relative search volumes* of key topics worldwide between 1 Jan 2020 and 30 Mar 2020.

Figure 1.

Open in a new tab

A: Date of first Covid-19 related death; B: Date Covid-19 declared a global pandemic by WHO; *: Search activity displayed as a scaled value between 0 and 100 with 100 indicating the maximum searching for a given topic(s) over a given time period. The brief rise in suicide-searching mid-Feb coincides with the suicide of a celebrity (Caroline Flack, 15thFeb).

In the UK, USA, and Italy there is an indication that suicide related searches started to fall as the number of COVID-19 deaths started to rise, with searches starting to increase again after the lockdown was announced in each country ( Figure 2). In the UK the spike in suicide related Google searches in February 2020 coincided with the suicide death of TV presenter Caroline Flack (February 15 th 2020). Similar falls and rises in depression and loneliness-related Google searches were observed in the UK and Italy (respectively), whereas in other countries searches for the topic of depression and loneliness remained stable throughout the period. Levels of anxiety (as indicated by searches) were stable in all countries, with the exception of Spain, where anxiety related searches started to rise after lockdown. In the USA, levels of fear started to rise as the number of COVID-19 deaths increased.

Figure 2. Trends in relative search volumes of key topics indicating mental distress in the UK, USA, Italy and Spain between 1 Jan 2020 and 30 Mar 2020.

Figure 2.

Open in a new tab

Showing figures for daily searches. A: Date of first Covid-19 related death; B: Date of first 10 deaths; C: Date of lockdown.

Trends in searching for terms indicating economic concerns are shown in Figure 3. There was a marked increase in searches for mortgage loans towards the latter end of the time series in the UK, Italy and USA, which coincides with the rising number of COVID-19 related deaths. It is noteworthy that these rises in the UK and USA pre-dated lockdown, indicating people were predicting this might happen or were feeling the impact of behavioural changes on their businesses (e.g. fewer customers in restaurants / cafes). Unemployment-related searches rose sharply just before lockdown in the USA and after lockdown in Italy and Spain. In the UK there is an indication that searches related to unemployment and food banks started to rise as the number of COVID-19 deaths started to increase.

Figure 3. Trends in relative search volumes of key topics indicating economic stressors in the UK, USA, Italy and Spain between 1 Jan 2020 and 30 Mar 2020.

Figure 3.

Open in a new tab

Showing figures for daily searches. A: Date of first Covid-19 related death; B: Date of first 10 deaths; C: Date of lockdown.

The specific social stressors that appear to have risen at the same time as the COVID-19 related deaths were divorce in Italy, pharmacy (as a marker of access to medication) in the UK, and education in the UK, Italy and Spain ( Figure 4). Abuse and divorce related searches remained stable in the UK, USA, and Spain. In Spain pharmacy related searches started to increase after lockdown was announced. In more recent days (after lockdown) there appears to be a rise in the number of Google searches for alcoholic drinks.

Figure 4. Trends in relative search volumes of key topics indicating social stressors in the UK, USA, Italy and Spain between 1 Jan 2020 and 30 Mar 2020.

Figure 4.

Open in a new tab

Showing figures for daily searches. A: Date of first Covid-19 related death; B: Date of first 10 deaths; C: Date of lockdown.

In the UK and Italy searches for cognitive behavioural therapy started to decline as these countries went into lockdown, potentially relating to difficulties accessing face-to-face care. Counselling searches also declined in Italy, but this decline appears to predate the first death due to COVID-19. In the USA, whilst searches for the key topics for treatment seeking remained stable, searches related to self-care started to rise after the lockdown was enforced ( Figure 5).

Figure 5. Trends in relative search volumes of key topics indicating treatment seeking in the UK, USA, Italy and Spain between 1 Jan 2020 and 30 Mar 2020.

Figure 5.

Open in a new tab

Showing figures for daily searches. A: Date of first Covid-19 related death; B: Date of first 10 deaths; C: Date of lockdown.

As a sensitivity analysis we compared the dataset downloaded and used in our main analysis with a dataset of averaged relative search values from datasets downloaded on 7 different days. The two datasets were well correlated for the majority of search topics in most countries ( Table 4), with the exception of topics in countries with low relative search values for that topic (typically below 10 - e.g. food bank in Italy and Spain (relative search values varied from 0 to 1)), and less populous countries. The use of the averaged dataset, however, did not alter our overall conclusions.

Table 4. Correlation between single day verses seven-day averaged Google Trend data.

Worldwide UK USA Spain Italy
Depression 0.99 0.96 0.98 0.89 0.64
Anxiety 0.98 0.87 0.95 0.93 0.81
Suicide 0.99 0.99 0.98 0.96 0.97
Fear 0.99 0.85 0.99 1.00 0.95
Loneliness 0.97 0.61 0.68 0.35 1.00
Eviction 0.99 0.91 0.90 0.68 0.84
Welfare 0.98 0.82 0.90 0.56 0.88
Mortgage Loan 1.00 1.00 1.00 0.53 0.99
Unemployment 1.00 0.97 1.00 1.00 0.99
Food Bank 0.91 0.99 0.97 0.54 0.69
Divorce 0.96 0.93 0.98 0.80 0.90
Abuse 0.93 0.87 0.95 0.65 0.53
Alcohol 1.00 1.00 1.00 0.94 1.00
Education 1.00 0.99 1.00 1.00 0.97
Pharmacy 0.99 0.99 0.98 0.99 1.00
CBT 0.98 0.93 0.96 0.59 0.77
Selfcare 0.95 0.64 0.89 0.40 0.59
Counselling 0.99 0.48 0.85 0.86 0.93
Crisis Hotline 0.73 0.47 0.77 0.49 0.45
Mindfulness 0.97 0.91 0.95 0.77 0.73
Open in a new tab

*complete correlation between two datasets is expressed as -1 or 1

Conclusions

Using near real-time search data from Google Trends, we show that there are individual concerns about work (unemployment) and the economic (mortgage loans and food banks) consequences of the pandemic around the world. There is an indication that relative searches related to mental distress (depression and suicide) started to fall as the number of COVID-19 deaths started to rise, with evidence that suicide related searches have started to increase after the initiation of lockdown. Searches for education and pharmacy (a marker of medication access) have started to rise. As face-to-face access to treatment declined in countries as a consequence of lockdowns, so did searches for cognitive behavioural therapy and counselling, whilst searches related to self-care increased.

The importance of considering population level psychological well-being and mental health during the COVID-19 pandemic has been recognised by the World Health Organisation 23. Changes in online searching behaviour assessed through Google trends has been used previously to forecast suicide occurrences in the UK and Ireland 24, 25. A greater number of searches for the term ‘depression’ was related to fewer suicides, whereas a greater number of searches for the term ‘suicide’ was related to more suicides which may have implications for the trends we found. However, a study in Australia did not find Google Trends to be a straightforward surveillance tool for monitoring suicide 26. The latter study found a lack of seasonality and only limited evidence of an association between searches related to suicide and unemployment levels. Increased Google searches related to psychological distress were associated with increases in defaulted mortgage payments, and unemployment after the 2008 recession in the USA 27.

Evidence from our study showing levels of mental distress are rising during the COVID-19 pandemic are consistent with emerging findings from Italy and the UK 28– 30. There is evidence that the impact of the public health measures are affecting different groups in different ways, with increased loneliness being reported in the elderly and higher levels of anxiety in those with poorer health 29, 30. Concerns related to loss of income, and practical challenges related to accessing food and shopping associated to the pandemic were also observed in other studies 29– 31.

In a global crisis like the COVID-19 pandemic, real time infection surveillance is a key priority in order to ensure the public health measures being introduced are effective in curbing the spread of the disease. Of equal importance is ensuring that the measures being implemented do not have unintended consequences, especially in relation to mental health and that public health messaging is tailored to evolving concerns. In Italy, Barari et al. (2020) recommended, based on a rapid representative survey, that communications should move from ‘stay at home’ to what the population could do while at home to ensure adherence to public health measures over time. In order to do this, real time data on individual concerns needs to be monitored, but this is difficult to do. We have shown that Google Trends could be utilised to monitor public concerns related to the pandemic. Alternative sources of data include frequently repeated (e.g. monthly) linked cross-sectional surveys of the general population, but these may be costly to implement and may provide information in a less timely fashion. In some countries (e.g. UK) there have been efforts to limit any negative impact on people’s finances through, for example, government funded financial safety nets – it is still too early to see whether these intervention strategies will have an impact on individual concerns.

Whilst there are several strengths to using this approach for monitoring individual concerns, there are limitations to this method. First, the approach requires individuals to have access to the internet and to be actively engaging in searching. This may, therefore, not represent the total population and may exclude important vulnerable groups (e.g. the elderly and those in resource poor settings). Second, there is no way of knowing who is searching for these topics. We tried to only include topics which would relate to individual concerns (e.g. unemployment versus recession). However, it has been noted previously, that Google Trends may not be valid for behavioural forecasting since we cannot ascertain who searches for certain terms and why 21. We were also unable to determine the sociodemographic characteristics of those conducting searches. Third, the format in which Google trends data are available for download does not allow us to estimate actual search volumes, although an indication of this can be gained by the relative stability of trend lines. In addition, trends may be affected by overall search volumes. If there is a surge in overall Google searching (e.g. during a global pandemic) this may push down the trend lines for other indicators that have remained constant in terms of search volume 32. Fourth, we present a descriptive analysis of Google trends data, and a statistical analysis which tests for differences in search activity before and after sentinel dates (i.e. an interrupted time-series analysis) may have strengthened our study. There were, however, at the time of this analysis limited data points during the post-outbreak period, which may have led to this type of analysis being underpowered. Since we were assessing whether it would be meaningful to use Google Trends data as a possible surveillance tool for monitoring public concerns related to the pandemic in near real-time we felt it was important to produce a timely responsive exploration of the data. Future studies, after more time has elapsed, should investigate differences in search activity using interrupted time-series. Lastly, it is unclear exactly how each Google topic is constructed and what search terms would be included. In order to ensure we used appropriate topics we checked that the most popular related queries/searches to our included topic accurately reflected the concept we were intending to investigate.

Despite these limitations, monitoring Google trends shows promise as a means of tracking changing public concerns and in weeks to come may enable policy makers to assess the impact of their interventions.

Data availability

Source data

All data are freely available directly from Google Trends ( https://trends.google.com/trends/)

Underlying data

Open Science Framework: Mapping population mental health concerns related to COVID-19 and the consequences of physical distancing: a Google trends analysis. https://doi.org/10.17605/OSF.IO/UNCW2 22

This project contains the following underlying data:

  • averages2020FINAL.xls (Averaged dataset for sensitivity analysis)

  • coviddata2020mainFINAL.xls (Combined downloaded data from Google Trends (2020))

  • coviddatamainFINAL.xls (Combined downloaded data from Google Trends (2019–2020))

Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

Funding Statement

This work was supported by the Wellcome Trust through an Institutional Strategic Support Fund Award to the University of Bristol [204813] which supports DK. DK is also supported through the Elizabeth Blackwell Institute for Health Research, University of Bristol.

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

[version 2; peer review: 2 approved]

References

  • 1. Worldometers: COVID-19 Coronavirus Pandemic. 2020. Reference Source [Google Scholar]
  • 2. Gunnell D, Appleby L, Arensman E, et al. : Suicide risk and prevention during the COVID-19 pandemic. Lancet Psychiatry. 2020; pii: S2215-0366(20)30171-1. 10.1016/S2215-0366(20)30171-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Holmes E, O’Connor R, Perry VH, et al. : Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science. Lancet Psychiatry. 2020; pii: S2215-0366(20)30168-1. 10.1016/S2215-0366(20)30168-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4. Arora VS, McKee M, Stuckler D: Google Trends: Opportunities and limitations in health and health policy research. Health Policy. 2019;123(3):338–41. 10.1016/j.healthpol.2019.01.001 [DOI] [PubMed] [Google Scholar]
  • 5. Gunnell D, Derges J, Chang SS, et al. : Searching for Suicide Methods: Accessibility of Information About Helium as a Method of Suicide on the Internet. Crisis. 2015;36(5):325–31. 10.1027/0227-5910/a000326 [DOI] [PubMed] [Google Scholar]
  • 6. Nuti SV, Wayda B, Ranasinghe I, et al. : The use of google trends in health care research: a systematic review. PLoS One. 2014;9(10):e109583. 10.1371/journal.pone.0109583 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7. Guardian T: First death from China mystery illness outbreak. 2020. Reference Source [Google Scholar]
  • 8. La Repubblica: Adriano Trevisan, the first coronavirus victim in Italy. 2020. Reference Source [Google Scholar]
  • 9. RTVE: The Valencian Community confirms the first death of a patient with a coronavirus in Spain on February 132020. 2020. Reference Source [Google Scholar]
  • 10. Guardian T: Patient with underlying health conditions becomes first fatality in UK – as it happened. 2020. Reference Source [Google Scholar]
  • 11. Guardian T: First coronavirus death in the US as Italy and France cancel public events. 2020. Reference Source [Google Scholar]
  • 12. World Health Organisation: Novel Coronavirus (2019-nCoV) SITUATION REPORT - 3.2020. Reference Source [Google Scholar]
  • 13. World Health Organisation: Coronavirus disease 2019 (COVID-19) Situation Report – 37.2020. Reference Source [Google Scholar]
  • 14. World Health Organisation: Coronavirus disease 2019 (COVID-19) Situation Report – 49.2020. Reference Source [Google Scholar]
  • 15. World Health Organisation: Coronavirus disease 2019 (COVID-19) Situation Report – 54.2020. Reference Source [Google Scholar]
  • 16. World Health Organisation: Coronavirus disease 2019 (COVID-19) Situation Report – 46.2020. Reference Source [Google Scholar]
  • 17. BBC: Coronavirus: Italy extends emergency measures nationwide.2020. Reference Source [Google Scholar]
  • 18. National Post: Spain to impose nationwide lockdown - El Mundo.2020. Reference Source [Google Scholar]
  • 19. Guardian T: Boris Johnson orders UK lockdown to be enforced by police.2020. Reference Source [Google Scholar]
  • 20. Financial Times: California goes into lockdown in battle with coronavirus. 2020. Reference Source [Google Scholar]
  • 21. Tran US, Andel R, Niederkrotenthaler T, et al. : Low validity of Google Trends for behavioral forecasting of national suicide rates. PLoS One. 2017;12(8):e0183149. 10.1371/journal.pone.0183149 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Knipe D, Evans H, Marchant A, et al. : Mapping population mental health concerns related to COVID-19 and the consequences of physical distancing: a Google trends analysis.2020. 10.17605/OSF.IO/D36X8 [DOI] [PMC free article] [PubMed]
  • 23. World Health Organisation: Mental health and psychosocial considerations during the COVID-19 outbreak. 2020. Reference Source [Google Scholar]
  • 24. Kristoufek L, Moat HS, Preis T: Estimating suicide occurrence statistics using Google Trends. EPJ Data Science. 2016;5(1):32 10.1140/epjds/s13688-016-0094-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25. Barros JM, Melia R, Francis K, et al. : The Validity of Google Trends Search Volumes for Behavioral Forecasting of National Suicide Rates in Ireland. Int J Environ Res Public Health. 2019;16(17): pii: E3201. 10.3390/ijerph16173201 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26. Page A, Chang SS, Gunnell D: Surveillance of Australian suicidal behaviour using the internet? Aust N Z J Psychiatry. 2011;45(12):1020–2. 10.3109/00048674.2011.623660 [DOI] [PubMed] [Google Scholar]
  • 27. Ayers JW, Althouse BM, Allem JP, et al. : Novel surveillance of psychological distress during the great recession. J Affect Disord. 2012;142(1–3):323–30. 10.1016/j.jad.2012.05.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. consortium CP: Intial research findings on COVID-19 and mental health in the UK.2020. Reference Source [Google Scholar]
  • 29. Barari S, Caria S, Davola A, et al. : Evaluating COVID-19 Public Health Messaging in Italy: Self-Reported Compliance and Growing Mental Health Concerns. medRxiv. 2020. 10.1101/2020.03.27.20042820 [DOI] [Google Scholar]
  • 30. Sciences: TAoM: Survey results: Understanding people’s concerns about the mental health impacts of the COVID-19 pandemic2020.2020. Reference Source [Google Scholar]
  • 31. Yilmazkuday H: COVID-19 Effects on the U.S. Unemployment: Nationwide and State-Level Evidence from Google Trends.2020. Reference Source [Google Scholar]
  • 32. Churick N: How to Use Google Trends for Keyword Research: 7 Effective Ways.2019. Reference Source [Google Scholar]
Wellcome Open Res. 2020 Jul 1. doi: 10.21956/wellcomeopenres.17574.r39018

Reviewer response for version 2

Olivia J Kirtley 1

Many thanks to the authors for their thoughtful and thorough responses to mine and the other reviewer's comments. I am satisfied that my queries regarding the manuscript have been addressed by the authors' amendments.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

Psychology; suicide and self-harm research; theoretical models of suicidal behaviour.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Wellcome Open Res. 2020 Jun 11. doi: 10.21956/wellcomeopenres.17574.r39017

Reviewer response for version 2

John Gunn 1

I am very satisfied with the adjustments to the manuscript as well as the authors responses to my suggestions. Overall I think this is a strong piece that highlights the potential for utilizing search trends as a tool for monitoring potential risk at a macro-level.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Partly

Are the conclusions drawn adequately supported by the results?

Partly

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Suicide, Theories of Suicide, Risk and Protective Factors for Suicide Across the Lifespan, Impact of Media on Suicide.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Wellcome Open Res. 2020 May 27. doi: 10.21956/wellcomeopenres.17407.r38594

Reviewer response for version 1

Olivia J Kirtley 1

Thank you for the invitation to review this manuscript.

The study utilised Google Trends data in order to provide insights into public concerns during COVID-19, across four specific countries (UK, USA, Italy and Spain), as well as worldwide. Search terms investigated fall within four areas: mental distress; economic stressors; social stressors and treatment seeking.

I am really pleased to see underlying data being shared via the Open Science Framework. Open science practices are underused within suicide research and clinical psychology/psychiatry research more generally, so this is encouraging to see. The authors have provided a healthy critique of Google Trends data within their conclusions section and are realistic about what these data can and cannot tell us.

I have a few comments and suggestions, which I hope will assist the authors in strengthening their manuscript.

Methods:

  1. I would like to see some further information provided regarding how search terms were chosen and entered into Google Trends, as this was not 100% clear to me from reading the description in the paper.

  2. Additionally, as Nuti et al. (2014) and Tran et al. (2017) both pointed out that different results can be obtained for the same search terms entered with and without quotation marks, it would also be useful for the authors to report in the methods section whether quotation marks were used.

    For the manuscript to be fully sound, points 1 and 2 should be addressed in the interests of replicability. 

  3. Again, relating to the search terms, can the authors please explain why search terms in relevant other languages, i.e. Spanish and Italian, were not used? If I understand correctly from the methods, only English language search terms were employed. In Tran et al.'s (2017) study, German search terms were used when investigating Google Trends data for Germany, Switzerland and Austria. Only including English language search terms may mean that generalisability of the findings is limited. 

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Not applicable

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Yes

Are the conclusions drawn adequately supported by the results?

Yes

Are sufficient details of methods and analysis provided to allow replication by others?

Partly

Reviewer Expertise:

Psychology; suicide and self-harm research; theoretical models of suicidal behaviour.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Wellcome Open Res. 2020 Jun 3.
Duleeka Knipe 1

We thank the reviewer for taking the time to review our manuscript. We have responded to each point raised by the reviewer below:

1. I would like to see some further information provided regarding how search terms were chosen and entered into Google Trends, as this was not 100% clear to me from reading the description in the paper.

Thank you for highlighting this point. We have added the following text into the methods section of the manuscript to clarify how we selected the search topics we used in our analysis:

“ We identified topics to be included in this analysis based on mental distress markers which we hypothesised would be associated with the pandemic, and stressors which were being discussed in the media, Government and WHO policy briefings, and preliminary review of concerns in relation to suicide (1) at the time of this analysis (March 2020). We also drew upon the experiences of the lead author (DK) who was acting as a UK community support volunteer during the outbreak, to select topics to explore under the themes of economic and social stressors. The topics explored under the treatment seeking theme were selected based on popular methods of treatment seeking.” 

2. Additionally, as Nuti  et al. (2014) and Tran  et al. (2017) both pointed out that different results can be obtained for the same search terms entered with and without quotation marks, it would also be useful for the authors to report in the methods section whether quotation marks were used.

The reviewer raises an important point regarding the use of quotation marks. In our search strategy we did not use quotation marks because we were searching with Google topics (i.e. categories that Google assigns based on search terms that have similar meaning – see table 3). The addition of quotation marks means that Google Trends will provide data on search terms rather than topics. We chose to search for topics as opposed to terms, because the former allows for the comparison of concepts that are similar across countries. We have added the following text to the methods section of the manuscript:

“ Previous analyses have highlighted that different data are provided for search terms entered with and without quotation marks in Google Trends (1, 2) . As we are exploring search topics in Google Trends, as opposed to terms, the use of quotation marks is not applicable.”

3. Again relating to the search terms, can the authors please explain why search terms in relevant other languages, i.e. Spanish and Italian, were not used? If I understand correctly from the methods, only English language search terms were employed. In Tran  et al.'s (2017) study, German search terms were used when investigating Google Trends data for Germany, Switzerland and Austria. Only including English language search terms may mean that generalisability of the findings is limited. 

We thank the reviewer for this comment. The search strategy used in this analysis is different to that used in previous analysis of Google Trends which employed search terms in each of the different languages of the countries included in their studies. This approach requires a detailed knowledge of each of the languages included to ensure that the concepts that are being explored are adequately captured based on the search terms used. This was part of the rationale for choosing to use Google Trend topics as opposed to search terms. Google topics are a group of search terms identified by Google to share similar meanings across languages/countries. We have added additional text to the methods section (underlined):

“We explored the changes in trends of key topics in Google Trends. Topics are a group of related terms that share the same concept in any language. The use of topics in Google Trends negates the need for translating search terms.”

Wellcome Open Res. 2020 May 12. doi: 10.21956/wellcomeopenres.17407.r38590

Reviewer response for version 1

John Gunn 1

This is a timely and important work that explores the trends in google searches related to mental health in the wake of the COVID-19 pandemic. While the work is valuable, there are two main points I would like to see the authors address:

  1. While the authors acknowledge the flaws in google/trends analyses that are brought out from Tran et al. (2017) – they do not address the lack of reliability in google/trends data. As indicated, data was collected on a single day, 2 April 2020. However, google/trends data scores vary day-to-day. In order to overcome this shortcoming, Tran et al. (2017) recommend collecting trend data each day of a 7-day period. A final score can be computed by average across these 7-days, thus improving upon the poor reliability.

  2. I would also be curious what the rationale was for providing descriptions of the trends, but exploring them statistically? Much of the google/trends work I’ve seen compares a time point prior to and following an event to determine if there are statistically significant differences. For example, do the searches in the weeks prior to lockdown differ significantly from the searches in the weeks following? Do these months of 2020 differ significantly from the same period in time in 2019, 2018, 2017,…?  I think this would add to the strength of this piece – or in the least should be discussed as a limitation with a rationale provided.

The above concerns are what drove my responses of “partly” to the questions of whether the study design is appropriate and technically sound and whether or not the conclusions drawn are adequately supported by the results. For this manuscript to be sound, I would expect item 1 to be addressed. While I think it would be useful for the authors to provide a statistical analysis per point 2, acknowledgment of this limitation and a rationale for why they did a descriptive analysis would suffice.

Is the work clearly and accurately presented and does it cite the current literature?

Yes

If applicable, is the statistical analysis and its interpretation appropriate?

Yes

Are all the source data underlying the results available to ensure full reproducibility?

Yes

Is the study design appropriate and is the work technically sound?

Partly

Are the conclusions drawn adequately supported by the results?

Partly

Are sufficient details of methods and analysis provided to allow replication by others?

Yes

Reviewer Expertise:

Suicide, Theories of Suicide, Risk and Protective Factors for Suicide Across the Lifespan, Impact of Media on Suicide.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Wellcome Open Res. 2020 Jun 3.
Duleeka Knipe 1

We thank the reviewer for taking the time to review our manuscript. We have responded to each point raised by the reviewer below:

1. While the authors acknowledge the flaws in google/trends analyses that are brought out from Tran  et al. (2017) – they do not address the lack of reliability in google/trends data. As indicated, data was collected on a single day, 2 April 2020. However, google/trends data scores vary day-to-day. In order to overcome this shortcoming, Tran  et al. (2017) recommend collecting trend data each day of a 7-day period. A final score can be computed by average across these 7-days, thus improving upon the poor reliability.

We thank the reviewer for this valuable point. Tran et al (2017) do highlight in their paper that research should ideally use averaged relative search values. However, they also state that the “slightly” different search values retrieved on different days were more apparent for earlier years than latter years. In addition, they found the differences to be a particular problem for small countries (i.e. Austria/Switzerland, population size ~9 million). Our analysis is based on data from populous countries (populations >50 million) and includes data points from the start of 2020 so it is likely the reliability issues highlighted in the paper will not be applicable to our analysis. However to reassure the reviewer and ourselves, we conducted the data download of the data used in this analysis over a 7 day period and compared the 7-day averaged daily relative search values (RSVs) with those used in our original analysis. We find that for the majority of search topics there was a strong correlation between the two datasets. For topics where there were low search volumes and therefore low daily RSVs (generally less than 10), the two datasets were less strongly correlated. We have added the following text to the manuscript:  

METHODS

“Previous analysis of Google Trends data has highlighted that slightly different relative search values are provided by Google for the same search (with the same parameters) on different days  (3) . This has been highlighted to be a problem when data for earlier years are downloaded and for smaller countries. As a sensitivity analysis we generated an averaged dataset for the topics included in this analysis. The averaged dataset was created by re-running the same Google Trends search query with the same parameters for time period (i.e. 1 Jan 2020 – 30 Mar 2020), location (e.g. USA), and each set of topics (e.g. mental distress topics) on 7 different days. We took the average value for each search topic (20 topics) on each of the 90 data points for all 5 settings, as it was recorded on each of the 7 separate days. We then correlated that 7-day average with values for the 90 days (1 Jan 2020- 30 Mar 2020) used in the main analysis (which used values as they stood when the data were extracted on 2 Apr 2020). We checked to see whether the use of the averaged dataset would have altered our overall conclusions.”

RESULTS

"As a sensitivity analysis we compared the dataset downloaded and used in our main analysis with a dataset of averaged relative search values from datasets downloaded on 7 different days. The two datasets were well correlated for the majority of search topics in most countries (Table 4), with the exception of topics in countries with low relative search values for that topic (typically below 10 - e.g. food bank in Italy and Spain (relative search values varied from 0 to 1)), and less populous countries. The use of the averaged dataset, however, did not alter our overall conclusions. "

2. I would also be curious what the rationale was for providing descriptions of the trends, but exploring them statistically? Much of the google/trends work I’ve seen compares a time point prior to and following an event to determine if there are statistically significant differences. For example, do the searches in the weeks prior to lockdown differ significantly from the searches in the weeks following? Do these months of 2020 differ significantly from the same period in time in 2019, 2018, 2017,…?  I think this would add to the strength of this piece – or in the least should be discussed as a limitation with a rationale provided.

The reviewer raises an important point regarding our analysis strategy. One of the reasons for conducting this study was to see whether it would be meaningful to use Google Trends data as a possible surveillance tool for monitoring public concerns related to the pandemic in near real-time. Based on this we were keen to provide a first look at the data quickly in response to the crisis. We discussed the possibility of conducting an interrupted time-series analysis but felt that there would not have been sufficient data points during the ‘after’ period to conduct a meaningful analysis. We have added the following text to the limitations of our manuscript:

“ Fourth, we present a descriptive analysis of Google trends data, and a statistical analysis which tests for differences in search activity before and after sentinel dates (i.e. an interrupted time-series analysis) may have strengthened our study. There were, however, at the time of this analysis limited data points during the post-outbreak period, which may have led to this type of analysis being underpowered. Since we were assessing whether it would be meaningful to use Google Trends data as a possible surveillance tool for monitoring public concerns related to the pandemic in near real-time we felt it was important to produce a timely responsive exploration of the data. Future studies, after more time has elapsed, should investigate differences in search activity using interrupted time-series. ”   

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Data Citations

    1. Knipe D, Evans H, Marchant A, et al. : Mapping population mental health concerns related to COVID-19 and the consequences of physical distancing: a Google trends analysis.2020. 10.17605/OSF.IO/D36X8 [DOI] [PMC free article] [PubMed]

    Data Availability Statement

    Source data

    All data are freely available directly from Google Trends ( https://trends.google.com/trends/)

    Underlying data

    Open Science Framework: Mapping population mental health concerns related to COVID-19 and the consequences of physical distancing: a Google trends analysis. https://doi.org/10.17605/OSF.IO/UNCW2 22

    This project contains the following underlying data:

    • averages2020FINAL.xls (Averaged dataset for sensitivity analysis)

    • coviddata2020mainFINAL.xls (Combined downloaded data from Google Trends (2020))

    • coviddatamainFINAL.xls (Combined downloaded data from Google Trends (2019–2020))

    Data are available under the terms of the Creative Commons Zero "No rights reserved" data waiver (CC0 1.0 Public domain dedication).

    Articles from Wellcome Open Research are provided here courtesy of The Wellcome Trust

    RESOURCES