Skip to main content
PMC home page
PLOS ONE logoLink to PLOS ONE
. 2020 Jul 24;15(7):e0236777. doi: 10.1371/journal.pone.0236777

Google searches for suicide and suicide risk factors in the early stages of the COVID-19 pandemic

Emily A Halford 1, Alison M Lake 1, Madelyn S Gould 1,2,3,*
Editor: Neal Doran4
PMCID: PMC7380602  PMID: 32706835

Abstract

A novel coronavirus (SARS-CoV-2), which causes the COVID-19 respiratory illness, emerged in December of 2019 and has since spread globally. The dramatic lifestyle changes and stressors associated with this pandemic pose a threat to mental health and have the potential to exacerbate risk factors for suicide. We used autoregressive integrated moving average (ARIMA) models to assess Google Trends data representing searches in the United States for 18 terms related to suicide and known suicide risk factors following the emergence of COVID-19. Although the relative proportion of Google searches for suicide-related queries was lower than predicted during the early pandemic period, searches for the following queries representative of financial difficulty were dramatically elevated: “I lost my job” (226%; 95%CI, 120%-333%), “laid off” (1164%; 95%CI, 395%-1932%), “unemployment” (1238%; 95%CI, 560%-1915%), and “furlough” (5717%; 95%CI, 2769%-8665%). Searches for the Disaster Distress Helpline, which was promoted as a source of help for those impacted by COVID-19, were also remarkably elevated (3021%; 95%CI, 873%-5169%). Google searches for other queries representative of help-seeking and general mental health concerns were moderately elevated. It appears that some indices of suicidality have fallen in the United States in this early stage of the pandemic, but that COVID-19 may have caused an increase in suicide risk factors that could yield long-term increases in suicidality and suicide rates.

Introduction

A novel coronavirus (SARS-CoV-2), which causes the COVID-19 respiratory illness, emerged in Wuhan, China in December of 2019. On February 29, 2020, the first death attributed to COVID-19 within the United States occurred in Washington State.

As the virus has spread, the public health community has been rightfully focused on strained hospital systems and the rising death toll. However, COVID-19 poses a serious threat to mental health as well. Social distancing is one of the primary measures implemented to slow the virus’s spread, and while necessary to protect the public from COVID-19, this practice may have detrimental secondary effects on unemployment, loneliness, and pre-existing mental illnesses, all of which are known risk factors for suicide [1–5]. Additionally, the virus itself has the potential to exacerbate other suicide risk factors through a national sense of anxiety, grief as the virus claims more lives, and increased stress levels for essential workers [5, 6].

The aim of this analysis is to evaluate the impact of COVID-19 on suicidality in the United States during the pandemic’s early stages, as well as the virus’s effect on risk factors which might predict long-term impacts on suicide. Google Trends data was chosen for this study because it is far more rapidly available than alternative data sources such as mortality data, and because significant evidence exists that establishes an association between Google search behavior and actual suicidal behavior both within the United States [7–10] and internationally [11–16]. In fact, Google searches for “how to suicide,” “how to kill yourself,” and “painless suicide” were found to be better predictors of completed suicides in the United States than traditional self-reported measures of suicide risk [9]. Associations have also been found between searches for mental health diagnoses and other known risk factors for suicide and actual suicide rates [8, 11–13]. Searches for queries representing unemployment are frequently associated with suicide [8, 10, 11] and are of particular relevance to the present study given the far-reaching economic damage caused by the COVID-19 pandemic. Therefore, we assessed Google Trends data representing searches for terms related to suicide and known suicide risk factors following the emergence of COVID-19 in the United States with the aim of providing insight into how the early stages of this pandemic may have impacted suicidality and suicide rates.

Methods

Query selection

There were four primary domains which were examined due to their known associations with suicide risk [5]: suicide-specific queries, help-seeking queries, general mental health queries, and queries related to financial difficulty. We did not try to be exhaustive by including every possible search term that a user may have utilized within each category, but rather aimed to have 3 to 6 queries representative of each domain.

Queries were selected for the suicide-specific, general mental health, and financial difficulty categories based upon a review of the literature and use of the “Related Queries” tool available on the Google Trends interface. These finalized categories are as follows: suicide-specific queries (“suicide–squad” [11, 13, 14, 16], “kill myself,” “kill yourself,” “suicide methods + suicide method,” “commit suicide” [7], and “fast suicide + easy suicide + quick suicide + painless suicide” [9]), general mental health queries (“depression” [11], “panic attack,” and “anxiety” [8, 11–13), queries related to financial difficulty (“unemployment” [10, 11], “I lost my job,” “laid off” [8], “furlough” (a mandatory temporary leave for employees), and “loan”). The uncategorized query “loneliness” was also included to assess possible impacts of the social distancing interventions implemented to slow the spread of COVID-19.

Queries for the help-seeking category were intended to represent the two most prominent and widely-used crisis services in the United States (National Suicide Prevention Lifeline, the national network of local crisis centers, https://suicidepreventionlifeline.org/; Crisis Text Line, the global organization providing crisis intervention over text message, https://www.crisistextline.org/), as well as the Disaster Distress Helpline, a national hotline providing crisis counseling for those impacted by any natural or human-caused disaster that has been promoted during the COVID-19 pandemic (https://www.samhsa.gov/find-help/disaster-distress-helpline). This finalized category contains the following queries: “crisis text line,” “national suicide prevention lifeline + national suicide hotline,” and “disaster distress helpline + disaster distress hotline.”

Google Trends (https://trends.google.com/trends/?geo=US) allows queries containing certain words to be excluded from the relative search proportion using the “-”sign, and searches for the popular movie Suicide Squad were thus excluded from our “suicide” search query by requesting data for “suicide–squad” [17]. We considered using this feature to filter out searches for “depression” that included “economy,” “economic,” and “are we going into a” to help ensure that we captured searches related to emotional depression rather than an economic depression. However, doing so made no meaningful change in observed search volume so we decided to analyze the simple “depression” query used in three earlier studies [11, 13, 14]. Google trends also allows for the relative search volume associated with several search queries to be combined using the “+” sign. This tool was used to yield a combined relative search proportion for common variations on terms or conceptually linked terms, resulting in the following queries: “national suicide prevention lifeline + national suicide hotline,” “disaster distress helpline + disaster distress hotline,” “suicide methods + suicide method,” and “fast suicide + easy suicide + quick suicide + painless suicide.”

Data

The gtrendsR package in R was used to retrieve weekly search data from March 3, 2019 through April 18, 2020 for 14 of the 18 search queries. The current version of the gtrendsR package lacks the additive “+” feature, so data associated with the following 4 queries were obtained directly from the Google Trends website (https://trends.google.com/trends/?geo=US): “national suicide prevention lifeline + national suicide hotline,” “disaster distress helpline + disaster distress hotline,” “suicide methods + suicide method,” and “fast suicide + easy suicide + quick suicide + painless suicide.”

Data for each query is not an absolute number of searches or an absolute proportion; rather data for each query is rescaled to represent a relative search proportion. Weekly search volume for a given query is first divided by total Google search volume for that week. The proportions are then scaled from 0 to 100, with 100 assigned to the week within the designated data collection period (in this case, March 3, 2019-April 18, 2020) and specified location (in this case, the United States) when the given query's proportion of the total search volume was greatest. A relative proportion of 25, for example, indicates a proportion 25% as high as the given query’s highest relative proportion in the study period.

Relative search volume for a given query includes all Google searches containing the specific queried words, with or without additional words. For example, searches for “how to kill myself” would be included in the search volume for our query “kill myself.”

Time period and location

Data was limited to Google searches conducted within the United States between March 3, 2019 and April 18, 2020. The first death attributed to COVID-19 within the United States occurred on February 29, 2020, and the following day (March 1, 2020) was selected as the beginning of the early pandemic period to reflect the shift in public perception that was associated with this event. One year of data (March 3, 2019 –February 29, 2020) was used to train the ARIMA models as longer time periods would include surges in suicide-related queries associated with unrelated events such as the deaths by suicide of Kate Spade and Anthony Bourdain. Given that Google Trends provides weekly data for time intervals of this length, March 3, 2019 represents the beginning of the week as close as possible to falling exactly one year prior to the beginning of the early pandemic period beginning March 1, 2020.

Analytic strategy

Autoregressive integrated moving average (ARIMA) modeling was used to obtain predicted weekly relative search volume for each query for the March 1, 2020 to April 18, 2020 early pandemic period (Supplement 1). The four aforementioned query categories were entirely conceptual and queries were not combined during the model-building stage. Queries were therefore analyzed separately, yielding 18 total models constructed using search volume relative to all Google searches within the United States within the study period. Weekly search data from the preceding year (March 3, 2019 through February 29, 2020) was used to train the ARIMA models. Percent difference was calculated between this predicted relative search proportion and observed relative search proportion for each query at each week of the study period. Mean percent difference for each query with bootstrap 95% confidence intervals were then calculated for the full study period. All analyses were conducted in R version 3.6.1.

Results

Average relative search volume associated with four of the six suicide-specific search queries was lower than predicted during the study period (Fig 1). These queries were “suicide method + suicide methods” (-32%; 95%CI, -40% to -24%), “suicide–squad” (-20%; 95%CI, -27% to -14%), “commit suicide” (-18%; 95%CI, -24% to -12%), and “kill yourself” (-10%; 95%CI, -16% to -5%). Confidence intervals for mean percent difference associated with the remaining 2 suicide-related queries (“fast suicide + easy suicide + quick suicide” + painless suicide and “kill myself”) contained zero.

Fig 1. Google searches between March 1 and April 18, 2020.

Fig 1

Open in a new tab

(A) Average percent difference between observed relative search proportion and relative search proportion predicted with pre-COVID-19 data. Percentage greater than expected shown for all search queries with 95% confidence intervals. (B) Data presented is the same as in panel A, but limited to search queries with percent greater than expected. <|100%|.

Among the queries indicative of general mental health, average relative search volume was elevated for “depression” (3%; 95%CI, 1%-6%) and “panic attack” (12%; 95%CI, 2%-21%). Relative search volume for “anxiety” did not differ from predicted volume.

Average relative search volume for “loneliness” was 24% greater than expected (95%CI, 12%-37%).

Average relative search volume was elevated for all queries indicative of help-seeking, including “national suicide prevention lifeline + national suicide hotline” (9%; 95%CI, 1%-18%), “crisis text line” (36%; 95%CI, 18%-54%), and “disaster distress helpline + disaster distress hotline” which experienced a large surge in search volume with average relative search volume reaching 3021% greater than expected (95%CI, 873%-5169%).

Average relative search volume was also sharply elevated for every query indicative of financial difficulty, including “loan” (37%; 95%CI, 17%-58%), “I lost my job” (226%; 95%CI, 120%-333%), “laid off” (1164%; 95%CI, 395%-1932%), “unemployment” (1238%; 95%CI, 560%-1915%), and “furlough” (5717%; 95%CI, 2769%-8665%).

Discussion

Google search behavior appears to have dramatically changed following the emergence of COVID-19 in the United States, particularly for queries related to financial difficulty (“I lost my job,” 226%; “laid off,” 1164%; “unemployment,” 1238%; “furlough,” 5717%) and the Disaster Distress Helpline (“disaster distress helpline + disaster distress hotline,” 3021%). Searches related to suicide are lower than expected, while searches related to general mental health and help-seeking are moderately elevated. Search volumes for suicide-specific terms such as “suicide,” “commit suicide,” and “kill yourself,” which have previously been shown to have a positive correlation with suicide rates [7, 9, 11] are all lower than predicted. However, searches for queries representative of financial difficulty have also been shown to have a positive association with suicide rates [8, 10, 11] and the scale at which search volume for these queries increased in the present study is remarkable. This marked increase in search volume is particularly concerning considering that out of eighty-nine search queries evaluated for correlation with monthly suicide rates in the United States between 2004 and 2017, the strongest positive correlation was found for the query “laid off” with a lag time of 2 months [8]. It seems as though individuals are grappling with the immediate stresses of job loss and isolation and are consequently reaching out for help from crisis services, but that the impact on suicidality has not yet occurred.

There is significant cause for concern regarding the long-term impact of COVID-19 on suicide in the United States. While it is common for suicide rates to decrease in the immediate aftermath of national crises such as the September 11 attacks [18], there is evidence of long-term increases in suicide rates due to the 1918 flu pandemic [19] and the 2003 SARS outbreak in Hong-Kong [20]. Additionally, COVID-19 and the national response to the virus affect many suicide risk factors in ways that have been described as a “perfect storm” for suicide [5]. Increased isolation, unhealthy home-life situations, financial difficulties, grief associated with the death of loved ones, and other exacerbated risk factors may lead to an increase in suicidality and suicide attempts in the future. Many of these suicide predictors have already seen meaningful changes reflected in Google Trends data, such as an apparent increase in searches potentially representative of unemployment, panic attacks, loneliness, and the need for crisis hotlines.

Although we cannot know for certain the motivation or context behind any particular Google search, alternative data sources corroborate the patterns shown in the search data. Unemployment claims in the United States were over 20 times higher during the week ending on April 18, 2020 than during the comparable week in 2019 (4,267,395 compared to 211,762) and over 26 million claims have been filed since the start of the pandemic [21]. Additionally, although weekly calls to the National Suicide Prevention Lifeline have remained stable, calls to the Disaster Distress Helpline during the early pandemic period increased 1073% relative to the equivalent weeks in 2019 (personal communication, Sean Murphy PhD., Vibrant Emotional Health, administrative agency for these crisis services). As such, the increase in Google searches for the Disaster Distress Helpline is correlated with an increase in actual call volume, a potential indicator of the success of the promotion efforts for this service during the COVID-19 pandemic.

Although we do not have data regarding absolute changes in demand for Crisis Text Line services since the start of the pandemic, data insights published on the organization’s website (www.crisistextline.org) are closely aligned with the trends observed in Google search behavior. The Crisis Text Line reports increases in conversations about grief, the coronavirus disease, loneliness, sexual assault and emotional abuse during quarantine, anxiety, depression, financial concerns, and stress associated with frontline work. Interestingly, reports of suicidal ideation have simultaneously fallen from 28% of chats to 22% of chats. Increases in Google searches for “crisis text line” indicate that more people may be reaching out to this service for support, and insights from these Crisis Text Line conversations support the hypothesis that Americans are seeking help for the serious emotional difficulties associated with COVID-19 while experiencing less suicidality.

The present study has several important limitations. Google search data is particularly prone to selection bias as access to and inclination to use search engines likely differ substantially between sub-populations. Additionally, lack of data on actual suicidal behavior prohibits firm conclusions from being drawn regarding the predictive capabilities of the search queries included in this study. Inclusion of mortality data would also allow for sub-group analyses, which are crucially important given the differing relationships between search behavior and suicidal behavior previously observed based upon age and gender [8, 9, 11, 13, 15]. Once 2020 mortality data becomes available, the present study can be built upon by assessing the relationship between the changes in search behavior observed in the present study and changes in suicide rates within the United States. Finally, we cannot be sure how the Google search results for any of the queries assessed in this study may mediate a relationship between search behavior and suicidality or suicide attempts. For example, the telephone number for the National Suicide Prevention Lifeline is the first result displayed for many queries associated with suicidal ideation, such as “kill myself,” and search results highlighting such sources of help may impact post-search behavior.

The results of the current study highlight the ongoing emotional and financial struggles of Americans in the face of the COVID-19 pandemic. Although relative search volume for explicitly suicide-related queries has dropped in these early stages of the pandemic’s progression, long-term suicidality is an important concern given the impact of COVID-19 on known risk factors for suicide.

Supporting information

S1 File. Weekly observed and predicted relative Google search proportion.

(DOCX)

Click here for additional data file. (7.2MB, docx)

Acknowledgments

We would like to acknowledge Dr. Hanga Galfalvy, Associate Professor of Biostatistics (in Psychiatry) at the Columbia University Medical Center, for her consultation on the analyses.

Data Availability

All relevant data are within the manuscript and its Supporting Information files.

Funding Statement

This project was supported by institutional funds provided by the Division of Child and Adolescent Psychiatry, New York State Psychiatric Institute. There was no additional external funding received for this study.

References

  • 1.Chang S, Stucklet D, Yip P, Gunnell D. Impact of 2008 global economic crisis on suicide: time trend study in 54 countries. BMJ. 2013. September 17;347:f5239 10.1136/bmj.f5239 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Gunnell D, Appleby L, Arensman E, Hawton K, John A, Kapur N, et al. Suicide risk and prevention during the COVID-19 pandemic. Lancet Psychiatry. Epub 2020 Apr 21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Stuckler D, Basu S, Suhrcke M, Coutts A, McKee M. The public health effect of economic crises and alternative policy responses in Europe: an empirical analysis. Lancet. 2009. July 25;374(9686):315–23. 10.1016/S0140-6736(09)61124-7 [DOI] [PubMed] [Google Scholar]
  • 4.Trout DL. The role of social isolation in suicide. Suicide Life Threat Behav. 1980. Spring;10(1):10–23. 10.1111/j.1943-278x.1980.tb00693.x [DOI] [PubMed] [Google Scholar]
  • 5.Reger MA, Stanley IH, Joiner TE. Suicide mortality and coronavirus disease 2019—A perfect storm? JAMA Psychiatry. Epub 2020 Apr 10. [DOI] [PubMed] [Google Scholar]
  • 6.Yao H, Chen HJ, Xu YF. Patients with mental health disorders in the COVID-19 epidemic. Lancet Psychiatry 2020. April 1;7(4):e21 10.1016/S2215-0366(20)30090-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Gunn JF, Lester D. Using Google searches on the internet to monitor suicidal behavior. J. Affect. Disord. 2013;148:411–2. 10.1016/j.jad.2012.11.004 [DOI] [PubMed] [Google Scholar]
  • 8.Lee JY. Search trends preceding increases in suicide: A cross-correlation study of monthly Google search volume and suicide rate using transfer function models. J. Affect. Disord. 2020;262:155–64. 10.1016/j.jad.2019.11.014 [DOI] [PubMed] [Google Scholar]
  • 9.Ma-Kellams C, Or F, Baek JH, Kawachi I. Rethinking suicide surveillance: Google search data and self-reported suicidality differentially estimate completed suicide risk. Clin. Psychol. Sci. 2016;4(3):480–4. [Google Scholar]
  • 10.Parker J, Cuthbertson C, Loveridge S, Skidmore M, Dyar W. Forecasting state-level premature deaths from alcohol, drugs, and suicides using Google Trends data. J. Affect. Disord. 2017;213:9–15. 10.1016/j.jad.2016.10.038 [DOI] [PubMed] [Google Scholar]
  • 11.Arora VS, Stuckler D, McKee M. Tracking search engine queries for suicide in the United Kingdom. Public Health. 2016;137:147–53. 10.1016/j.puhe.2015.10.015 [DOI] [PubMed] [Google Scholar]
  • 12.Yang AC, Tsai S, Huang NE, Peng C. Association of Internet search trends with suicide death in Taipei City, Taiwan, 2004–2009. J. Affect. Disord. 2011;132:179–84. 10.1016/j.jad.2011.01.019 [DOI] [PubMed] [Google Scholar]
  • 13.Barros JM, Melia R, Francis K, Bogue J, O’Sullivan M, Young K, et al. The validity of Google Trends search volumes for behavioral forecasting of national suicide rates in Ireland. Int. J. Environ. Res. Public Health. 2019. September;16(17):3201. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Kristoufek L, Moat HS, Preis T. Estimating suicide occurrence statistics using Google Trends. EPJ Data Sci. 2016;5(1):32 10.1140/epjds/s13688-016-0094-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Hagihara A, Miyazaki S. Internet suicide searches and the incidence of suicide in young people in Japan. Eur. Arch. Psychiatry Clin. Neurosci. 2012;262:39–46. 10.1007/s00406-011-0212-8 [DOI] [PubMed] [Google Scholar]
  • 16.Song TM, Song J, An JY, Hayman LL, Woo JM. Psychological and social factors affecting Internet searches on suicide in Korea: a big data analysis of google search trends. Yonsei Med. J. 2014;55(1):254–263. 10.3349/ymj.2014.55.1.254 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Ayers JW, Althouse BM, Leas EC, Dredze M, Allem J. Internet searches for suicide following the release of 13 Reasons Why. JAMA Intern. Med. 2017. October;177(10):1527–9. 10.1001/jamainternmed.2017.3333 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Claassen CA, Carmody T, Stewart SM, Bossarte RM, Larkin GL, Woodward WA, et al. Effect of 11 September 2001 terrorist attacks in the USA on suicide in areas surrounding the crash sites. Br J Psychiatry. 2010. May;196(5):359–64. 10.1192/bjp.bp.109.071928 [DOI] [PubMed] [Google Scholar]
  • 19.Wasserman IM. The impact of epidemic, war, prohibition and media on suicide: United States, 1910–1920. Suicide Life Threat Behav. 1992. Summer;22(2):240–54. [PubMed] [Google Scholar]
  • 20.Cheung YT, Chau PH, Yip PS. A revisit on older adults suicides and severe acute respiratory syndrome (SARS) epidemic in Hong Kong. Int J Geriatr Psychiatry. 2008. December;23:1231–8. 10.1002/gps.2056 [DOI] [PubMed] [Google Scholar]
  • 21.U.S. Department of Labor. Unemployment weekly insurance claims. 2020, April 18. [Press Release]. Retrieved from: https://www.dol.gov/ui/data.pdf, April 18, 2020.

Decision Letter 0

Neal Doran

23 Jun 2020

PONE-D-20-12386

Google searches for suicide and suicide risk factors in the early stages of the COVID-19 pandemic

PLOS ONE

Dear Dr. Gould,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

As the reviewers note below, some elements of the methodology are unclear or require revision, and in some cases the reviewers have suggested clear ways to address the concerns. Additionally, the paper would benefit from more careful interpretation, in particular regarding the extent to which internet searches are indicative of offline behavior.

Please submit your revised manuscript by Aug 07 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Neal Doran

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

3. Thank you for stating in your Funding Statement:

"This project was partially supported from institutional funds provided by the Division of Child and Adolescent Psychiatry, New York State Psychiatric Institute"

Please provide an amended statement that declares *all* the funding or sources of support (whether external or internal to your organization) received during this study, as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now.  Please also include the statement “There was no additional external funding received for this study.” in your updated Funding Statement.

Please include your amended Funding Statement within your cover letter. We will change the online submission form on your behalf.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In the introduction the authors could also refer to previous work that has shown that Google search behavior is in fact related to future suicidality (e.g., Arendt & Scherr, 2017a, 2017b).

Search terms such as “depression” can be regarded ambiguous given the economic impact of COVID-19 and discussions about whether the pandemic drove into an economic depression. It would be advisable to rule out such impact by also including (i.e., controlling for) economy related search terms at the same time in models in order to rule out the ambiguity of certain search terms. Interestingly, this has been done with the term “suicide -squad” that I assume in many cases refers to the movie and has therefore been explicitly excluded in order to not make the other query shares “disappear” relative to it. This would also be a strategy to handle other ambiguity around depression related search terms.

What was the rationale for looking at the specific time interval from March 3 to April 18? Please clarify.

In the results section, for some of the search terms the percentage sign is missing (p. 6, l. 146-147). Are the Google trends numbers calculated within each group of queries or have they all been lumped together and are presented relative to all? This information needs to be updated and included in the methods section.

The discussion section should refrain from giving the impression that the Google searches can automatically be translated into actual behaviors. More nuance would be good. Moreover, for certain search terms, certain languages, and only in certain countries Google suggests suicide prevention resources to some users in form of a Suicide Prevention Result (SPR) (see Haim et al., 2019; Scherr et al., 2019; Arendt et al., 2020), which is likely to yield differential (protective) effects on post-search behaviors (i.e., not the same effects for all across the board). This could be discussed in order to bring more nuance to the findings, or to explain counterintuitive findings. It might actually be that google increased the display frequency in some countries as a reaction to COVID-19.

Although Google trends data can be a good indicator for actual behavior especially in the suicide domain (Arendt & Scherr, 2017a, 2017b), conclusions cannot be drawn between the search queries given that we don't even know in how far there is an overlap of people who googled for these different groups of search queries (i.e., problem of an ecological fallacy). Above and beyond that, it is also worthwhile to discuss the opposite effect direction in that people with a higher vulnerability are stressed more (see Scherr, Toma, & Schuster, 2019) through the corona crisis, and therefore Google for mental health resources or suicide-related terms in order to find support (see e.g., Scherr & Reinemann, 2016) not the other way round.

Reviewer #2: Thank you for the opportunity to review this paper. It is a clear, well-written and interesting piece exploring trends in Google searches for suicide-related terms during the COVID-19 pandemic, and the results suggest some interesting trends in behaviour during this time. While overall it appears to be a sound piece of work, I have several suggestions that may improve its clarity.

1) Firstly, it would be useful to explain in the introduction why you chose to look at Google searches as your indicator of suicidality. Why was this chosen above other indicators such as, for example, the number of calls to mental health helplines, number of admissions to emergency departments for suicide attempts, etc.? How reliable an indicator is the number of Google searches e.g. what is the evidence that an increase in Google searches corresponds to an increase in behaviour? Providing this context in the introduction will strengthen your paper and the implications of your findings.

2) How did you decide on your search terms? A short description of this process would be useful to justify your chosen terms. It is likely that people experiencing suicidality may not include the word “suicide” in their Google search, and may instead use more general terms (e.g. “painless death” rather than “painless suicide”).

3) In the results section, it would be useful to discuss the magnitude of your effects (some of which are considerable). At present, this section reports the search terms that had an increase during the period of the study, but not the size of this increase. The variation here is quite large and worthy of comment – it ranges from a 3% increase for “depression” through to over 5700% for “furlough,” yet both of these terms are discussed as if the findings were relatively equal. I would suggest that the magnitude of the effect for some of these terms is a bigger finding than merely the fact that there was an increase. Similarly, the increase in searches for “Disaster Distress Helpline” and “Crisis Text Line” are mentioned in the discussion section as though they are relatively equal, although one saw an increase of 36% and the other and increase of over 3000%.

4) It would be useful to explain some of your terms for an international audience – although I understand that your study was focused on America, I was not familiar with some of the terms and services described. In particular, the Disaster Distress Helpline and Crisis Text Line receive some attention in the discussion section, but I was not sure what these services are or how they differ from each other. I also believe that “furlough” may be an American term, and considering its effect size it would be useful to make sure that international readers understand what it means.

5) Similarly to my first comment, your discussion section would be greatly strengthened if a little more evidence for your methods was provided. For example, is there evidence to support your claim that a drop in suicide-related terms equates to a drop in suicidality during the study period? This is implied, but if it could be supported by evidence it would make your claims stronger. If not, adding this as a limitation would be worthwhile.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Jul 24;15(7):e0236777. doi: 10.1371/journal.pone.0236777.r002

Author response to Decision Letter 0

1 Jul 2020

Dear Dr. Doran:

We greatly appreciate the thorough review of our manuscript. We have revised the manuscript

to address the journal requirements and each concern raised by the two reviewers. Our

responses to each of the suggestions are described below. For your convenience, each of the

reviewers’ suggestions precedes our response, and we have provided page and line numbers to

facilitate your finding the revisions in the manuscript. These page and line numbers correspond

to those in the version of the manuscript with changes highlighted using the track changes

mode in MS Word.

Journal Requirements:

1. Please ensure that your manuscript meets PLOS ONE’s style requirements, including those for file naming.

Files have been renamed to meet PLOS ONE’s style requirements (see Fig1.tiff and S1_File.pdf).

2. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly.

Captions have been added for the Supporting Information file at the end of the manuscript (see p. 15).

3. Please provide an amended statement that declares *all* the funding or sources of support (whether external or internal to your organization) received during this study, as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now. Please also include the statement “There was no additional external funding received for this study.” in your updated Funding Statement.

Please include your amended Funding Statement within your cover letter. We will change the online submission form on your behalf.

Our amended funding statement is as follows: This project was supported by institutional funds provided by the Division of Child and Adolescent Psychiatry, New York State Psychiatric Institute. There was no additional external funding received for this study.

Concerns of Reviewer 1:

1. In the introduction the authors could also refer to previous work that has shown that Google search behavior is in fact related to future suicidality.

Reviewer 2 has expressed a similar concern. We have added references to previous work evaluating the relationship between Google search behavior and observed suicide rates to the Introduction and Discussion sections.

Specifically, we have added the following passage to the Introduction (p. 3, l. 87 – p. 4, l. 99):

“Google Trends data was chosen for this study because it is far more rapidly available than alternative data sources such as mortality data, and because significant evidence exists that establishes an association between Google search behavior and actual suicidal behavior both within the United States (Gunn & Lester 2013; Lee 2020; Ma-Kellams et al. 2016; Parker et al. 2017) and internationally (Arora et al. 2016; Yang et al. 2011; Barros et al. 2019; Kristoufek et al. 2016; Hagihara & Miyazaki 2012; Song et al. 2014). In fact, Ma-Kellams et al. (2016) found that Google searches for “how to suicide,” “how to kill yourself,” and “painless suicide” were better predictors of completed suicides in the United States than traditional self-reported measures of suicide risk. Associations have also been found between searches for mental health diagnoses and other known risk factors for suicide and actual suicide rates (Arora et al. 2016; Yang et al. 2011; Barros et al. 2019; Lee et al. 2020). Searches for queries representing unemployment are frequently associated with suicide (Arora et al. 2016; Lee 2020; Parker et al. 2017)…..“

We have added the following passage to the Discussion (p. 10, l. 250 – p. 11, l. 260):

“Search volumes for suicide-specific terms such as “suicide,” “commit suicide,” and “kill yourself,” which have previously been shown to have a positive correlation with suicide rates (Arora et al. 2016; Gunn & Lester 2013; Ma-Kellams et al. 2016) are all lower than predicted. However, searches for queries representative of financial difficulty have also been shown to have a positive association with suicide rates (Arora et al. 2016; Parker et al. 2017; Lee 2020) and the scale at which search volume for these queries increased in the present study is remarkable. This marked increase in search volume is particularly concerning considering that out of eighty-nine search queries evaluated for correlation with monthly suicide rates in the United States between 2004 and 2017, the strongest positive correlation was found for the query “laid off” with a lag time of 2 months (Lee 2020).”

2. Search terms such as “depression” can be regarded ambiguous given the economic impact of COVID-19 and discussions about whether the pandemic drove into an economic depression. It would be advisable to rule out such impact by also including (i.e., controlling for) economy related search terms at the same time in models in order to rule out the ambiguity of certain search terms. Interestingly, this has been done with the term “suicide –squad” that I assume in many cases refers to the movie and has therefore been explicitly excluded in order to not make the other query shares “disappear” relative to it. This would also be a strategy to handle other ambiguity around depression related search terms.

In light of this question, we conducted three additional analyses subtracting out the words “economy,” “economic,” and “are we going into a” from the “depression” query. Overall search volume and the patterns in search volume during the study period both remained almost identical when removing any of these three phrases from the “depression” query, indicating that very few searches which clearly indicated an interest in an economic depression were contributing to our results. Additionally, the change in search volume for the “depression” query was quite small in our study, and if it were truly confounded with economic depression we would likely see a much greater percentage change given the enormous changes observed for the other economic terms such as “unemployment.” We therefore decided that it was justifiable to leave the “depression” query as is, and to not specifically select out these economic terms. This information has been added to the Methods section (see p. 6, l. 140 – l. 145), in a new section (Query selection) that more thoroughly describes our query selection process. In regards to the depression query, we specified:

“We considered using this feature to filter out searches for “depression” that included “economy,” “economic,” and “are we going into a” to help ensure that we captured searches related to emotional depression rather than an economic depression. However, doing so made no meaningful change in observed search volume so we decided to analyze the simple “depression” query used by Arora et al. (2016), Barros et al. (2019), and Kristoufek et al. (2016).”

3. What was the rationale for looking at the specific time interval from March 3 to April 18? Please clarify.

We first want to clarify that this study was limited to searches within the United States. The first death attributed to COVID-19 in the United States occurred on February 29, 2020, and we selected the following day (March 1, 2020) as the beginning of our analysis period given the shift in public perception of the virus that we witnessed occurring around that time. April 18 was chosen as the end of the study period because search data for later dates was not available to us when we conducted our analyses in late April. We decided to use one year of training data to train the ARIMA models as two years of data would include significant peaks in suicide-related queries associated with the deaths by suicide of Kate Spade and Anthony Bourdain. Given the weekly format of Google Trends data for periods longer than 90 days, March 3 represents the beginning of the week that was as close as possible to being exactly one year prior to March 1st.

The methods section has been clarified and an additional “Time period and location” section (see p. 8, l. 189 – l. 199) has been added to elaborate on the selection of these dates.

4. In the results section, for some of the search terms the percentage sign is missing (p. 6, l. 146-147).

Percentage signs have been added where appropriate on these lines in the Results section (now p. 10, l. 241) in the revised manuscript).

5. Are the Google trends numbers calculated within each group of queries or have they all been lumped together and are presented relative to all? This information needs to be updated and included in the methods section.

Each query was analyzed separately, so all are presented relative to all Google searches made in the United States during our study period. This has been clarified in the Methods section (see p. 8, l. 204 – p. 9, l. 207) with the following statement:

“The four aforementioned query categories were entirely conceptual and queries were not combined during the model-building stage. Queries were therefore analyzed separately, yielding 18 total models constructed using search volume relative to all Google searches within the United States within the study period. ”

6. The discussion section should refrain from giving the impression that the Google searches can automatically be translated into actual behaviors. More nuance would be good. Moreover, for certain search terms, certain languages, and only in certain countries Google suggests suicide prevention resources to some users in form of a Suicide Prevention Result (SPR) (see Haim et al., 2019; Scherr et al., 2019; Arendt et al., 2020), which is likely to yield differential (protective) effects on post-search behaviors (i.e., not the same effects for all across the board). This could be discussed in order to bring more nuance to the findings, or to explain counterintuitive findings. It might actually be that google increased the display frequency in some countries as a reaction to COVID-19. Although Google trends data can be a good indicator for actual behavior especially in the suicide domain (Arendt & Scherr, 2017a, 2017b), conclusions cannot be drawn between the search queries given that we don't even know in how far there is an overlap of people who googled for these different groups of search queries (i.e., problem of an ecological fallacy). Above and beyond that, it is also worthwhile to discuss the opposite effect direction in that people with a higher vulnerability are stressed more (see Scherr, Toma, & Schuster, 2019) through the corona crisis, and therefore Google for mental health resources or suicide-related terms in order to find support (see e.g., Scherr & Reinemann, 2016) not the other way round.

We have added depth and nuance to the discussion section. The existing evidence supporting a relationship between Google search behavior and suicide deaths has been readdressed in the discussion, and we have clarified that ambiguity remains regarding the true association between Google search behavior and suicidality during the COVID-19 pandemic.

We have added a limitations section to the discussion (p. 13, l. 303 – l. 318) which describes our inability to draw conclusions from Google Trends data, as well as the possible mediating effect of Google’s Suicide Prevention Result on the relationship between searches and suicidal behavior. See below:

“The present study has several important limitations. Google search data is particularly prone to selection bias as access to and inclination to use search engines likely differ substantially between sub-populations. Additionally, lack of data on actual suicidal behavior prohibits firm conclusions from being drawn regarding the predictive capabilities of the search queries included in this study. Inclusion of mortality data would also allow for sub-group analyses, which are crucially important given the differing relationships between search behavior and suicidal behavior previously observed based upon age and gender (Arora et al. 2016; Barros et al. 2019; Lee 2020; Ma-Kellams et al. 2016; Hagihara & Miyazaki 2012). Once 2020 mortality data becomes available, the present study can be built upon by assessing the relationship between the changes in search behavior observed in the present study and changes in suicide rates within the United States. Finally, we cannot be sure how the Google search results for any of the queries assessed in this study may mediate a relationship between search behavior and suicidality or suicide attempts. For example, the telephone number for the National Suicide Prevention Lifeline is the first result displayed for many queries associated with suicidal ideation, such as “kill myself,” and search results highlighting such sources of help may impact post-search behavior.”

Concerns of Reviewer 2:

1. Firstly, it would be useful to explain in the introduction why you chose to look at Google searches as your indicator of suicidality. Why was this chosen above other indicators such as, for example, the number of calls to mental health helplines, number of admissions to emergency departments for suicide attempts, etc.? How reliable an indicator is the number of Google searches e.g. what is the evidence that an increase in Google searches corresponds to an increase in behaviour? Providing this context in the introduction will strengthen your paper and the implications of your findings.

Google search data was chosen as an indicator of suicidality because it is far more rapidly available than most other data sources. Additionally, many emergency departments were so overloaded with COVID-19 patients that non-COVID-19 patients, including suicidal individuals, had difficulty accessing care or postponed treatment in expectation of issues accessing care. It would therefore have been difficult to accurately assess suicidality using that measure, as suicidality is probably very underrepresented by ER data right now. While calls to helplines certainly represent an important data source (and changes in call volume to two such helplines are referenced in the Discussion section), they are primarily an indicator of help-seeking behavior and we were also interested in suicide risk and economic indicators in particular.

As noted in our reply to the first concern raised by Reviewer 1, significant evidence exists that establishes a relationship between Google search behavior and national suicide rates in several countries. This evidence, as well as our general rationale for using Google searches, has been expanded upon in our Introduction section (see p. 3, l. 87 – p. 4, l. 99) and is also provided below for your convenience:

“Google Trends data was chosen for this study because it is far more rapidly available than alternative data sources such as mortality data, and because significant evidence exists that establishes an association between Google search behavior and actual suicidal behavior both within the United States (Gunn & Lester 2013; Lee 2020; Ma-Kellams et al. 2016; Parker et al. 2017) and internationally (Arora et al. 2016; Yang et al. 2011; Barros et al. 2019; Kristoufek et al. 2016; Hagihara & Miyazaki 2012; Song et al. 2014). In fact, Ma-Kellams et al. (2016) found that Google searches for “how to suicide,” “how to kill yourself,” and “painless suicide” were better predictors of completed suicides in the United States than traditional self-reported measures of suicide risk. Associations have also been found between searches for mental health diagnoses and other known risk factors for suicide and actual suicide rates (Arora et al. 2016; Yang et al. 2011; Barros et al. 2019; Lee et al. 2020). Searches for queries representing unemployment are frequently associated with suicide (Arora et al. 2016; Lee 2020; Parker et al. 2017), and are of particular relevance to the present study given the far-reaching economic damage caused by the COVID-19 pandemic. Therefore, we assessed Google Trends data representing searches for terms related to suicide and known suicide risk factors following the emergence of COVID-19 in the United States with the aim of providing insight into how the early stages of this pandemic may have impacted suicidality and suicide rates.”

We also expanded our Discussion to highlight the association between Google searches and suicidal behavior. Please see our response to your fifth concern for details.

2. How did you decide on your search terms? A short description of this process would be useful to justify your chosen terms. It is likely that people experiencing suicidality may not include the word “suicide” in their Google search, and may instead use more general terms (e.g. “painless death” rather than “painless suicide”).

The categories of explicitly suicide-related queries, help-seeking queries, queries related to financial difficulty, and general mental health problems were selected before the individual queries themselves were decided upon. These categories were chosen due to their known association with suicide risk (Reger et al. 2020).

Queries for the explicitly suicide-related, financial difficulty, and general mental health problems were selected through a literature review and use of the “Related queries” tool on the Google trends website. Queries for the help-seeking category were designed to represent the National Suicide Prevention Lifeline, the Crisis Text Line, and the Disaster Distress Helpline.

A separate section (“Query selection,” see p. 4, l. 109 – p. 6, l. 150) has been added to the Methods section to elaborate on this process. For your convenience, this section has been provided here as well:

“There were four primary domains which were examined due to their known associations with suicide risk (Reger et al. 2020): suicide-specific queries, help-seeking queries, general mental health queries, and queries related to financial difficulty. We did not try to be exhaustive by including every possible search term that a user may have utilized within each category, but rather aimed to have 3 to 6 queries representative of each domain.

Queries were selected for the suicide-specific, general mental health, and financial difficulty categories based upon a review of the literature and use of the “Related Queries” tool available on the Google Trends interface. These finalized categories are as follows: suicide-specific queries (“suicide –squad” [Arora et al. 2016; Barros et al. 2019; Kristoufek et al. 2016; Song et al. 2014], “kill myself,” “kill yourself,” “suicide methods + suicide method,” “commit suicide” [Gunn & Lester 2013], and “fast suicide + easy suicide + quick suicide + painless suicide” [Ma-Kellams et al. 2016]), general mental health queries (“depression” [Arora et al. 2016], “panic attack,” and “anxiety” [Yang et al. 2011; Barros et al. 2019; Arora et al. 2016; Lee et al. 2020]), queries related to financial difficulty (“unemployment” [Arora et al. 2016; Parker et al. 2017], “I lost my job,” “laid off” [Lee 2020], “furlough” (a mandatory temporary leave for employees), and “loan”). The uncategorized query “loneliness” was also included to assess possible impacts of the social distancing interventions implemented to slow the spread of COVID-19.

Queries for the help-seeking category were intended to represent the two most prominent and widely-used crisis services in the United States (National Suicide Prevention Lifeline, the national network of local crisis centers, https://suicidepreventionlifeline.org/; Crisis Text Line, the global organization providing crisis intervention over text message, https://www.crisistextline.org/), as well as the Disaster Distress Helpline, a national hotline providing crisis counseling for those impacted by any natural or human-caused disaster that has been promoted during the COVID-19 pandemic (https://www.samhsa.gov/find-help/disaster-distress-helpline). This finalized category contains the following queries: “crisis text line,” “national suicide prevention lifeline + national suicide hotline,” and “disaster distress helpline + disaster distress hotline.”

Google Trends (https://trends.google.com/trends/?geo=US) allows queries containing certain words to be excluded from the relative search proportion using the “-“ sign, and searches for the popular movie Suicide Squad were thus excluded from our “suicide” search query by requesting data for “suicide –squad” (Ayers et al. 2017). We considered using this feature to filter out searches for “depression” that included “economy,” “economic,” and “are we going into a” to help ensure that we captured searches related to emotional depression rather than an economic depression. However, doing so made no meaningful change in observed search volume so we decided to analyze the simple “depression” query used by Arora et al. (2016), Barros et al. (2019), and Kristoufek et al. (2016). Google trends also allows for the relative search volume associated with several search queries to be combined using the “+” sign. This tool was used to yield a combined relative search proportion for common variations on terms or conceptually linked terms, resulting in the following queries: “national suicide prevention lifeline + national suicide hotline,” “disaster distress helpline + disaster distress hotline,” “suicide methods + suicide method,” and “fast suicide + easy suicide + quick suicide + painless suicide.” “

3. In the results section, it would be useful to discuss the magnitude of your effects (some of which are considerable). At present, this section reports the search terms that had an increase during the period of the study, but not the size of this increase. The variation here is quite large and worthy of comment – it ranges from a 3% increase for “depression” through to over 5700% for “furlough,” yet both of these terms are discussed as if the findings were relatively equal. I would suggest that the magnitude of the effect for some of these terms is a bigger finding than merely the fact that there was an increase. Similarly, the increase in searches for “Disaster Distress Helpline” and “Crisis Text Line” are mentioned in the discussion section as though they are relatively equal, although one saw an increase of 36% and the other and increase of over 3000%.

We present the percentage change in the results. The discussion section has been revised to address these percentage changes in a more nuanced manner and to highlight findings of greater magnitude (see p. 10, l. 245 – p. 11, 256). Specifically, we write:

“Google search behavior appears to have dramatically changed following the emergence of COVID-19 in the United States, particularly for queries related to financial difficulty (“I lost my job,” 226%; “laid off,” 1164%; “unemployment,” 1238%; “furlough,” 5717%) and the Disaster Distress Helpline (“disaster distress helpline + disaster distress hotline,” 3021%). Searches related to suicide are lower than expected, while searches related to general mental health and help-seeking are moderately elevated. Search volumes for suicide-specific terms such as “suicide,” “commit suicide,” and “kill yourself,” which have previously been shown to have a positive correlation with suicide rates (Arora et al. 2016; Gunn & Lester 2013; Ma-Kellams et al. 2016) are all lower than predicted. However, searches for queries representative of financial difficulty have also been shown to have a positive association with suicide rates (Arora et al. 2016; Parker et al. 2017; Lee 2020) and the scale at which search volume for these queries increased in the present study is remarkable.”

The abstract has been revised to reflect these changes as well.

4. It would be useful to explain some of your terms for an international audience – although I understand that your study was focused on America, I was not familiar with some of the terms and services described. In particular, the Disaster Distress Helpline and Crisis Text Line receive some attention in the discussion section, but I was not sure what these services are or how they differ from each other. I also believe that “furlough” may be an American term, and considering its effect size it would be useful to make sure that international readers understand what it means.

We now realize that we have to be more cognizant of the international readership. Furlough has been briefly defined as “a mandatory temporary leave for employees ” (see p. 5, l. 124), and descriptions of the services provided by the Crisis Text Line and the Disaster Distress Helpline have been added to the “Query selection” section of the Methods (see p. 5, l. 127 – l. 134). For your convenience, that description is provided here as well: “Queries for the help-seeking category were intended to represent the two most prominent and widely-used crisis services in the United States (National Suicide Prevention Lifeline, the national network of local crisis centers, https://suicidepreventionlifeline.org/; Crisis Text Line, the global organization providing crisis intervention over text message, https://www.crisistextline.org/), as well as the Disaster Distress Helpline, a national hotline providing crisis counseling for those impacted by any natural or human-caused disaster has been promoted during the COVID-19 pandemic (https://www.samhsa.gov/find-help/disaster-distress-helpline). ”

5. Similarly to my first comment, your discussion section would be greatly strengthened if a little more evidence for your methods was provided. For example, is there evidence to support your claim that a drop in suicide-related terms equates to a drop in suicidality during the study period? This is implied, but if it could be supported by evidence it would make your claims stronger. If not, adding this as a limitation would be worthwhile.

As mentioned in our response to your first comment, several references have been made in the Introduction (see p. 3, l. 87 – p. 4, l. 99) to research which has established associations between Google search behavior and actual suicidal behavior. See our response to your first comment for the summary of the available research included in our Introduction section.

We have also added the following passage to the Discussion section (p. 10, l. 250 – p. 11, l. 260) to emphasize the importance of the changes we observed in queries related to financial difficulty: “Search volumes for suicide-specific terms such as “suicide,” “commit suicide,” and “kill yourself,” which have previously been shown to have a positive correlation with suicide rates (Arora et al. 2016; Gunn & Lester 2013; Ma-Kellams et al. 2016) are all lower than predicted. However, searches for queries representative of financial difficulty have also been shown to have a positive association with suicide rates (Arora et al. 2016; Parker et al. 2017; Lee 2020) and the scale at which search volume for these queries increased in the present study is remarkable. This marked increase in search volume is particularly concerning considering that out of eighty-nine search queries evaluated for correlation with monthly suicide rates in the United States between 2004 and 2017, the strongest positive correlation was found for the query “laid off” with a lag time of 2 months (Lee 2020).”

Again, we appreciate the thorough review given to our manuscript, and consider the manuscript to be improved because of the reviewers' thoughtful comments.

Thank you for reconsidering the manuscript.

Sincerely,

Madelyn S. Gould, Ph.D., M.P.H.

Irving Philips Professor of Epidemiology in Psychiatry

Columbia University Irving Medical Center

1051 Riverside Drive, Mailbox 72

New York, NY 10032

Tel: (646)774-5763

Fax: (212)543-5281

gouldm@nyspi.columbia.edu

Citations:

Arora VS, Stuckler D, McKee M. Tracking search engine queries for suicide in the United

Kingdom. Public Health. 2016;137:147-53.

Barros JM, Melia R, Francis K, Bogue J, O’Sullivan M, Young K, et al. The validity of Google

Trends search volumes for behavioral forecasting of national suicide rates in Ireland. Int. J. Environ. Res. Public Health. 2019 Sept;16(17):3201.

Gunn JF, Lester D. Using Google searches on the internet to monitor suicidal behavior. J. Affect.

Disord. 2013;148:411-2.

Hagihara A, Miyazaki S. Internet suicide searches and the incidence of suicide in young people

in Japan. Eur. Arch. Psychiatry Clin. Neurosci. 2012;262:39-46.

Kristoufek L, Moat HS, Preis T. Estimating suicide occurrence statistics using Google Trends. EPJ

Data Sci. 2016;5(1):32.

Lee JY. Search trends preceding increases in suicide: A cross-correlation study of monthly

Google search volume and suicide rate using transfer function models. J. Affect. Disord.

2020;262:155-64.

Ma-Kellams C, Or F, Baek JH, Kawachi I. Rethinking suicide surveillance: Google search data and

self-reported suicidality differentially estimate completed suicide risk. Clin. Psychol. Sci.

2016;4(3):480-4.

Parker J, Cuthbertson C, Loveridge S, Skidmore M, Dyar W. Forecasting state-level premature

deaths from alcohol, drugs, and suicides using Google Trends data. J. Affect. Disord. 2017;213:9-15.

Song TM, Song J, An JY, Hayman LL, Woo JM. Psychological and social factors affecting Internet

searches on suicide in Korea: a big data analysis of google search trends. Yonsei Med. J. 2014;55(1):254-263.

Yang AC, Tsai S, Huang NE, Peng C. Association of Internet search trends with suicide death in

Taipei City, Taiwan, 2004-2009. J. Affect. Disord. 2011;132:179-84.

Decision Letter 1

Neal Doran

15 Jul 2020

Google searches for suicide and suicide risk factors in the early stages of the COVID-19 pandemic

PONE-D-20-12386R1

Dear Dr. Gould,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Neal Doran

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Acceptance letter

Neal Doran

17 Jul 2020

PONE-D-20-12386R1

Google searches for suicide and suicide risk factors in the early stages of the COVID-19 pandemic

Dear Dr. Gould:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Neal Doran

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 File. Weekly observed and predicted relative Google search proportion.

    (DOCX)

    Click here for additional data file. (7.2MB, docx)

    Data Availability Statement

    All relevant data are within the manuscript and its Supporting Information files.

    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES