Skip to main content
PMC home page
BMJ - PMC COVID-19 Collection logoLink to BMJ - PMC COVID-19 Collection
. 2023 Mar 8;380:e074224. doi: 10.1136/bmj-2022-074224

Comparison of mental health symptoms before and during the covid-19 pandemic: evidence from a systematic review and meta-analysis of 134 cohorts

Ying Sun 1, Yin Wu 1,2, Suiqiong Fan 1, Tiffany Dal Santo 1,2, Letong Li 1, Xiaowen Jiang 1, Kexin Li 1, Yutong Wang 1, Amina Tasleem 1, Ankur Krishnan 1, Chen He 1, Olivia Bonardi 1, Jill T Boruff 3, Danielle B Rice 4,5, Sarah Markham 6, Brooke Levis 1, Marleine Azar 1, Ian Thombs-Vite 1, Dipika Neupane 1, Branka Agic 7,8, Christine Fahim 9, Michael S Martin 10,11, Sanjeev Sockalingam 7,12, Gustavo Turecki 2,13, Andrea Benedetti 14,15,16, Brett D Thombs 1,2,14,15,17,18,✉
PMCID: PMC9992728  PMID: 36889797

Abstract

Objective

To synthesise results of mental health outcomes in cohorts before and during the covid-19 pandemic.

Design

Systematic review.

Data sources

Medline, PsycINFO, CINAHL, Embase, Web of Science, China National Knowledge Infrastructure, Wanfang, medRxiv, and Open Science Framework Preprints.

Eligibility criteria for selecting studies

Studies comparing general mental health, anxiety symptoms, or depression symptoms assessed from 1 January 2020 or later with outcomes collected from 1 January 2018 to 31 December 2019 in any population, and comprising ≥90% of the same participants before and during the covid-19 pandemic or using statistical methods to account for missing data. Restricted maximum likelihood random effects meta-analyses (worse covid-19 outcomes representing positive change) were performed. Risk of bias was assessed using an adapted Joanna Briggs Institute Checklist for Prevalence Studies.

Results

As of 11 April 2022, 94 411 unique titles and abstracts including 137 unique studies from 134 cohorts were reviewed. Most of the studies were from high income (n=105, 77%) or upper middle income (n=28, 20%) countries. Among general population studies, no changes were found for general mental health (standardised mean difference (SMD)change 0.11, 95% confidence interval −0.00 to 0.22) or anxiety symptoms (0.05, −0.04 to 0.13), but depression symptoms worsened minimally (0.12, 0.01 to 0.24). Among women or female participants, general mental health (0.22, 0.08 to 0.35), anxiety symptoms (0.20, 0.12 to 0.29), and depression symptoms (0.22, 0.05 to 0.40) worsened by minimal to small amounts. In 27 other analyses across outcome domains among subgroups other than women or female participants, five analyses suggested that symptoms worsened by minimal or small amounts, and two suggested minimal or small improvements. No other subgroup experienced changes across all outcome domains. In three studies with data from March to April 2020 and late 2020, symptoms were unchanged from pre-covid-19 levels at both assessments or increased initially then returned to pre-covid-19 levels. Substantial heterogeneity and risk of bias were present across analyses.

Conclusions

High risk of bias in many studies and substantial heterogeneity suggest caution in interpreting results. Nonetheless, most symptom change estimates for general mental health, anxiety symptoms, and depression symptoms were close to zero and not statistically significant, and significant changes were of minimal to small magnitudes. Small negative changes occurred for women or female participants in all domains. The authors will update the results of this systematic review as more evidence accrues, with study results posted online (https://www.depressd.ca/covid-19-mental-health).

Review registration

PROSPERO CRD42020179703.

Introduction

Concerns about covid-19 related mental health are substantial,1 2 3 but the sheer volume of low quality evidence has posed a barrier to evidence synthesis and decision making.4 5 6 Vast numbers of cross sectional studies have reported proportions of participants with scores above thresholds on easy-to-administer mental health scales as representing the “prevalence” of mental health problems, without comparisons with scores before the covid-19 pandemic.5 These scales are not, however, intended to estimate prevalence—thresholds are typically set for screening and to identify far more people than those who have a mental disorder; thus, proportions above thresholds substantially overestimate prevalence.7 8 9 10 11 Nonetheless, many study authors and media stories have concluded that the world’s population is experiencing a covid-19 mental health “pandemic” or “tsunami.”12

Many systematic reviews on covid-19 related mental health have synthesised results from cross sectional studies. Two previous systematic reviews compared pre-covid-19 findings with those during the pandemic.13 14 One reviewed 65 studies published up to January 2021 and found a small increase in mental health symptoms in early 2020 (standardised mean difference (SMD) 0.11, 95% confidence interval 0.04 to 0.17).13 The other searched for studies up to March 2021, included 43 studies, and reported that combined depression and anxiety symptoms worsened early in the pandemic (SMD 0.39, 95% credible interval 0.03 to 0.76).14 Both reviews, however, searched a limited number of English language databases, and many relevant studies have been published since they were conducted.

We are conducting an ongoing series of living systematic reviews15 on covid-19 related mental health, including a review of studies that compared mental health during covid-19 with pre-pandemic levels in the same cohort.4 5 In the present study, we compared general mental health, anxiety symptoms, and depression symptoms in the general population and other groups during covid-19 with outcomes from the same cohorts before covid-19.

Methods

We registered our series of systematic reviews in PROSPERO, and our protocol is available online (https://osf.io/96csg/).16 Results are reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.17 Supplementary material 1 describes minor amendments to the protocol.

Eligible studies

We included studies on any population that compared eligible outcomes assessed from 1 January 2018 to 31 December 2019 when China first reported covid-19 to the World Health Organization,18 with the same outcomes collected from 1 January 2020 or later. Studies had to report data from cohorts comprising at least 90% of the same participants between pre-covid-19 and pandemic periods or to use statistical methods to account for missing data. We did not include repeated cross sectional surveys or studies with fewer than 100 participants.

Eligible outcomes included continuous scores on validated mental health symptom questionnaires, proportion of participants above a threshold on a validated symptom questionnaire, or proportion of participants meeting criteria for a mental disorder using a validated diagnostic interview. In our ongoing living systematic review, we defined outcomes broadly to include, for example, anxiety symptoms, depression symptoms, general mental health, stress, loneliness, anger, grief, and burnout. In the present report, we included only general mental health (eg, general symptoms, mental health related quality of life), anxiety symptoms, and depression symptoms because few studies reported other outcomes. Results for other outcome domains are available online (https://www.depressd.ca/covid-19-mental-health).

Identification and selection of eligible studies

Using a strategy designed by an experienced health sciences librarian, we searched Medline (Ovid), PsycINFO (Ovid), CINAHL (EBSCO), Embase (Ovid), Web of Science Core Collection: Citation Indexes, China National Knowledge Infrastructure, Wanfang, medRxiv, and Open Science Framework Preprints. Because of the need for rapid evidence early in the pandemic, we did not formally peer review the search strategy; however, covid-19 terms were developed in collaboration with other librarians working on the topic and updated as covid-19 specific subject headings became available (see supplementary material 2). We initially searched from 31 December 2019 to 13 April 2020, then daily until 28 December 2020, and thereafter weekly.

Search results were uploaded into DistillerSR (Evidence Partners, Ottawa, Canada), where we identified and removed duplicate references. Two independent reviewers evaluated titles and abstracts in random order. If either reviewer deemed a study potentially eligible, a full text review was completed—also by two independent reviewers. Any discrepancies were resolved through consensus, with a third investigator consulted as necessary. An inclusion and exclusion coding guide was developed and pre-tested, and team members were trained over several sessions (see supplementary material 3).

Data extraction and synthesis

One reviewer extracted data from each included study using a standardised form in DistillerSR, and a second reviewer validated the data using the DistillerSR Quality Control function. Reviewers extracted publication characteristics (eg, first author, publication year, journal); population characteristics, including eligibility criteria, recruitment method, number of participants, and population group (general population, older adults, young adults, children and adolescents, parents, university students, people with pre-existing medical conditions, people with pre-existing mental health conditions, medical staff, and groups defined by sex or gender in the present report, although we extracted any group for which we found data); mental health outcomes and assessment timing; and adequacy of study methods and reporting. We used World Bank classifications for country income and region.19 To assess risk of bias and adequacy of study methods and reporting (sampling frame, recruitment methods, sample size, setting and participant descriptions, participation or response rate, outcome assessment methods, standardisation of assessments, statistical analyses, follow-up rate), we adapted the Joanna Briggs Institute Checklist for Prevalence Studies20 (see supplementary material 4).

For continuous outcomes, we extracted estimates as the SMD effect size with 95% confidence intervals for change from pre-covid-19 to during the pandemic. If such data were not provided, we calculated Hedges g,21 as described elsewhere.22 Positive SMD estimates represent a worsening of mental health and negative estimates represent an improvement. For proportions, we calculated missing 95% confidence intervals using Agresti and Coull’s approximate method for binomial proportions.23 For proportion changes, we generated missing 95% confidence intervals using Newcombe’s method for differences between binomial proportions based on paired data,24 which requires the number of participants above a threshold to be known at both assessment points. If these data were not available, we assumed that 50% of cases above a threshold during pre-covid-19 assessments continued to be above the threshold during the pandemic. We confirmed that results did not differ substantively if we assumed values within a plausible range (30-70%).

Owing to pitfalls in interpreting proportions of participants who crossed a dichotomous threshold, we prioritised continuous data (see box 1). For each population group with continuous outcomes for at least two studies in a domain, we pooled SMDs through restricted maximum likelihood random effects meta-analysis. Heterogeneity was assessed using the I2 statistic. For studies with more than one continuous outcome in a domain (eg, two depression symptom measures), we pooled SMDs within the study before fitting the meta-analysis.

Box 1. Interpreting standardised mean differences effect sizes and changes in proportion above a threshold on mental health measures.

Changes in symptoms assessed with mental health patient reported outcome measures in covid-19 have been reported as changes in continuous scores and as the proportion of study participants above a threshold. Continuously measured symptom changes are presented in terms of standardised mean difference (SMD), which describe change in terms of within group standard deviations rather than raw change scores, which are measure specific and not easily compared across measures. To illustrate, the figure below shows the amount of change, assuming a normal distribution, for a SMD of 0.25. The hypothetical purple distribution represents pre-covid-19 scores and the orange distribution represents post-covid-19 scores, with a mean symptom increase of SMD 0.25. With a threshold located at 1 standard deviation above the pre-covid-19 mean, the proportion of participants above the threshold would change from 16% to 23%. With a threshold 2 standard deviations above the pre-covid-19 mean, the proportion would change from 2% to 4%.

graphic file with name suny074224.fa.jpg

Open in a new tab

When studies report an increase or decrease in the proportion of participants above a measure threshold, dichotomous thresholds used for this purpose are sometimes labelled as thresholds for clinically significant symptoms or as reflecting the presence of a condition, such as depression.7 These designations are not, however, based on evidence that a threshold represents a meaningful divide between impairment and non-impairment and do not reflect the presence of a mental disorder. Most commonly, these designations reflect a point on a measure that balances sensitivity and specificity when used for screening, which does not inform when score levels might become clinically meaningful.7 8 9 10 11

Thresholds on different symptom measures are often located at different places in the symptom distribution. This can lead to divergent estimates of proportions crossing a threshold, depending on the measure used, rather than because of actual differences in symptom changes. As the figure shows, the same change in symptoms in a hypothetical study sample would result in a 7% increase in participants at or above the threshold on one measure (first vertical line, 1 standard deviation above pre-covid-19 distribution mean) but an increase of only 2% on another (second vertical line, 2 standard deviations above pre-covid-19 distribution mean).

Although we have prioritised interpretation of changes in continuous score, in the supplementary material we also report proportions above thresholds, as they can be informative, such as when they are reported for two time points in the same study or as an indicator if some level of change may have occurred. We have, however, avoided interpretation of the magnitudes of proportions above thresholds.

Meta-analyses were performed in R (R version 3.6.3, Rstudio Version 1.2.5042) using the rma.uni function in the metafor package.25 Forest plots were generated using the forest.rma function in metafor. We characterised changes as minimal (SMD <0.20), small (0.20), medium (0.50), or large (0.80).26

Patient and public involvement

Sarah Markham, an experienced patient advisor and member of The BMJ’s international patient panel, was a research team member from project inception. She provided input on the design of the study, underwent training on study procedures, and was involved in selection of eligible studies. She provided comments on the content of this article.

Results

Search results and study selection

As of 11 April 2022 our searches identified 94 411 unique citations. We excluded 92 457 after review of titles and abstracts and 1523 after full text review, leaving 431 studies with longitudinal data. Of those, 276 studies did not include pre-covid-19 data, 11 only assessed outcomes not included in the present report (eg, loneliness), one used the same outcome measure but for different time periods pre-covid-19 (worst month in the past year) and during the pandemic (past month), and six duplicated data from another study, leaving 137 unique studies from 134 cohorts (fig 1).

Fig 1.

Fig 1

Open in a new tab

Flow of studies through review

Characteristics of included studies

Supplementary material 5 provides a more detailed version of study characteristics and outcomes. Supplementary table 1 shows the characteristics of the included studies. All cohort studies reported covid-19 outcome data collected in 2020 (nine studies from Asia in January or February, 125 studies from Asia and elsewhere in March or later). Only three cohort studies reported results from multiple time points in 2020, and one also reported results from 2021. Overall, 105 (77%) studies were from high income countries, 28 (20%) studies from upper middle income countries, one (1%) study from a mix of high income and upper middle income country samples, three (2%) studies from lower middle income countries, and none from low income countries. Fifty two (38%) studies were from Europe and Central Asia, 46 (34%) from East Asia and the Pacific, 28 (20%) from North America, and 11 (8%) from other regions. Study populations comprised adults in 104 (76%) studies; children or adolescents in 30 (24%) studies, including 27 that focused mostly or entirely on adolescents (10-19 years); and a mixture of children (<9 years) and adolescents in three (2%) studies. No study focused only on children.

Risk of rias and adequacy of study methods and reporting

Supplementary table 2 shows ratings for risk of bias and adequacy of methods and reporting. Of the 137 included studies, 37 (27%) used sampling frames that closely represented the target population, 32 (23%) used census or random sampling methods, 13 (9%) had response rates of ≥75%, and 43 (31%) successfully followed up with ≥75% of participants or included methods to deal with loss to follow-up. “Yes” ratings were between 73% and 100% for sample size, participant and setting description, valid assessment methods, standard outcome collection methods, and appropriately analysed results.

Change in mental health symptoms

Supplementary tables 3-5 show changes in mental health symptoms for individual studies by population category: supplementary table 3 for general mental health symptoms, supplementary table 4 for anxiety symptoms, and supplementary table 5 for depression symptoms. Table 1 shows the results of meta-analyses. Among the three cohorts in which changes were assessed in March to June 2020 and again in September to November 2020 or in 2021, symptoms from later assessments were stable or improved from early 2020. No meta-analysis results changed in sensitivity analyes (see supplementary material 5).

Table 1.

Meta-analyses of continuous general mental health, anxiety symptoms, and depression symptoms by population group

General mental health Anxiety symptoms Depression symptoms
No of cohorts (No of participants) Hedges g SMD* (95% CI) I2 (%) No of cohorts (No of participants) Hedges g SMD* (95% CI) I2 (%) No of cohorts (No of participants) Hedges g SMD* (95% CI) I2 (%)
General population 11 (30 185) 0.11 (−0.00 to 0.22) 97 4 (2632) 0.05 (−0.04 to 0.13) 37 4 (3470) 0.12 (0.01 to 0.24) 81
Women or female participants 6 (10 329) 0.22 (0.08 to 0.35) 91 5† (3500) 0.20 (0.12 to 0.29) 41 7† (3851) 0.22 (0.05 to 0.40) 89
Men or male participants 6 (11 546) 0.11 (−0.12 to 0.35) 98 4 (1271) 0.07 (−0.01 to 0.14) 0 7 (3905) 0.01 (−0.14 to 0.16) 82
Older adults 11 (9960) −0.01 (−0.12 to 0.11) 93 6† (7193) 0.14 (−0.00 to 0.28) 93 7† (7419) 0.22 (0.06 to 0.38) 95
Young adults 2 (4221) 0.16 (−0.07 to 0.39) 96 2 (4602) 0.05 (−0.16 to 0.27) 95 4 (8043) 0.02 (−0.15 to 0.18) 96
University students 6‡§ (6957) 0.00 (−0.17 to 0.17) 95 16‡ (12 642) −0.07 (−0.21 to 0.06) 96 19‡ (26 164) 0.14 (0.01 to 0.26) 98
Children and adolescents 16 (11 505) 0.19 (−0.05 to 0.42) 99 8 (12 064) 0.02 (−0.12 to 0.16) 96 10 (11 679) 0.06 (−0.08 to 0.20) 96
Parents 3 (932) 0.39 (0.21 to 0.56) 57 1 (147) 0.25 (0.02 to 0.49) — 5 (1639) 0.15 (−0.05 to 0.35) 87
People with pre-existing medical conditions 12 (6511) 0.10 (−0.01 to 0.20) 86 11 (5775) 0.08 (−0.04 to 0.21)¶** 89 16 (21 594) 0.04 (−0.04 to 0.12)††‡‡ 90
People with pre-existing mental health conditions 2 (457) −0.22 (−0.35 to −0.09) 0 3 (12 362) 0.12 (−0.11 to 0.35) 80 3 (12 352) −0.05 (−0.08 to −0.03) 0
Medical staff — — — — — — 1 (180) 0.11 (−0.09 to 0.32) —
People from sexual or gender minority groups — — — 3 (3743) 0.23 (−0.09 to 0.54) 98 3 (3741) 0.19 (0.10 to 0.28) 67
Open in a new tab

See supplementary material for reference details of studies.

CI=confidence interval; SMD=standardised mean difference.

*

Positive effect sizes indicate worse mental health during covid-19 pandemic compared with pre-covid-19.

†

Cancer survivors and matched controls from Rentscher et als29 counted as one cohort although shown separately in forest plot.

‡

Students from Lithuania and Germany from Truskauskaite-Kuneviciene et als66 counted as one cohort although shown separately in forest plot.

§

Sensitivity analysis conducted with results from Savage et alS64 from October 2020 instead of April 2020; SMD 0.00 (95% CI −0.17 to 0.17), I2=94.8%.

Sensitivity analysis conducted with results from Henry et als118 from September to October 2020 instead of April 2020; SMD 0.05 (95% CI −0.04 to 0.14), I2=76%.

**

Sensitivity analysis conducted with results from Henry et als118 from March 2021 instead of April 2020; SMD 0.04 (95% CI −0.04 to 0.13), I2=76%.

††

Sensitivity analysis conducted with results from Henry et als118 from September to October 2020 instead of April 2020; SMD 0.03 (95% CI −0.05 to 0.11), I2=91%.

‡‡

Sensitivity analysis conducted with results from Henry et als118 from March 2021 instead of April 2020; SMD 0.03 (95% CI −0.05 to 0.11), I2=91%.

General mental health

The forest plots in supplementary figures 1a-k show the SMD change for general mental health in studies of general populations and subgroups. The estimated change in the general population cohorts was minimal and not significant (supplementary figure 1a; 11 cohorts, n=30 185; SMDchange 0.11, 95% confidence interval −0.00 to 0.22; I2=97%). Among subgroups, a small, significant worsening of general mental health was observed for women or female participants (supplementary figure 1b; six cohorts, n=10 329; SMDchange 0.22, 0.08 to 0.35; I2=91%) and a small to medium significant worsening for parents (supplementary figure 1h; three cohorts, n=932; SMDchange 0.39, 0.21 to 0.56; I2=57%). Symptoms improved by a small amount among people with pre-existing mental health conditions (supplementary figure 1j; two cohorts, n=457; SMDchange −0.22, −0.35 to −0.09; I2=0%). No other subgroup changes were statistically significantly different from zero. I2 across analyses was high (57% to 99%), except among people with pre-existing mental health conditions (0%).

Anxiety symptoms

The forest plots in supplementary figures 2a-l show the SMD change for anxiety symptoms in studies of general populations and subgroups. The estimated change in the general population cohorts was not significant and was close to zero (supplementary figure 2a; four cohorts, n=2632; SMDchange 0.05, −0.04 to 0.13; I2=37%). Anxiety symptoms worsened significantly by small amounts among women or female participants (supplementary figure 2b; five cohorts, n=3500; SMDchange 0.20, 0.12 to 0.29; I2=41%) and parents (one cohort, n=147; SMDchange 0.25, 0.02 to 0.49). Estimates were non-statistically significant and close to zero for all other subgroups. I2 ranged from 0% to 41% for the general population, women or female participants, and men or male participants but was higher for the other subgroups (80% to 98%).

Depression symptoms

The forest plots in supplementary figures 3a-m show the change in SMDs for depression symptoms in studies of general populations and subgroups. The estimated change in general population cohorts increased statistically significantly by a minimal amount (supplementary figure 3a; four cohorts, n=3470; SMDchange 0.12, 0.01 to 0.24; I2=81%). The estimated change also increased significantly by minimal to small amounts among women or female participants (supplementary figure 3b; seven cohorts, n=3851; SMDchange 0.22, 0.05 to 0.40, I2=89%), older adults (supplementary figure 3d; seven cohorts, n=7419; SMDchange 0.22, 0.06 to 0.38, I2=95%), university students (supplementary figure 3f; 19 cohorts, n=26 164; SMDchange 0.14, 0.01 to 0.26, I2=98%), and people who self-identified as belonging to a sexual or gender minority group (supplementary figure 3k, three cohorts, n=3741; SMDchange 0.19, 0.10 to 0.28; I2=67%). The estimated change improved minimally for people with pre-existing mental health conditions (supplementary figure 3j, three cohorts, n=12 352; SMDchange −0.05, −0.08 to −0.03; I2=0%). The I2 statistic was 0% for people with pre-existing mental health conditions and 67% to 98% in other analyses.

Discussion

We reviewed more than 94 000 citations and included 137 studies from 134 cohorts that compared mental health during the covid-19 pandemic with assessments pre-covid-19. All studies assessed covid-19 symptoms during at least one time point in 2020, which in most cases was in the first half of the year. Only three studies assessed symptoms in the first months of the pandemic (March to June 2020) and again in late 2020 (September to November), and only one reported results from 2021.

Many analyses showed substantial heterogeneity, which suggests that point estimates should be interpreted cautiously. Consistency did, however, exist across analyses in that most estimates of symptom changes were close to zero and not statistically significant, and changes that were identified were of minimal to small magnitudes. Among general population studies, we did not find changes in general mental health or anxiety symptoms, and the worsening of depression symptoms was minimal (SMDchange 0.12).

Among subgroups, women or female participants were the only group that experienced a worsening of symptoms across outcome domains; all by small amounts (SMDchanges 0.20 to 0.22). Depression symptoms worsened by minimal to small amounts for older adults, university students, and people who self-identified as belonging to a sexual or gender minority group, but not for other groups. Although based on small numbers of studies and participants, general mental health (three studies, n=932) and anxiety symptoms (one study, 147 participants) were shown to worsen for parents. General mental health and depression symptoms were shown to improve for people with pre-existing mental health conditions, but these findings were based on only two studies (n=457) for general mental health, and improvement was negligible even though statistically significant for depression symptoms (SMDchange 0.05). Among the three cohorts in which changes were assessed separately in early 2020 (March to June) and again later in 2020 (September to November) or in 2021, symptoms from later assessments were shown to be either stable or improve from early 2020.

Comparison with other studies

Our finding that mental health was either unchanged or worsened by minimal to small amounts in the general population and subgroups is consistent with results from a more limited systematic review of 65 longitudinal studies from early in the pandemic13 and somewhat smaller than a second systematic review of 43 studies.14 We know of only one study that has evaluated mental disorders using validated diagnostic methods. That study, from Norway,27 which was not eligible for our review, evaluated prevalence of current mental disorders in a series of cross sectional random samples collected from 28 January to 11 March 2020 (n=563, 15.4%, 95% confidence interval 12.5% to 18.8%), 12 March to 31 May 2020 (n=691, 9.0%, 7.1% to 11.4%), 1 June to 31 July 2020 (n=530, 14.3%, 11.5% to 17.5%), and 1 August to 18 September 2020 (n=370, 11.9%, 9.0% to 15.6%). The authors concluded that the prevalence of mental health disorders was stable or slightly decreased across the pandemic.

The largest study on suicide during the pandemic28 29 included monthly data from official government sources on suicide occurences from 21 countries from 1 January 2019 or earlier to 31 July 2020 and found no evidence of a statistically significant increase in any country or region; statistically significant decreases did, however, occur in 12 countries or regions.28 Results were similar in a subsequent update with data from 33 countries across the first nine to 15 months of the pandemic.29

We found that women or female participants experienced small negative changes, in aggregate, for general mental health, anxiety symptoms, and depression symptoms during the early part of the pandemic. This finding is consistent with a previous analysis of a subset of studies from the present review with direct within study comparisons between mental health of women or female participants and men or male participants (n=10 unique cohorts).30 That study found statistically significantly, albeit minimally greater, worsening of general mental health and anxiety symptoms among women than among men (SMDdifference-change 0.15 for both); changes in depressive symptoms were worse but not statistically significant for women (SMDdifference-change 0.12).30

Significant worsening of symptoms among women or female members of the population is of concern. This is an aggregate result that, even though small, suggests that the disproportionate effect of the pandemic on women or female members of the population has influenced mental health. In terms of vulnerabilities, most single parent families tend to be headed by women, and women earn less and are more likely to live in poverty than men.31 32 33 34 35 Women are also overrepresented in healthcare jobs and provide most family and elder care.31 32 33 34 Intimate partner violence towards women increased during the pandemic.36 The small overall change in mental health symptoms suggests that many women have been resilient but that among some an important worsening of symptoms occurred. Indeed, although most of our analyses found no changes or minimal to small negative changes in mental health, they do suggest that the pandemic negatively influenced the mental health of some people, which is consistent with, for example, reports of increased visits for mental health and sustance misuse.37 38

Nonetheless, the patterns of findings from our review, along with evidence on mental health disorders and suicide, converge to suggest that the effects of covid-19 on mental health are more nuanced than the “tsunami” descriptor or other similar terms used by some investigators and in many media articles.12 Short news cycles that emphasise bad news,39 40 anecdotes, and an uncritical reliance on cross sectional studies and unvalidated, difficult-to-interpret survey tools that inquire about mental health and wellbeing during the pandemic among convenience samples might at least partially explain this discrepancy. Illustrating the pitfalls of interpreting studies that ask questions about emotional reactions to covid-19, separate from mental health, a longitudinal study of 2345 young men from Switzerland found that depression symptoms and stress significantly decreased during covid-19 compared with pre-pandemic levels.41 The study also reported results from a series of unvalidated single items, administered only during the pandemic, that queried about emotional reactions to the pandemic and specifically assigned covid-19 as the cause (eg, “due to covid-19, I experienced . . .”); these items suggested high levels of distress, which became the focus of the study’s conclusions without mention of unchanged mental health indices. Together with the findings from our systematic review, this suggests that many or most people have experienced different aspects of covid-19 as highly unpleasant or distressing, that most people have been resilient, and that population level mental health has not changed by large amounts, although for some it has changed negatively.

Policy implications

The lack of evidence of a large scale decline in mental health so far in the context of covid-19 could be because people are resilient and have made the best of a difficult situation. Indeed, although evidence is limited, data suggest, for example, that suicide generally declines during periods of societal conflict.42 43 44 45 46 War and pandemics have different characteristics, but in both there is a shared threat and common focus on collective action to tackle that threat.

The minimal changes in mental health that we found could also reflect steps that governments have taken to support mental health. The World Health Organization, other pan-national organisations, and governments across the globe have produced strategies for dealing with mental health and have invested in resources to support public mental health,47 48 even in countries where mental health had not been a priority previously.49 50 It is not known to what degree these efforts have been effective, but it is possible that government action has played an important role. However, the negative changes that we detected in some groups, particularly for women and female cohorts, early in the pandemic, underline the need for continued surveillance to determine the degree to which negative mental health changes may be ongoing and require additional resources.

In terms of research, the results of our study underline gaps in mental health surveillance across countries. Since early in the pandemic, the need for high quality surveys with appropriately representative probabilistic sampling methods and pre-pandemic data has been emphasised.51 However, we found few examples of mental health surveillance frameworks that generated high quality data based on this type of sampling. Investment in more robust mental health surveillance mechanisms that can be used to identify and deal with mental health needs at all times, including in times of crisis, is needed.

Strengths and limitations of this review

Strengths of our systematic review include using rigorous best practice methods; searching nine databases, including two Chinese databases; no restrictions on language; and the ability to update findings as evidence emerged via our living systematic review approach (see https://www.depressd.ca/covid-19-mental-health). Our systematic review also has limitations that suggest some caution in interpreting results. Firstly, we did not peer review our search strategy given the urgency of generating mental health data early in the pandemic. An experienced librarian did, however, develop the search strategy and develop covid-19 search terms in collaboration with other experienced librarians. We reviewed studies included in other reviews and did not identify any that we missed; to the contrary, in our review we identified many more studies than other similar reviews with overlapping search periods.13 14 Secondly, aside from several population level randomly sampled surveys, most studies included in our systematic review had limitations related to study sampling frames and recruitment methods, response and follow-up rates, and missing follow-up data. Thirdly, we did not include repeated cross sectional studies, which may have excluded some studies with useful evidence. Fourthly, heterogeneity was high in most of the meta-analyses that we conducted. Fifthly, only a handful of studies reported results from late 2020, although the few studies that did suggested that symptoms were stable or reduced from earlier in the pandemic. Sixthly, although we were able to synthesise results from several vulnerable groups, including older adults and people with pre-existing medical conditions, there were few studies for other groups, such as people with low socioeconomic status, and there were no studies on children (ages 0-9 years). Similarly, there was little evidence from low income or lower middle income countries or from some areas of the world, such as sub-Saharan Africa. Seventhly, we did not include studies with fewer than 100 participants; however, in a previous covid-19 mental health systematic review with 65 total studies, studies with fewer than 100 participants comprised only 1% of total participants.13 Eighthly, the evidence base is rapidly evolving, and main results could change, although our own living systematic review format will allow updating as this occurs. Finally, we did not assess possible publication bias, although the largely null findings suggest that this was not likely to be an important factor.

Conclusions

We reviewed 137 studies with data from 134 unique cohorts. Across population groups, results suggest that, rather than a mental health crisis, at a population level there has been a high level of resilience during covid-19, and changes in general mental health, anxiety symptoms, and depression symptoms have been minimal to small with no changes detected in most analyses. There were few robust studies with vulnerable groups, however, and it is possible that some population groups experience mental health issues that differ from those of the general population or from other groups. The pandemic and the long term ramifications continue to affect societies across the world, and it will be important to continue to assess mental health. The pandemic has affected the lives of many people, and some are now experiencing mental health difficulties for the first time. Governments should continue to ensure that mental health supports are available and respond to population needs. We will update the results of this systematic review as more evidence accrues, with study results posted online (https://www.depressd.ca/covid-19-mental-health).

What is already known on this topic

  • Large numbers of studies and media reports conclude that covid-19 has led to widespread decline in population mental health

  • Most existing evidence reviews have been based on cross sectional studies and conclusions based on proportions of study respondents above thresholds on mental health measures

  • Such methods are not intended for estimating prevalence and can be highly misleading

What this study adds

  • Synthesised evidence from 137 studies that compared general mental health, anxiety symptoms, or depression symptoms during the pandemic with outcomes pre-covid-19 in the same participant cohort showed no negative changes in mental health at the general population level for general mental health or anxiety symptoms but a minimal worsening of depression symptoms

  • Among subgroups, women and female cohorts appear to have experienced worsening of general mental health, anxiety symptoms, and depression symptoms

  • These findings are consistent with evidence that women and female members of society have experienced a disproportionately greater burden from the pandemic

Web extra.

Extra material supplied by authors

Supplementary information: Supplementary material 1-5, tables 1-5, figures 1-3, and references of included studies

suny074224.ww.pdf (5.7MB, pdf)

Contributors: YS, YWu, DBR, AB, and BDT conceived and designed the study. JTB designed the database searches. KL and AK carried out the searches. YS, YWu, SF, TDS, LL, XJ, KL, YWang, AT, AK, CH, OB, DBR, SM, MA, ITV, DN, EW, AY, and BDT contributed to evaluation of study eligibility and data extraction. YS and SF coordinated the study. YS, YWu, BL, AB, and BDT analysed the data. BA, CF, MSM, SS, and GT contributed to interpretation of results as knowledge translation partners. YS and BDT drafted the manuscript. All authors provided a critical review and approved the final manuscript. BDT is the guarantor; he had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analyses. The corresponding author and attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: This study was funded by the Canadian Institutes of Health Research (CMS-171703, MS1-173070, GA4-177758, WI2-179944) and McGill Interdisciplinary Initiative in Infection and Immunity Emergency covid-19 Research Fund (R2-42). YWu and BL were supported by a Fonds de recherche du Québec – Santé (FRQS) postdoctoral training fellowship. TDS was supported by a Canadian Institutes of Health Research Masters Award, DBR was supported by a Vanier Canada Graduate Scholarship, AB was supported by FRQS senior researcher salary awards, and BDT was supported by a Tier 1 Canada Research Chair, all outside of the present work. No funder had any role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Competing interests: All authors have completed the ICJME uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare: support from the Canadian Institutes of Health Research and McGill Interdisciplinary Initiative in Infection and Immunity Emergency covid-19 Research Fund; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years. All authors declare no relationships or activities that could appear to have influenced the submitted work. BDT and AB declared that they were authors of an included study (see supplementary reference S118.)

Ethical approval. Not required.

The manuscript’s guarantor (BDT) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned and registered have been explained.

Dissemination to participants and related patient and public communities: Dissemination of findings will be facilitated via posting to the project website (https://www.depressd.ca/covid-19-mental-health), including an infographic with study results; other Canadian and international mental health and covid-19 knowledge exchange portals; a companion article from a patient partner; social media posting; a press release; and direct communication with members of governments.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data availability statement

Study data and statistical codes are available via the Open Science Framework (https://osf.io/96csg/).

References

  • 1.World Health Organization. Mental health and COVID-19. Accessed 29 Jan 2023. https://www.who.int/europe/emergencies/situations/covid-19/mental-health-and-covid-19.
  • 2.Centers for Disease Control and Prevention. Coping with Stress. Accessed 29 Jan 2023. https://www.cdc.gov/mentalhealth/stress-coping/cope-with-stress/index.html.
  • 3. Pfefferbaum B, North CS. Mental health and the Covid-19 pandemic. N Engl J Med 2020;383:510-2. 10.1056/NEJMp2008017.  [DOI] [PubMed] [Google Scholar]
  • 4.The DEPRESSD Project. Living systematic review of mental health in COVID-19. Accessed 29 Jan 2023. https://www.depressd.ca/covid-19-mental-health.
  • 5. Thombs BD, Bonardi O, Rice DB, et al. Curating evidence on mental health during COVID-19: A living systematic review. J Psychosom Res 2020;133:110113. 10.1016/j.jpsychores.2020.110113.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Glasziou PP, Sanders S, Hoffmann T. Waste in covid-19 research. BMJ 2020;369:m1847. 10.1136/bmj.m1847.  [DOI] [PubMed] [Google Scholar]
  • 7. Thombs BD, Kwakkenbos L, Levis AW, Benedetti A. Addressing overestimation of the prevalence of depression based on self-report screening questionnaires. CMAJ 2018;190:E44-9. 10.1503/cmaj.170691.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Levis B, Yan XW, He C, Sun Y, Benedetti A, Thombs BD. Comparison of depression prevalence estimates in meta-analyses based on screening tools and rating scales versus diagnostic interviews: a meta-research review. BMC Med 2019;17:65. 10.1186/s12916-019-1297-6.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Levis B, Benedetti A, Ioannidis JPA, et al. Patient Health Questionnaire-9 scores do not accurately estimate depression prevalence: individual participant data meta-analysis. J Clin Epidemiol 2020;122:115-128.e1. 10.1016/j.jclinepi.2020.02.002.  [DOI] [PubMed] [Google Scholar]
  • 10. Lyubenova A, Neupane D, Levis B, et al. Depression prevalence based on the Edinburgh Postnatal Depression Scale compared to Structured Clinical Interview for DSM DIsorders classification: Systematic review and individual participant data meta-analysis. Int J Methods Psychiatr Res 2021;30:e1860. 10.1002/mpr.1860.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Brehaut E, Neupane D, Levis B, et al. Depression prevalence using the HADS-D compared to SCID major depression classification: An individual participant data meta-analysis. J Psychosom Res 2020;139:110256. 10.1016/j.jpsychores.2020.110256.  [DOI] [PubMed] [Google Scholar]
  • 12.Bentall R. Has the pandemic really caused a ‘tsunami’ of mental health problems? The Guardian. 9 Feb 2021. Accessed 29 Jan 2023. https://www.theguardian.com/commentisfree/2021/feb/09/pandemic-mental-health-problems-research-coronavirus.
  • 13. Robinson E, Sutin AR, Daly M, Jones A. A systematic review and meta-analysis of longitudinal cohort studies comparing mental health before versus during the COVID-19 pandemic in 2020. J Affect Disord 2022;296:567-76. 10.1016/j.jad.2021.09.098.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14. Salanti G, Peter N, Tonia T, et al. The Impact of the COVID-19 Pandemic and Associated Control Measures on the Mental Health of the General Population. A Systematic Review and Dose–Response Meta-analysis. Ann Intern Med 2022;175:1560-71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15. Elliott JH, Synnot A, Turner T, et al. Living Systematic Review Network . Living systematic review: 1. Introduction-the why, what, when, and how. J Clin Epidemiol 2017;91:23-30. 10.1016/j.jclinepi.2017.08.010.  [DOI] [PubMed] [Google Scholar]
  • 16.Thombs B, Bonardi O, Rice D. Mental health during the COVID-19 pandemic: protocol for a living systematic review of symptom levels, factors associated with symptoms, and intervention effectiveness. Open Science Framework. https://osf.io/96csg/.
  • 17. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. 10.1136/bmj.n71.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.World Health Organization. Rolling updates on coronavirus disease (COVID-19) 2020. Accessed 29 Jan 2023. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/events-as-they-happen.
  • 19.The World Bank. World Bank Country and Lending Groups – World Bank Data Help Desk. Accessed 29 Jan 2023. https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups.
  • 20.Joanna Briggs Institute. The Joanna Briggs Institute Critical Appraisal tools for use in JBI Systematic Reviews: Checklist for Prevalence Studies. Accessed 29 Jan 2023. https://jbi.global/critical-appraisal-tools
  • 21. Hedges L. Estimation of effect size from a series of independent experiments. Psychol Bull 1982;92:490-9. 10.1037/0033-2909.92.2.490. [DOI] [Google Scholar]
  • 22. Borenstein M, Hedges LV, Higgins JPT, et al. , eds, Effect sizes based on means. Introduction to meta-analysis. Wiley & Sons, 2009. 10.1002/9780470743386.ch4. [DOI] [Google Scholar]
  • 23. Agresti A, Coull B. Approximate is better than “exact” for interval estimation of binomial proportions. Am Stat 1998;52:119-26. 10.2307/2685469. [DOI] [Google Scholar]
  • 24. Newcombe RG. Improved confidence intervals for the difference between binomial proportions based on paired data. Stat Med 1998;17:2635-50. .  [DOI] [PubMed] [Google Scholar]
  • 25. Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw 2010;36:1-48. 10.18637/jss.v036.i03. [DOI] [Google Scholar]
  • 26. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Routledge, 1988. [Google Scholar]
  • 27. Knudsen AKS, Stene-Larsen K, Gustavson K, et al. Prevalence of mental disorders, suicidal ideation and suicides in the general population before and during the COVID-19 pandemic in Norway: A population-based repeated cross-sectional analysis. Lancet Reg Health Eur 2021;4:100071. 10.1016/j.lanepe.2021.100071.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Pirkis J, John A, Shin S, et al. Suicide trends in the early months of the COVID-19 pandemic: an interrupted time-series analysis of preliminary data from 21 countries. Lancet Psychiatry 2021;8:579-88. 10.1016/S2215-0366(21)00091-2.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29. Pirkis J, Gunnell D, Shin S, et al. Suicide numbers during the first 9-15 months of the COVID-19 pandemic compared with pre-existing trends: An interrupted time series analysis in 33 countries. EClinicalMedicine 2022;51:101573. 10.1016/j.eclinm.2022.101573.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30. Dal Santo T, Sun Y, Wu Y, et al. Systematic review of mental health symptom changes by sex or gender in early-COVID-19 compared to pre-pandemic. Sci Rep 2022;12:11417. 10.1038/s41598-022-14746-1.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31. Connor J, Madhavan S, Mokashi M, et al. Health risks and outcomes that disproportionately affect women during the Covid-19 pandemic: A review. Soc Sci Med 2020;266:113364. 10.1016/j.socscimed.2020.113364.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.United Nations Sustainable Development Group. Policy Brief: The Impact of COVID-19 on Women. Accessed 29 Jan 2023. https://unsdg.un.org/resources/policy-brief-impact-covid-19-women.
  • 33.Women UN. From insights to action: Gender equality in the wake of COVID-19. Accessed 29 Jan 2023. https://www.unwomen.org/en/digital-library/publications/2020/09/gender-equality-in-the-wake-of-covid-19.
  • 34.de Paz C, Muller M, Munoz Boudet AM, et al. Gender Dimensions of the COVID-19 Pandemic. World Bank Open Knowledge Repository, 16 Apr 2020. http://hdl.handle.net/10986/33622
  • 35.United States Census Bureau. Custodial Mothers and Fathers and Their Child Support: 2015. Accessed 29 Jan 2023. https://www.census.gov/library/publications/2018/demo/p60-262.html.
  • 36. Piquero AR, Jennings WG, Jemison E, Kaukinen C, Knaul FM. Domestic violence during the COVID-19 pandemic - Evidence from a systematic review and meta-analysis. J Crim Justice 2021;74:101806. 10.1016/j.jcrimjus.2021.101806.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Myran DT, Cantor N, Rhodes E, et al. Physician health care visits for mental health and substance use during the COVID-19 pandemic in Ontario, Canada. JAMA Netw Open 2022;5:e2143160. 10.1001/jamanetworkopen.2021.43160.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Saunders NR, Kurdyak P, Stukel TA, et al. Utilization of physician-based mental health care services among children and adolescents before and during the COVID-19 pandemic in Ontario, Canada. JAMA Pediatr 2022;176:e216298. 10.1001/jamapediatrics.2021.6298.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Pinker S. The media exaggerates negative news. This distortion has consequences. The Guardian. 17 Feb 2018. https://www.theguardian.com/commentisfree/2018/feb/17/steven-pinker-media-negative-news
  • 40.BBC. Psychology: Why bad news dominates the headlines. 29 Jul 2014. https://www.bbc.com/future/article/20140728-why-is-all-the-news-bad
  • 41. Marmet S, Wicki M, Gmel G, et al. The psychological impact of the COVID-19 crisis on young Swiss men participating in a cohort study. Swiss Med Wkly 2021;151:w30028. 10.4414/SMW.2021.w30028.  [DOI] [PubMed] [Google Scholar]
  • 42. Lester D. Suicide rates before, during and after the world wars. Eur Psychiatry 1994;9:262-4. https://psycnet.apa.org/record/1995-25489-001 10.1017/S092493380000362X. [DOI] [Google Scholar]
  • 43. Lester D. The effect of war on suicide rates. A study of France from 1826 to 1913. Eur Arch Psychiatry Clin Neurosci 1993;242:248-9. 10.1007/BF02189971.  [DOI] [PubMed] [Google Scholar]
  • 44. Somasundaram DJ, Rajadurai S. War and suicide in northern Sri Lanka. Acta Psychiatr Scand 1995;91:1-4. 10.1111/j.1600-0447.1995.tb09733.x.  [DOI] [PubMed] [Google Scholar]
  • 45. Aida T. Revisiting suicide rate during wartime: Evidence from the Sri Lankan civil war. PLoS One 2020;15:e0240487. 10.1371/journal.pone.0240487.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46. Marshall J. Political integration and the role of war in suicide. Soc Forces 1981;59:771-85. 10.2307/2578193. [DOI] [Google Scholar]
  • 47. McCartan C, Adell T, Cameron J, et al. A scoping review of international policy responses to mental health recovery during the COVID-19 pandemic. Health Res Policy Syst 2021;19:58. 10.1186/s12961-020-00652-3.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. Arendt F, Markiewitz A, Mestas M, Scherr S. COVID-19 pandemic, government responses, and public mental health: Investigating consequences through crisis hotline calls in two countries. Soc Sci Med 2020;265:113532. 10.1016/j.socscimed.2020.113532.  [DOI] [PubMed] [Google Scholar]
  • 49. Ju Y, Zhang Y, Wang X, Li W, Ng RMK, Li L. China’s mental health support in response to COVID-19: progression, challenges and reflection. Global Health 2020;16:102. 10.1186/s12992-020-00634-8.  [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Wang V. China long avoided discussing mental health. The pandemic changed that. New York Times. 21 Dec 2020. Accessed 29 Jan 2023. https://www.nytimes.com/2020/12/21/world/asia/china-covid-mental-health.html.
  • 51. Pierce M, McManus S, Jessop C, et al. Says who? The significance of sampling in mental health surveys during COVID-19. Lancet Psychiatry 2020;7:567-8. 10.1016/S2215-0366(20)30237-6.  [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary information: Supplementary material 1-5, tables 1-5, figures 1-3, and references of included studies

suny074224.ww.pdf (5.7MB, pdf)

Data Availability Statement

Study data and statistical codes are available via the Open Science Framework (https://osf.io/96csg/).

Articles from The BMJ are provided here courtesy of BMJ Publishing Group

RESOURCES