Skip to main content
PMC home page
PLOS ONE logoLink to PLOS ONE
. 2021 Jul 9;16(7):e0253116. doi: 10.1371/journal.pone.0253116

Government responses and COVID-19 deaths: Global evidence across multiple pandemic waves

Thomas Hale 1,*,#, Noam Angrist 1, Andrew J Hale 2,#, Beatriz Kira 1,‡, Saptarshi Majumdar 1,#, Anna Petherick 1,‡, Toby Phillips 1,‡, Devi Sridhar 3,‡, Robin N Thompson 4,5,‡, Samuel Webster 6,‡, Yuxi Zhang 1,#
Editor: Holly Seale7
PMCID: PMC8270409  PMID: 34242239

Abstract

We provide an assessment of the impact of government closure and containment measures on deaths from COVID-19 across sequential waves of the COVID-19 pandemic globally. Daily data was collected on a range of containment and closure policies for 186 countries from January 1, 2020 until March 11th, 2021. These data were combined into an aggregate stringency index (SI) score for each country on each day (range: 0–100). Countries were divided into successive waves via a mathematical algorithm to identify peaks and troughs of disease. Within our period of analysis, 63 countries experienced at least one wave, 40 countries experienced two waves, and 10 countries saw three waves, as defined by our approach. Within each wave, regression was used to assess the relationship between the strength of government stringency and subsequent deaths related to COVID-19 with a number of controls for time and country-specific demographic, health system, and economic characteristics. Across the full period of our analysis and 113 countries, an increase of 10 points on the SI was linked to 6 percentage points (P < 0.001, 95% CI = [5%, 7%]) lower average daily deaths. In the first wave, in countries that ultimately experiences 3 waves of the pandemic to date, ten additional points on the SI resulted in lower average daily deaths by 21 percentage points (P < .001, 95% CI = [8%, 16%]). This effect was sustained in the third wave with reductions in deaths of 28 percentage points (P < .001, 95% CI = [13%, 21%]). Moreover, interaction effects show that government policies were effective in reducing deaths in all waves in all groups of countries. These findings highlight the enduring importance of non-pharmaceutical responses to COVID-19 over time.

Introduction

The COVID-19 pandemic has upended healthcare, cultural, financial, and government systems worldwide. While vaccines are being deployed rapidly one year after the outbreak began, control of the COVID-19 pandemic continues to rely largely on government non-pharmaceutical interventions (NPIs) [1–6]. Such interventions have proliferated worldwide, including school closings, travel restrictions, public gathering bans, and stay-at-home orders [7]. These policies aim to create physical distancing or otherwise slow the spread of COVID-19, often in concert with testing and contact tracing regimes of varying robustness [3, 8–11]. In some cases, closure and containment measures have been extreme, with unprecedented social, cultural, and financial implications [12–14]. Governments have varied significantly in both the degree of their interventions and how quickly they adopt them [15, 16].

Robust evidence now shows that, under most conditions, early adoption of stringent NPIs is associated with a reduction in transmission [16–27]. However, with full vaccination still elusive, and with the possibility of vaccine-escaping variants emerging, NPIs can be expected to continue playing a significant role in managing the pandemic in the foreseeable future [28–31]. A critical question is therefore whether we can expect NPIs to continue working as the pandemic stretches into longer time frames. On the one hand, we might expect effectiveness to improve over time as governments learn how to better calibrate and target policies [32, 33]. On the other hand, we may expect the opposite to the extent individuals grow tired of COVID-19 restrictions, as responses become politicized, or as economic disruption grows more severe [34–39].

As a first step toward investigating these trends, we explore the average effects of NPIs on the spread of the disease globally. We present a global analysis of governments’ responses to date, and a global assessment of their relationship to the spread of the pandemic over time. We tracked 186 governments’ responses across a series of non-pharmaceutical interventions and created a composite index that captured how, over time, each country’s government responded. Looking at peaks and troughs in countries’ death tolls, we divide each outbreak into successive waves, limiting our analysis to the 113 countries that recorded at least one death per day, on average, in the period of analysis. We hypothesized that the overall stringency of governments’ interventions would affect the subsequent rate of deaths related to COVID-19 in each wave, comparing the effects of policies and deaths both across countries and across waves. This article seeks to make three contributions. First, we explore whether the effects of policies on deaths varies across each wave of the pandemic. Second, we develop an approach to operationalize in observational analysis an approach to variation in the effect of policy over time, a feature of some epidemiological models [40]. Third, our analysis includes a global distribution of countries experiencing significant outbreaks of COVID-19.

Methods

Data collection

We collected information on 186 national governments’ responses across a range of NPIs (see S1 Table in S1 File). These measures were recorded for each day in each country, creating a measure of variation in government responses both across countries and across time. Data were collected by the authors and trained research assistants from publicly available sources such as news articles, government briefings, and international organizations. Data collectors coded government responses on a simple binary or ordinal scale registering the stringency of a given policy. Indicators C1-C7 and H1 (see S1 Table in S1 File) were further classified as either “targeted” (meaning they apply only in a geographically concentrated area) or “general” (meaning they apply throughout the entire jurisdiction). The data cover over 186 countries from January 1, 2020 onwards, though we limit our analysis up until March 11th, 2021 and to only the 113 countries that experienced one or more deaths per day, on average, during the period of analysis. Because we do not use human data or tissue, or involve human subjects, approval by the university review board was not required. To ensure accuracy and consistency, data collectors were required to pass an online training and to participate in regular team review meetings. Each data point was verified by at least two data collectors independently, and includes notes and source materials to substantiate each observation. Importantly, for all NPIs, we record only the official policies at the national level, not how well they are implemented or enforced.

Measuring government non-pharmaceutical interventions

Our primary measure of governments’ NPIs is a composite Stringency Index (SI) that records the number and restrictiveness of government containment and closure measures, calculated as follows. For each of the nine relevant policy response indicators, we create a score by taking the ordinal value, adding 0.5 if the policy is general rather than targeted, and rescaling each of these to range from 0–100. Conservatively, we assign a score of 0 to any indicators missing data, and reject any country-days where more than one of the indicators is missing. The mean of these nine scores gives the composite SI.

We rely chiefly on this simple, unweighted SI because this approach is most transparent and easiest to interpret [41]. Composite indices have the value of facilitating comparison across countries, albeit with the trade-off of condensing information. In practice, however, we observe that most countries in the time period of analysis adopted most NPIs as a package, further justifying the use of a composite measure [7]. The robustness of SI is also supported by the high correlations between this and alternative indexes in both level of stringency and shifts over time [42].

The degree of government response is measured by the value of the SI for a country on a particular day, or, in the cross-sectional analyses, the average level of stringency from January 1, 2020 to March 11th, 2021.

In addition to the nine indicators that comprise the stringency index, we include as a control measures of governments’ testing and contact tracing (S1 Table in S1 File). This controls for the potential that effects on deaths are confounded with increased identification of COVID-19 cases and deaths. These are also recorded on an ordinal scale representing the breadth and thoroughness of the policy. Additionally, we use an additional index, the government response index, as a robustness measure. The government response index consolidates an expanded set of indicators, including, for example, economic support and vaccination policies.

Dividing countries into outbreak weaves

We identify each country’s respective outbreak “waves.” Accounting for specific waves is critical. First, this is important to answer the question of whether the effectiveness of government policies is sustained over time and across different phases of the pandemic. Second, accounting for waves has a methodological advantage of enabling more precise calibration to the specific slope of the curve in each wave, rather than applying a one-size-fits-all estimation technique across all waves. Even within a country, waves often have substantially different peaks, troughs, and slopes.

Studies on COVID-19 and the previous ten virus-driven pandemics since 1889 have found similar wave patterns [43, 44]. There is no scientific agreement on the definition of a “wave”, or on the question of whether the driving force of this pattern is human behaviour or the spread of the virus [45, 46]. Yet, the literature includes various models to predict future waves, and some studies elaborate different time-variant associations between the spread of the virus and non-epidemic dynamics such as human behaviour, government measures and enforcement, as well as environmental factors, such as temperature [33, 47–51]. Some studies find COVID-19 waves can share characteristics, such as the vulnerability of the elderly population; others find the demographic structure of cases and deaths shift substantially across waves [52, 53]. Researchers have also found that different waves in a single country may be dominated by clusters and outbreaks in dispersed locations [54].

While the literature does not offer a precise definition of “waves”, there is a broad consensus that a wave is a phase of disease that is more substantial than a “sporadic outbreak”, and that comprises a rising phase and a subsequent falling phase [46, 55]. As a first step in identifying waves, we therefore locate peaks and troughs for each wave in each country and split waves between troughs accordingly. We smooth curves using a local non-parametric polynomial regression, a generalized form of a moving average, to avoid misidentification of waves due to noisy data or fluctuation in reporting of deaths. Specifically, we use locally weighted scatterplot smoothing, also known as LOWESS regression. We identify waves as the point before and after troughs. We consider waves distinct if their troughs are more than approximately one month apart, otherwise we include this together as one wave. We chose this threshold to correspond to the lag we hypothesize between NPIs and deaths, discussed below. In addition, we only count a wave if more than 20 cases occurred in that given wave (to distinguish a wave from a sporadic outbreak). Given the lack of consensus in the literature on how to define a wave, we believe the approach selected is transparent and simple, facilitating analysis, but we do not claim it is the only or best way to divide the phases of the pandemic. Finally, we consider the period from early January 2020 to March 11th, 2021. Since then, additional waves have emerged in multiple countries, however, these waves are ongoing and the effects of policies on deaths during these waves are not yet able to be fully detected.

For example, Fig 1 below shows peaks and troughs in India, South Africa, and the United States and. As of March 2021, India has one major and extended wave from April 2021-January 2022; South Africa had 2 waves: the first in July-September, and the second in December-February; and the United States had 3 waves: the first in April-May, the second in July-August, and the third in December-February. These examples clearly highlight the need to take a data-driven approach to identifying waves. Waves occur in different countries at different points in time, and last for different lengths of time. Moreover, a standardized approach is needed to draw boundaries between waves and sporadic outbreaks or “noisy” data. Our approach is certainly not the only way to identify waves, but provides a useful tool to facilitate analysis of the effect of NPIs.

Fig 1. Identifying waves of COVID-19 in different countries.

Fig 1

Open in a new tab

India, South Africa, and the United States exemplify the need for data-driven distinctions for enumerating waves, which occurred in different months, to different degrees and over different time horizons.

Outcome variables

We estimate the relationship between government interventions and the intensity of the COVID-19 outbreak by country. Information on confirmed COVID-19 cases and deaths were taken from John Hopkins University [56]. However, the true number of cases and deaths, as well as the reproduction number, are the subjects of significant uncertainty and a major topic of ongoing research [12, 57–59]. Observationally, the true number of cases is difficult to measure consistently because different countries have tested for COVID-19 more or less widely, and report case information in variable ways [3, 60]. We take a conservative approach and use recorded deaths as our main outcome, which we expect to be reported more consistently and captures the public health consequence of the epidemic most directly.

We analyse deaths in terms of the daily natural log in deaths in a country. We use the log transformation to account for the exponential trajectory of the epidemic growth curve. This specification also facilitates analysis because the first differences in logs approximate percentage changes, meaning the coefficients from our regressions are readily interpretable.

Statistical analysis

Our approach aligns with well-established empirical methods to study how government measures address historical epidemics and during the COVID-19 pandemic [61, 62]. We estimate how the degree of government response relates to changes in deaths using Ordinary Least Squares (OLS) regression, with the country-day as the unit of analysis. We include country fixed effects to control for variations in country-specific factors. To this end, effects of policies are estimated based on within country changes in deaths for 113 countries [63]. Importantly, country fixed effects control for all country-specific characteristics that do not vary over the period of analysis, such as the level of wealth, pre-existing robustness of the health care system, the government’s overall capacity to implement policy, or the population’s general tendency to follow government advice or not. We also control for deaths at the beginning of the time period of analysis to account for variations in baseline deaths in a given country and wave. As a robustness check to ensure county fixed effects models are not biased, we also estimate models that directly include controls for country-specific factors using data from the World Bank; these controls include adult mortality rates, hospital bed per 1000 people, number of physicians per 1000 people, prevalence of diabetes, population density, total population, and GDP per capita. Additional robustness checks use subnational jurisdictions in countries with substantial policy heterogeneity, such as the United States and Brazil. We also include robustness tests with an alternative index, the Government Response Index, as well as including controls for testing and contact tracing policies.

We estimate these models across all country-days during the entire period of analysis from January 2020 to March 11th 2021. We include countries with at least one death per day on average over this period. Since effects are identified based on within country changes in deaths, countries that had very few deaths will not have enough variation to enable a credible analysis. If we did not include country fixed effects, identification would also be derived from across-country differences in policy and resulting deaths. However, we avoid this estimation approach since it introduces bias in the form of confounded variables such as wealth, age-structure, and health care capacity variation across countries. 113 countries are included in the final sample. We estimate wave-specific effects using interaction terms for each wave, interacted with the stringency index.

This kind of observational study faces two key inferential challenges: how to control for the effect of time, including the “natural” growth and diminishing of the disease, which is unobserved, as well as the possibility of reverse causality or other forms of endogeneity. For example, a positive relationship between current stringency and new deaths does not necessarily mean stringency increases deaths; rather deaths might trigger a policy response. We address these issues in three ways.

First, we lag the explanatory variable by four weeks. This period reflects our best estimate, based on the existing literature, of the lag between behavioural change, transmission, the emergence of disease, and, ultimately, death. Studies find that NPIs significantly reduces the reproduction number of SARS-CoV-2 in the subsequent 28 days after introducing the policies, provided public events are banned throughout [64]. However, the event of introducing a lockdown contributes very little to mitigating a pandemic wave after 30 days [65]. Second, we use log of deaths as the dependent variable, approximating the exponential growth rate of the epidemiological curve. Third, we include a time trend in regressions to account for natural growth patterns in deaths as well as normalize the time period to account for the time in which the epidemic reached different countries across the globe. With this specification, which is now common in the literature, we identify likely effects of NPIs on the number of deaths from COVID-19. However, we suggest a modest interpretation of causality given the inherent challenges of identification for this research design.

Results

Variation in the speed and intensity of government responses and waves of disease

We observed significant variation in both the level of stringency and the time at which policies are adopted across national governments. While there is a near-universal increase in countries adopting containment and closure measures over time, with most countries moving to stringent measures after the first week of March 2020, the varied spread of the disease globally means that some countries adopted “lockdown” measures before local transmission began and some after. As the pandemic has progressed, countries increasingly differ in the NPIs they adopt. While information campaigns, international travel controls, and testing and contact measures tend to remain in place, closure and containment policies have waxed and waned7.

We also observe variation in how many waves of disease countries experience. At the time of writing, most countries (63) have experienced two waves, 40 have only experienced one and 10 have experienced three or more. The number of waves continues to evolve, revealing the need for a wave-specific analysis and motivating future work on this subject. In the Supplement, we include figures for a series of example country and their respective waves.

Regression results

Our core question is how NPIs, as measured by the Stringency Index, affect deaths in each country during each wave. Table 1 presents the results. Each coefficient shows how a one point increase in the stringency of government response four weeks prior relates to the difference in log of daily deaths in a given country on an average day. Further models are reported in the supplement as robustness checks (S2.1–S2.4 Tables in S1 File). Overall, the results strongly indicate that more stringent responses led to fewer deaths. Column (1) shows that 1 point of higher stringency 28 days in the past leads to an average reduction in deaths in all countries across all waves of -.006 (P < 0.001, 95% CI = [-0.005, -0.007]). In countries with one wave (column 2), we find an estimated reduction of -.005 (P < 0.001, 95% CI = [-0.003, -0.007]) with each additional point of stringency. In countries with a second wave (column 3), we find a reduction in deaths of -.004 (P < 0.001, 95% CI = [-0.002, -0.006]) in the first wave and -.008 (P < 0.001, 95% CI = [-0.006, -0.01]) in the second wave. In countries with a third wave (column 4), we find a reduction of -.021 (P < 0.001, 95% CI = [-0.017, -0.025]) in the first wave, -.031 (P < 0.001, 95% CI = [-0.027, -0.035]) in the second wave and -.028 (P < 0.001, 95% CI = [-0.024, -0.032]) in the third wave. All p-values in all cases are P < .001 indicating highly statistically significant results. Since first differences in logs approximate percentage changes and are asymptotically identical at small values, these effects can be interpreted as percentage points. This means that a 10-point difference in SI would be expected to lead, four weeks later, to 28 percentage point fewer deaths during the third wave. We further note a high R-squared of .75 to .81 across all regressions, indicating that our estimation model fits the data well. Because different groups of countries experienced one, two, or three waves of disease, it is not possible to compare the magnitudes of the coefficients across all columns in Table 1. However, the overall results clearly show an enduring relationship between policies and deaths across countries and across waves.

Table 1. Association of government response stringency and deaths by wave.

(1) (2) (3) (4)
Pooled Estimates for All Countries One-Wave Countries Two-Wave Countries Three-Wave Countries
LAGGED STRINGENCY BY 28 DAYS -0.006***
(0.001)
[0.000]
LAGGED STRINGENCY BY 28 DAYS: WAVE 1 -0.005*** -0.004*** -0.021***
(0.001) (0.001) (0.002)
[0.000] [0.000] [0.000]
LAGGED STRINGENCY BY 28 DAYS: WAVE 2 -0.008*** -0.031***
(0.001) (0.002)
[0.000] [0.000]
LAGGED STRINGENCY BY 28 DAYS: WAVE 3 -0.028***
(0.002)
[0.000]
COUNTRIES 113 40 63 10
R2 0.76 0.75 0.76 0.81
COUNTRY FIXED EFFECTS Yes Yes Yes Yes
TIME TREND Yes Yes Yes Yes
LAGGED DEATHS CONTROL Yes Yes Yes Yes
Open in a new tab

Notes: All regressions coefficients are included in the table, followed by standard errors in parentheses and p-values in square brackets. Stars signify statistical significance at conventional thresholds.

We also include a series of robustness tests in the S2.1–S2.4 Tables in S1 File. These robustness tests show analysis using subnational units for large heterogeneous countries such as the United States and Brazil. Additional robustness tests include specific controls, such as GDP hospital bed availability and comorbidities, testing and contract tracing policies, as well as using an alternative index, the government response index. All robustness tests show similar trends, with consistent relationship between more stringent NPIs and fewer deaths, and persistent effects across multiple waves of the pandemic.

Discussion

Our data show that government responses do indeed have a statistically robust and substantively significant relationship with deaths related to COVID-19. Moreover, this relationship endures across multiple waves of disease. At the same time, the findings reveals that the effect of NPIs has varied over time and across countries to some degree. This might be for a variety of reasons: for example, perhaps vaccine rollouts coupled with government policy counteracted potential fatigue in adherence with policies, resulting in sustained effectiveness of government policies in reducing deaths. In this paper our aim is not to identify precise reasons for differences in effectiveness across waves, which likely diverge across countries, but rather to identify the first-order patterns in the effectiveness of NPIs over time. This motivates future research into effectiveness of policies during specific waves of the pandemic.

Our study has several limitations. Like any policy intervention, the effect of the responses we measured is likely to be highly contingent on local political and social contexts. For instance, the state-by-state level response in the United States has been heterogeneous, while our primary analysis is conducted at the national level (however, as S2.3 Table in S1 File shows, these patterns also hold across subnational jurisdictions). Nor do we measure the extent to which government interventions are successfully implemented. In addition, the relationships reported do not account for potential confounders that might have otherwise reduced deaths, such as seasonality, climate, or spontaneous behavioural change in response to changing risk perceptions or social norms in the population. While these factors have not yet been fully established for COVID-19, if they are, they will need to be accounted for to more reliably estimate the effect of government policies on growth in deaths. In spite of these limitations, our approach offers a global and comprehensive view of governmental response to COVID-19 to date with the best information available. By measuring a range of indicators, composite indices mitigate the possibility that any one indicator may be over- or mis-interpreted. By the same token, composite measures also make strong assumptions about what kinds of information are included. If the information left out is systematically correlated with the outcomes of interest, or systematically under- or overvalued compared to other indicators, such composite indices may introduce measurement bias.

Our results are in line with findings of changes in NPIs and the development of pandemics over time. Multi-country analyses have found the implementation NPIs was significantly associated with an overall reduction in COVID-19 incidence [61]. An analysis of the effect of five policy responses on COVID-19 deaths in 11 European countries found a significant impact of interventions implemented several weeks before late March 2020, though these results were strongly driven by the experiences of Spain and Italy [16]. These findings hold in single-country studies and those focusing on subnational regions. A modelling study finds that without NPIs the number of COVID-19 cases would have been 51 to 125 folds higher in different cities and provinces in China [66]. The NPIs implemented in New York City were estimated to have reduced cases numbers by 72% and deaths by 76% [67]. While some studies shows the relationship between single NPI and the reduction in COVID-19 incidences or reproduction number, others try to identify the more effective blends of NPI sub-categories [68–70]. However, researchers also find the synergistic benefits of implementing a suite of multiple NPIs, which lends support to our approach of assessing the relationship between overall NPIs and deaths [71, 72].

Apart from the general association between NPIs and the spread of disease, researchers have also looked at how such relationships change over the course of previous pandemics. For example, earlier stringency measures decreased death rates in the 1918–1919 influenza pandemic by as much as 50% [73, 74]. That pandemic was characterised by three waves, with second and third waves occurring only after the relaxation of the “main battery of NPIs” [43, 75]. More recently, researchers start to look at how the association between NPIs and cases and deaths vary across different COVID-19 waves. A study investigating the performance of different NPIs across waves in 133 countries finds the most effective NPI blends change from gathering restrictions, facial coverings and school closures to facial coverings, gathering restrictions and international travel restrictions in the second wave. It concludes that the impact of NPIs had obvious spatiotemporal variations across countries by waves before vaccine rollouts [76]. According to a studying on 114 subnational areas in 7 European countries, the combined effectiveness of 17 NPIs on reducing local cases and deaths is still statistically significant, but the effect sizes in the second wave were smaller relative to that in the first wave [33]. The global analysis of varying association between NPIs and COVID-19 deaths in our paper echoes these findings and contributes to the emerging evidence base on how the relationship between NPIs and the spread of COVID-19 varies or remains constant over time.

Going forward, it will be important to continue monitoring government responses as the pandemic evolves. More granular analyses looking at the implementation and effectiveness of national policies, the role of individual measures and various combinations of policies, as well as the role of subnational governments or other social institutions can contribute further to this line of investigation. Further work should seek to explain variation in the effectiveness of NPIs across countries, examining both cross-sectional and longitudinal factors. Of particular importance, researchers can examine the role of path dependency, assessing how experiences in earlier waves affect the effectiveness of NPIs in later waves.

Supporting information

S1 File

(DOCX)

Click here for additional data file. (395.3KB, docx)

Acknowledgments

We are grateful to the strong support from students and staff at the Blavatnik School of Government and across Oxford University for contributing time and energy to data collection. We thank Rafael Goldszmidt, Andy Eggers, and Devika Singh for helpful comments.

Data Availability

All underlying data are freely available, and continuously updated, on the website of the Oxford COVID-19 Government Response Tracker: https://www.bsg.ox.ac.uk/research/research-projects/coronavirus-government-response-tracker.

Funding Statement

The authors received no specific funding for this work.

References

  • 1.Hunter DJ. Covid-19 and the Stiff Upper Lip—The Pandemic Response in the United Kingdom. N Engl J Med. 2020;382(16):e31. doi: 10.1056/NEJMp2005755 [DOI] [PubMed] [Google Scholar]
  • 2.Gates B. Responding to Covid-19—A Once-in-a-Century Pandemic? N Engl J Med. 2020;382(18):1677–1679. doi: 10.1056/NEJMp2003762 [DOI] [PubMed] [Google Scholar]
  • 3.Gostin LO, Wiley LF. Governmental Public Health Powers During the COVID-19 Pandemic. JAMA. 2020;323(21):2137–2138. doi: 10.1001/jama.2020.5460 [DOI] [PubMed] [Google Scholar]
  • 4.Cowling BJ, Aiello AE. Public Health Measures to Slow Community Spread of Coronavirus Disease 2019. J Infect Dis. 2020;221(11):1749–1751. doi: 10.1093/infdis/jiaa123 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Fong MW, Gao H, Wong JY, et al. Nonpharmaceutical Measures for Pandemic Influenza in Nonhealthcare Settings—Social Distancing Measures. Emerg Infect Dis. 2020;26(5):976–984. doi: 10.3201/eid2605.190995 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Haushofer J, Metcalf CJE. Which interventions work best in a pandemic? Science. Published online May 21, 2020. doi: 10.1136/science.abb6144 [DOI] [PubMed] [Google Scholar]
  • 7.Hale T, Angrist N, Goldszmidt R, et al. A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker). Nat. Hum. Behav. Published online March 8, 2021. doi: 10.1038/s41562-021-01079-8 [DOI] [PubMed] [Google Scholar]
  • 8.Viner RM, Russell SJ, Croker H, et al. School closure and management practices during coronavirus outbreaks including COVID-19: a rapid systematic review. Lancet Child Adolesc Health. 2020;4(5):397–404. doi: 10.1016/S2352-4642(20)30095-X [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Wang CJ, Ng CY, Brook RH. Response to COVID-19 in Taiwan. JAMA. 2020;323(14):341–1342. doi: 10.1001/jama.2020.3151 [DOI] [PubMed] [Google Scholar]
  • 10.Steinbrook R. Contact Tracing, Testing, and Control of COVID-19—Learning from Taiwan. JAMA Intern Med. 2020;180(9):1163–1164. doi: 10.1001/jamainternmed.2020.2072 [DOI] [PubMed] [Google Scholar]
  • 11.Chu DK, Akl EA, Duda S, et al. Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis. The Lancet. 2020;395(10242):1973–1987. doi: 10.1016/S0140-6736(20)31142-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Fauci AS, Lane HC, Redfield RR. Covid-19—Navigating the Uncharted. N Engl J Med. 2020;382(13):1268–1269. doi: 10.1056/NEJMe2002387 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Maxwell DN, Perl TM, Cutrell JB. “The Art of War” in the Era of Coronavirus Disease 2019 (COVID-19). Clin Infect Dis. 2020;71(16):2215–2217. doi: 10.1093/cid/ciaa229 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Esposito S, Principi N. School Closure During the Coronavirus Disease 2019 (COVID-19) Pandemic. JAMA Pediatr. 2020;174(10):921–922. doi: 10.1001/jamapediatrics.2020.1892 [DOI] [PubMed] [Google Scholar]
  • 15.Anderson RM, Heesterbeek H, Klinkenberg D, Hollingsworth TD. How will country-based mitigation measures influence the course of the COVID-19 epidemic? The Lancet. 2020;395(10228):931–934. doi: 10.1016/S0140-6736(20)30567-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Flaxman S, Mishra S, Gandy A, et al. Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe. Nature. 2020;584(7820):257–261. doi: 10.1038/s41586-020-2405-7 [DOI] [PubMed] [Google Scholar]
  • 17.Kupferschmidt K, Cohen J. Can China’s COVID-19 strategy work elsewhere? Science. 2020;367(6482):1061–1062. doi: 10.1126/science.367.6482.1061 [DOI] [PubMed] [Google Scholar]
  • 18.Chinazzi M, Davis JT, Ajelli M, et al. The effect of travel restrictions on the spread of the 2019 novel coronavirus (COVID-19) outbreak. Science. Published online March 6, 2020. doi: 10.1126/science.aba9757 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Pan A, Liu L, Wang C, et al. Association of Public Health Interventions With the Epidemiology of the COVID-19 Outbreak in Wuhan, China. JAMA. 2020;323(19):1915–1923. doi: 10.1001/jama.2020.6130 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Leung K, Wu JT, Liu D, Leung GM. First-wave COVID-19 transmissibility and severity in China outside Hubei after control measures, and second-wave scenario planning: a modelling impact assessment. The Lancet. 2020;395(10233):1382–1393. doi: 10.1016/S0140-6736(20)30746-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Davies NG, Kucharski AJ, Eggo RM, et al. Effects of non-pharmaceutical interventions on COVID-19 cases, deaths, and demand for hospital services in the UK: a modelling study. Lancet Public Health. 2020;5(7):e375–e385. doi: 10.1016/S2468-2667(20)30133-X [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.MacIntyre CR, Wang Q. Physical distancing, face masks, and eye protection for prevention of COVID-19. The Lancet. 2020;395(10242):1950–1951. doi: 10.1016/S0140-6736(20)31183-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Hsiang S, Allen D, Annan-Phan S, et al. The effect of large-scale anti-contagion policies on the COVID-19 pandemic. Nature. 2020;584(7820):262–267. doi: 10.1038/s41586-020-2404-8 [DOI] [PubMed] [Google Scholar]
  • 24.Caselli F, Grigoli F, Lian W, Sandri D. Protecting Lives and Livelihoods with Early and Tight Lockdowns. IMF Working Papers. Published online November 8, 2020. https://www.imf.org/en/Publications/WP/Issues/2020/11/08/Protecting-Lives-and-Livelihoods-with-Early-and-Tight-Lockdowns-49866 [Google Scholar]
  • 25.Chisadza C, Clance M, Gupta R. Government Effectiveness and the COVID-19 Pandemic. Sustainability. 2021;13(6). doi: 10.3390/su13063042 [DOI] [Google Scholar]
  • 26.Hosny A. The Sooner (and the Smarter), the Better: COVID-19 Containment Measures and Fiscal Responses. Covid Economics. Published online March 23, 2021. https://rule.asia/5d41145d48180046121b0109160105160f000719184a48081f0c125d3603134705360a0e1c1a010c4d12445a4f02110a.pdf#page=142 [Google Scholar]
  • 27.Allel K, Tapia-Muñoz T, Morris W. Country-level factors associated with the early spread of COVID-19 cases at 5, 10 and 15 days since the onset. Glob Public Health. 2020;15(11):1589–1602. doi: 10.1080/17441692.2020.1814835 [DOI] [PubMed] [Google Scholar]
  • 28.Moore S, Hill EM, Tildesley MJ, Dyson L, Keeling MJ. Vaccination and non-pharmaceutical interventions for COVID-19: a mathematical modelling study. Lancet Infect Dis. Published online March 18, 2021. doi: 10.1016/S1473-3099(21)00143-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Gozzi N, Bajardi P, Perra N. The importance of non-pharmaceutical interventions during the COVID-19 vaccine rollout. medRxiv. Preprint posted online January 9, 2021. doi: 10.1101/2021.01.09.21249480 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Wouters OJ, Shadlen KC, Salcher-Konrad M, et al. Challenges in ensuring global access to COVID-19 vaccines: production, affordability, allocation, and deployment. The Lancet. 2021;397(10278):1023–1034. doi: 10.1016/S0140-6736(21)00306-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Contreras S, Priesemann V. Risking further COVID-19 waves despite vaccination. Lancet Infect Dis. Published online March 18, 2021. doi: 10.1016/S1473-3099(21)00167-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Bontempi E. The europe second wave of COVID-19 infection and the Italy “strange” situation. Environ Res. 2021;193. doi: 10.1016/j.envres.2020.110476 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Sharma M, Mindermann S, Rogers-Smith C, et al. Understanding the effectiveness of government interventions in Europe’s second wave of COVID-19. medRxiv. Preprint posted online March 26, 2021. doi: 10.1101/2021.03.25.21254330 [DOI] [Google Scholar]
  • 34.Petherick A, Goldszmidt RG, Andrade EB, Furst R, Pott A, Wood A. A Worldwide Assessment of COVID-19 Pandemic-Policy Fatigue. SSRN. Preprint posted online February 1, 2021. doi: 10.2139/ssrn.3774252 [DOI] [Google Scholar]
  • 35.Goldstein P, Yeyati EL, Sartorio L. Lockdown Fatigue: The Diminishing Effects of Quarantines on the Spread of COVID-19.Covid Economics. Published online February 4, 2021. http://www.grade.org.pe/wp-content/uploads/CovidEconomics67-1.pdf#page=6 [Google Scholar]
  • 36.Brandtner C, Bettencourt LMA, Berman MG, Stier AJ. Creatures of the state? Metropolitan counties compensated for state inaction in initial U.S. response to COVID-19 pandemic. PLoS One. 2021;16(2):e0246249. doi: 10.1371/journal.pone.0246249 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Lasco G. Medical populism and the COVID-19 pandemic. Glob Public Health. 2020;15(10):1417–1429. doi: 10.1080/17441692.2020.1807581 [DOI] [PubMed] [Google Scholar]
  • 38.Chowdhury R, Luhar S, Khan N, Choudhury SR, Matin I, Franco OH. Long-term strategies to control COVID-19 in low and middle-income countries: an options overview of community-based, non-pharmacological interventions. Eur J Epidemiol. 2020;35(8):743–748. doi: 10.1007/s10654-020-00660-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Kusuma D, Pradeepa R, Khawaja KI, et al. Low uptake of COVID-19 prevention behaviours and high socioeconomic impact of lockdown measures in South Asia: Evidence from a large-scale multi-country surveillance programme. SSM Popul Health. 2021;13. doi: 10.1016/j.ssmph.2021.100751 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Rypdal K, Bianchi FM, Rypdal M. Intervention Fatigue is the Primary Cause of Strong Secondary Waves in the COVID-19 Pandemic. Int J Environ Res Public Health. 2020;17(24):9592. doi: 10.3390/ijerph17249592 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Greco S, Ishizaka A, Tasiou M, Torrisi G. On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness. Soc Indic Res. 2019;141(1):61–94. doi: 10.1007/s11205-017-1832-9 [DOI] [Google Scholar]
  • 42.Gros D, Ounnas A, Yeung TY-C. A New Covid Policy Stringency Index for Europe. Covid Economics. Published online January 28, 2021. https://joserobertoafonso.com.br/wp-content/uploads/2021/02/CovidEconomics66.pdf#page=120 [Google Scholar]
  • 43.Taubenberger JK, Morens DM. 1918 Influenza: The Mother of All Pandemics. Emerg Infect Dis. 2006;12(1):15–22. doi: 10.3201/eid1201.050979 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Honigsbaum M. The ‘Russian’ influenza in the UK: Lessons learned, opportunities missed. Vaccine. 2011;29:B11–B15. doi: 10.1016/j.vaccine.2011.03.063 [DOI] [PubMed] [Google Scholar]
  • 45.Cacciapaglia G, Cot C, Sannino F. Multiwave pandemic dynamics explained: how to tame the next wave of infectious diseases. Sci Rep. 2021;11(1):6638. doi: 10.1038/s41598-021-85875-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Zhang SX, Marioli FA, Gao R. A Second Wave? What Do People Mean by COVID Waves?–A Working Definition of Epidemic Waves. MedRxiv. Preprint posted online April 3, 2021. doi: 10.1101/2021.02.21.21252147 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Chowell G, Tariq A, Hyman JM. A novel sub-epidemic modeling framework for short-term forecasting epidemic waves. BMC Med. 2019;17(1):164. doi: 10.1186/s12916-019-1406-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Kaxiras E, Neofotistos G. Multiple Epidemic Wave Model of the COVID-19 Pandemic: Modeling Study. J Med Internet Res. 2020;22(7):e20912. doi: 10.2196/20912 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.He D, Dushoff J, Day T, Ma J, Earn DJD. Inferring the causes of the three waves of the 1918 influenza pandemic in England and Wales. Proc R Soc Lond B Biol Sci. 2013;280(1766). doi: 10.1098/rspb.2013.1345 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Aleta A, Martín-Corral D, Pastore y Piontti A, et al. Modelling the impact of testing, contact tracing and household quarantine on second waves of COVID-19. Nat Hum Behav. 2020;4(9):964–971. doi: 10.1038/s41562-020-0931-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Schmelz K. Enforcement may crowd out voluntary support for COVID-19 policies, especially where trust in government is weak and in a liberal society. PNAS. 2021;118(1):e2016385118. doi: 10.1073/pnas.2016385118 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Solis J, Franco-Paredes C, Henao-Martínez AF, Krsak M, Zimmer SM. Structural Vulnerability in the U.S. Revealed in Three Waves of COVID-19. Am J Trop Med Hyg 2020;103(1):25–27. doi: 10.4269/ajtmh.20-0391 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Ioannidis JPA, Axfors C, Contopoulos-Ioannidis DG. Second versus first wave of COVID-19 deaths: Shifts in age distribution and in nursing home fatalities. Environ Res. 2021;195. doi: 10.1016/j.envres.2021.110856 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Srivastava A, Chowell G. Modeling Study: Characterizing the Spatial Heterogeneity of the COVID-19 Pandemic through Shape Analysis of Epidemic Curves. Res Sq. Preprint posted online February 23, 2021. doi: 10.21203/rs.3.rs-223226/v1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Jefferson T, Heneghan C. Covid 19 –Epidemic ‘Waves.’ April 30, 2020. Accessed April 13, 2021. https://www.cebm.net/covid-19/covid-19-epidemic-waves/
  • 56.Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis. 2020;20(5):533–534. doi: 10.1016/S1473-3099(20)30120-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 57.Lipsitch M, Swerdlow DL, Finelli L. Defining the Epidemiology of Covid-19—Studies Needed. N Engl J Med. 2020;382(13):1194–1196. doi: 10.1056/NEJMp2002125 [DOI] [PubMed] [Google Scholar]
  • 58.Wallinga J, Teunis P. Different Epidemic Curves for Severe Acute Respiratory Syndrome Reveal Similar Impacts of Control Measures. Am J Epidemiol. 2004;160(6):509–516. doi: 10.1093/aje/kwh255 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Atkins KE, Wenzel NS, Ndeffo-Mbah M, Altice FL, Townsend JP, Galvani AP. Under-reporting and case fatality estimates for emerging epidemics. BMJ. 2015;350(mar163):h1115. doi: 10.1136/bmj.h1115 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Zou L, Ruan F, Huang M, et al. SARS-CoV-2 Viral Load in Upper Respiratory Specimens of Infected Patients. N Engl J Med. 2020;382(12):1177–1179. doi: 10.1056/NEJMc2001737 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Islam N, Sharp SJ, Chowell G, et al. Physical distancing interventions and incidence of coronavirus disease 2019: natural experiment in 149 countries. BMJ. Published online July 15, 2020. doi: 10.1136/bmj.m2743 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.Chernozhukov V, Kasahara H, Schrimpf P. Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S. J Econom. 2021;220(1):23–62. doi: 10.1016/j.jeconom.2020.09.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Dieleman JL, Templin T. Random-Effects, Fixed-Effects and the within-between Specification for Clustered Data in Observational Health Studies: A Simulation Study. PLoS One. 2014;9(10):e110257. doi: 10.1371/journal.pone.0110257 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Li Y, Campbell H, Kulkarni D, et al. The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries. Lancet Infect Dis. 2021;21(2):193–202. doi: 10.1016/S1473-3099(20)30785-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Oraby T, Tyshenko MG, Maldonado JC, et al. Modeling the effect of lockdown timing as a COVID-19 control measure in countries with differing social contacts. Sci Rep. 2021;11(3354). doi: 10.1038/s41598-021-82873-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Lai S, Ruktanonchai NW, Zhou L, et al. Effect of non-pharmaceutical interventions to contain COVID-19 in China. Nature. 2020;585(7825):410–413. doi: 10.1038/s41586-020-2293-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Yang J, Zhang Q, Cao Z, et al. The impact of non-pharmaceutical interventions on the prevention and control of COVID-19 in New York City. Chaos. 2021;31(2). doi: 10.1063/5.0040560 [DOI] [PubMed] [Google Scholar]
  • 68.IHME COVID-19 Forecasting Team. Modeling COVID-19 scenarios for the United States. Nat Med. 2021;27(1):94–105. doi: 10.1038/s41591-020-1132-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Askitas N, Tatsiramos K, Verheyden B. Estimating worldwide effects of non-pharmaceutical interventions on COVID-19 incidence and population mobility patterns using a multiple-event study. Sci Rep. 2021;11(1972). doi: 10.1038/s41598-021-81442-x [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Oruc BE, Baxter A, Keskinocak P, Asplund J, Serban N. Homebound by COVID19: the benefits and consequences of non-pharmaceutical intervention strategies. BMC Public Health. 2021;21(655). doi: 10.1186/s12889-021-10725-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Tian L, Li X, Qi F, et al. Harnessing peak transmission around symptom onset for non-pharmaceutical intervention and containment of the COVID-19 pandemic. Nature Commun. 2021;12(1147). doi: 10.1038/s41467-021-21385-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 72.Bo Y, Guo C, Lin C, et al. Effectiveness of non-pharmaceutical interventions on COVID-19 transmission in 190 countries from 23 January to 13 April 2020. Int J Infect Dis. 2021;102:247–253. doi: 10.1016/j.ijid.2020.10.066 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Bootsma MCJ, Ferguson NM. The effect of public health measures on the 1918 influenza pandemic in U.S. cities. PNAS. 2007;104(18):7588–7593. doi: 10.1073/pnas.0611071104 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Markel H, Lipman HB, Navarro JA, et al. Nonpharmaceutical Interventions Implemented by US Cities During the 1918–1919 Influenza Pandemic. JAMA. 2007;298(6):644–654. doi: 10.1001/jama.298.6.644 [DOI] [PubMed] [Google Scholar]
  • 75.Hatchett RJ, Mecher CE, Lipsitch M. Public health interventions and epidemic intensity during the 1918 influenza pandemic. PNAS. 2007;104(18):7582–7587. doi: 10.1073/pnas.0610941104 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 76.Ge Y, Zhang W, Liu H, et al. Effects of worldwide interventions and vaccination on COVID-19 between waves and countries. MedRxiv. Preprint posted online April 6, 2021. doi: 10.1101/2021.03.31.21254702 [DOI] [Google Scholar]

Decision Letter 0

Holly Seale

12 Oct 2020

PONE-D-20-20672

Global assessment of the relationship between government response measures and COVID-19 deaths

PLOS ONE

Dear Dr. Hale,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Nov 23 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Holly Seale

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. PLOS ONE publication criteria and journal policy require methods and data to be described in enough detail to allow suitably skilled investigators to fully replicate and evaluate your study.

In addition, authors must make all data underlying the findings described in their manuscript fully available without restriction.

We note that you state "Data were collected by the authors and trained research assistants from publicly available sources such as news articles, government briefings, and international organizations."

However, the sources are not listed and data is not provided.

Please include a table of the primary data used for your study with the sources of the data listed in a supplementary file.

3. Please amend your list of authors on the manuscript to ensure that each author is linked to an affiliation. Authors’ affiliations should reflect the institution where the work was done (if authors moved subsequently, you can also list the new affiliation stating “current affiliation:….” as necessary).

4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: No

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Here is a list of specific comments. Note: line and page numbering in reviews and comments is based on ruler applied in Editorial Manager-generated PDF.

1. There was no page number and line number. It might be difficult to locate the following comments precisely.

2. Methods, Data Collection: The section needed more details. There were a lot of words without details. The reproducibility might be low.

3. Methods, Data Collection, “data were collected by . . . ”: I suggest describing the data source in more details to ensure reproducibility.

4. Methods, Data Collection, “data collectors coded govenment responses . . . ”: For the ordinal scale, did you mean C1 in Table S1? If so, please refer it to Table S1. In addition, please provide more details on the binary scale.

5. Methods, Data Collection, “several indicators were . . . ”: Similarly, please refer “several indicators” to Table S1.

6. Methods, Data Collection, “the data over over 170 countries . . . ”: Would it be possible to list all countries (exactly how many countries) in the supplementary materials and indicate which countries’ data contributed to which parts of analyses?

7. Methods, Measuring the speed and degree of government non-pharmaceutical interventions, “foreach of the nine relevant policy response indicators . . . ”: I believed the relevant policy response indicators were listed in Table S1. If so, please point them out and spend some sentences describing them.

8. Methods, Measuring the speed and degree of government non-pharmaceutical interventions, “most countries in the time period of analysis adopted most NPIs as a package”: I failed to understand this from Figure S1.

9. Methods, Measuring the speed and degree of government non-pharmaceutical interventions: Were the principal component analysis and principal factor analysis suitable for non-normal variables (i.e., ordinal and/or binary)?

10. Methods, Measuring the speed and degree of government non-pharmaceutical interventions, “the average level of stringency from January 1, 2020 to May 27, 2020”: Why the average start January 1, 2020, but not the date when the first case recorded for a country?

11. Methods, Outcome Variables, “we analyze growth in deaths in terms of the daily log difference . . . ”: Was the difference from that of the previous day? What if the difference was negative?

12. Methods, Outcome Variables, “we consider the maximum daily number of new deaths . . . ”: Would the maximum daily number of new deaths be a fair measurement because not all countries recorded the first death on the same date? Wouldn’t the countries recorded the first death on a later date within the study window have a relatively smaller maximum? Should the maximum be measured at a fixed window since the first recorded death?

13. Methods, Statistical Analysis, “we estimate both cross-sectional models ...”: Somewhere in the Methods section, maybe the Data Collection section, I suggest introducing the cross-sectional and time-series study designs and relating the measuring the speed and degree of government non-pharmaceutical interventions and outcomes to these two study designs.

14. Methods, Statistical Analysis, “we estimate models that use both time and country fixed effects”: Were there any random effects in the longitudinal models?

15. Methods, Statistical Analysis, “we use the difference in logs as the dependent variable ...”: The outcome must be daily cumulative death events, not daily death events. Otherwise, how did you handle negative difference in the log transformation? What specific longitudinal models were used?

16. Results, Variation in the speed and intensity of government responses, “Figure 1 shows a positive correlation ...”: How did the residuals look like in this OLS regression? Was the OLS regression suitable to examine this relationship? This also applies to Figure 2.

17. Results, Variation in the speed and intensity of government responses: Shouldn’t there be a another model for the relationship between the government SI six weeks prior and the max daily new deaths?

18. Results, Variation in the speed and intensity of government responses, “Figure 4 explores these relationships . . . ”: What was the rationale to select these six countries?

19. Results, Regression Results: Please introduce the cross-country models in the Statistical Analyses section.

20. Figures 1–3: (a) Were all 170 countries included in this analysis? If not, please indicate how many countries were in this analysis and describe the reasons why some countries were not included. (b) What was the criteria for the 12 selected countries? (c) Please remind readers the y-axis was depicted in a log scale.

21. Table S2: Why were there 24 parameters (response indicators), not 9?

Reviewer #2: I apologize for the delay in responding to this review request, which I received on August 13, 2020. Prior to committing to this review, I notified the editor that I could not undertake the review in the prespecified 10-day review period; and that I have two potential competing interests -- namely, that I have met one of the co-authors of the manuscript (D. Sridhar) and am generally inclined to be favorable toward her work, and that (at the time of the review invitation) my own team had an article on this same exact topic being reviewed (Siedner et al. PLoS Med 2020;17:e1003244). In general, I thought that this manuscript was well written and obviously addresses an important public health issue. I also believe that the authors' release of their social distancing database into the public domain was a significant service to the medical/public health community. That said, there are a number of difficult issues related to the cross-sectional analysis and within-country heterogeneity that need to be addressed, however, and until they are, I am unsure as to what can be learned from the analysis. I do not see anything to suggest that this analysis is fatally flawed or that the analysis cannot be suitably revised. I am returning my review on September 11.

Major comments:

Broadly speaking, the authors conduct two sets of analyses: a cross-sectional analysis and a longitudinal analysis.

1. In the cross-sectional analysis, the authors calculate a Stringency Index that is averaged across the entire study period (Jan-May 2020), which is then used to calculate the two explanatory variables of primary interest: average stringency and speed of response. They then correlate these variables with the two outcome variables of primary interest: average daily growth in the death rate and zenith death rate. There are several concerns here:

a) In general, I would like to see more analyses teasing apart the general findings about the Stringency Index. Throughout the Results section, the authors write things like "a 10-point difference in SI would be expected to lead, six weeks later, to a daily growth rate in deaths nearly half a percentage point lower", but what changes would a country government need to implement to do that? eg Can they get there by closing schools and workplaces but keeping bars open?

b) The explanatory and outcome variables of primary interest, insofar as they compress longitudinal information into a single observation per country, mask considerable heterogeneity within countries, both over time (ie., temporal heterogeneity) as well as geographically (ie., regional heterogeneity).

c) It is unclear to me how the authors handled distancing measures that were implemented and then rescinded during the study period. For example, a country may have initiated a "lockdown" in March and then rescinded the lockdown in May. Do the authors use a "once on, always on"/"once treated, always treated" approach here? This is unclear to me from reading the Methods and should be clarified. I _think_ what the authors did is they calculated the Stringency Index for each day in each country, and then averaged across the days. So in a toy example with 3 days, if a country had a Stringency Index value of 0 on day 1, 50 on day 2, and 10 on day 3, the average Stringency Index would be calculated as (0+50+10)/3 = 20. In my understanding, if countries "turned on" and "turned off" social distancing measures at different time points during the study period, the Stringency Index would increase and decrease on a daily basis, but the average Stringency Index over the study period would largely reflect the maximum stringency as well as the number of days that social distancing was "turned on". Some difficulties with this specification can be better understood with a toy example. Supposing for example we have a 10-day study period in which Country A has a Stringency Index of 100 on days 1 & 2, then turns off social distancing completely and has a Stringency Index of 0 on days 3-10. The average Stringency Index for Country A through the study period would therefore be (100+100+0+...+0)/10 = 20. An alternative Country B might have limped along with a Stringency Index of 20 on each day throughout the study period, and would have an average Stringency Index of (20+20+...+20)/10 = 20. Under the authors' scoring, both countries would have equivalent values of the primary explanatory variable despite having arguably very different (formal) social distancing responses. Similarly, compressing the longitudinal data to create the cross-sectional outcome variables (average daily growth in the death rate and zenith death rate) also presents difficulties. Both China and Spain, for example, topped out at around 10,000 daily new cases (I don't know what their comparative death rates were)--and would therefore be treated equivalently in terms of the zenith outcome variable--but China has arguably been much more successful at containing their local epidemic compared with Spain, which is currently in a second surge.

d) For one of the explanatory variables (speed of response), the authors specified this variable as the number of days until the country achieved a Stringency Index of 40. They provide some text to justify this arbitrary threshold, which is appreciated, but I guess I am just not super convinced, partly because it is not even clear to me what a SI=40 means. The authors state that "almost every country reaches at least this threshold at some point"--were there specific components of the Stringency Index (eg., schools, public events) that drove this particular observation? Why not conduct some sensitivity analyses to see whether their findings are sensitive to the threshold choice? Or better yet, why not simply model the slope (ie., "speed of response") directly?

e) In terms of geographic heterogeneity, the authors did not account for the substantial within-country heterogeneity. The US, for example, would treated as a single country with a single observation for level of stringency and outcome. Obviously this is untrue, both in terms of the implementation of social distancing measures (Siedner et al. PLoS Med 2020;17:e1003244) as well as in the relaxation/rescinding of social distancing measures (Tsai et al. medRxiv 2020 Aug 7, doi:10.1101/2020.07.15.20154534)--and not just in the US, but also in other countries with a much stronger national level response, eg., China (Maier & Brockmann, Science 2020;368:742-6; Lai et al. Nature 2020 May 4, doi:10.1038/s41586-020-2293-x). The authors mention this limitation in the Discussion on page 14, but in my opinion this limitation is so glaring that I am not sure what we can learn from the cross-sectional analyses. (Perhaps the authors can be clearer about this in revised text.) The authors cite Cauchemez et al. (ref #40) in noting the significant country-to-country variability in government stringency and the 2009 influenza pandemic. But this is also well known in the context of COVID in the US (eg., Holtz et al. PNAS 2020;117:19837-43). Certainly if the measurement error (ie., due to lack of accounting for within-country heterogeneity) in the outcome variables is non-differential, the estimated associations would simply bias toward the null. But the same cannot be said for measurement error in the explanatory variables. Other investigators have dealt with this by collecting data on social distancing measures implemented at lower/subnational levels of jursdiction (eg., Hsiang et al. Nature 2020;584:262-7) or focused on a single country (as we did in our own work, Siedner et al. PLoS Med 2020;17:e1003244 and Tsai et al. medRxiv 2020 Aug 7, doi:10.1101/2020.07.15.20154534).

2. The authors also conducted longitudinal analyses, which would address concerns about masking within-country temporal heterogeneity (but not concerns about masking within-country geographic heterogeneity). Here they estimate fixed effects regression models, specifying country i on day t as the unit of analysis, lagging the explanatory variable by 6 weeks.

a) The fixed effects estimates are driven entirely by within-country changes in the Stringency Index and by within-country changes in mortality. So I would like to see some descriptive statistics (eg., intraclass correlations etc) quantifying the extent of within vs. between country variation.

b) It is unclear to me why the authors chose a lag of 6 weeks. In my opinion, there is considerable uncertainty about the appropriate interval one would expect between implementation of social distancing measures and any response in mortality. While the median incubation period is likely on the order of 3-5 days, the median time between symptom onset and death varies widely (eg., 8 days in Italy, two weeks in China). (Death probabilities, and time to death, would also be highly contingent on local comorbidity epidemiology as well as health system factors, but these would be unlikely to vary during the short time frame of the study and would therefore be differenced out in the analysis.) In our own work, we took a much more exploratory approach to modeling the association between social distancing and mortality (Siedner et al. PLoS Med 2020;17:e1003244). Perhaps the authors might adopt a similar approach and specify different lag periods to probe the extent to which their findings are sensitive to the choice of lag.

c) The authors motivate their lagged analysis, at the bottom of page 7, by suggesting that reverse causality could be at play. To rule this out, they probably should just probe the data rather than speculate. One could imagine a companion analysis in which the authors estimate the association between lagged deaths and the Stringency Index (to see whether it is indeed the case that "deaths might trigger a policy response"). As above, the extent to which the forward/reverse associations differ would depend on the extent of within-country variation over time.

One more specific point about the analyses:

d) It would be helpful if the authors could write out the estimating equations. It was not clear to me, either from the text or in Table 1, whether the authors estimated a single regression model containing both of the explanatory variables of interest (and also adjusting for the covariates listed in footnotes b & c), or whether the authors estimated two regression models containing 1 of the explanatory variables of interest (each of which adjusted for the covariates listed in footnotes b & c).

Minor comments:

3. The authors justify their use of death rates (rather than case rates) by asserting that they are likely to be reported more consistently. Is this true? (eg., Weinberger et al. JAMA Intern Med 2020 Jul 1, doi:10.1001/jamainternmed.2020.3391; Rivera et al. medRxiv 2020 Jun 27, doi:10.1101/2020.05.04.20090324).

4. Table S1-- Could the authors perhaps be a little more concrete about what goes into the different coding elements? For example, are restaurants included in C2 (workplaces) or C4 (restrictions on gatherings)? Are churches included in C3 (public events) or C4 (gatherings)?

5. To calculate the composite Stringency Index, the authors simply added up the individual components listed in Table S1. Obviously there are problems with granting equal weights to the different components, eg. canceling all public events (2 points) vs. closing some schools (also 2 points). This should be mentioned as a limitation.

b) To avoid imposing equal weights on the different elements, why not extract the first principal component (Table S2) and use that as the Stringency Index? Such an approach would allow the data to drive the weighting, similar to Filmer & Pritchett (Demography 2001;38:115-132).

6. While the Discussion section refers vaguely to "potential confounders", there is no mention of spontaneous behavioral change that may have occurred prior to implementation of any distancing measures. Such behavioral change was highly conditioned, both at the individual level (eg, Papageorge et al. COVID Econ 2020;40:1-45; Weill et al PNAS 2020 Aug 18;117(33):19658-19660) and at the area level (eg., Wright et al. SSRN 17 Jun 2020, doi:10.2139/ssrn.3573637), by preexisting sociodemographic factors, which is important from an equity perspective; and this evidence of pre-policy behavioral change also reinforces the notion that the epidemic itself, rather than lockdown-style distancing measures, was the cause of any observed economic fallout (eg., Sheridan et al. PNAS 2020 Aug 25;117 (34):20468-20473). To the extent that implementation of any distancing measures may have been implemented in response to either a worsening local epidemic or behavior change already occurring, the authors estimates are likely to be biased away from the null rather than toward the null--which would lead them to overestimate the effectiveness of social distancing measures in curbing the COVID epidemic.

7. There are several priority statements scattered throughout the manuscript (eg, page 3, "most comprehensive view"; page 15, "most comprehensive test"; etc). I am uncertain as to whether this is truly the case. For example, Islam et al. (BMJ 2020;370:m2743-- not cited in this manuscript, which is probably an oversight) used the authors' data on distancing measures to estimate the association between distancing measures and COVID-19 incidence-- this does not necessarily undercut the novelty of the database on distancing measures (although cf. Cheng et al. Nature Hum Behav 2020;4:756-768; Zheng et al. Scientific Data 2020;7:286; and Desvars-Larrive et al. Scientific Data 2020;7:285-- I am sure there are others), but it does tend to undercut the novelty of the analyses presented in this specific manuscript. In any case, the manuscript would probably be improved with (a) more comprehensive referencing of these alternative efforts; (b) better contextualizing of the present analyses in relation to other published literature and preprints (how does the authors' database compare with these other efforts? what are the gaps in the alternative databases and how do they compare with the authors' database? etc); and (c) deleting the priority claims, which are probably unnecessary anyway.

Alexander Tsai, MD

Massachusetts General Hospital

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Alexander Tsai

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2021 Jul 9;16(7):e0253116. doi: 10.1371/journal.pone.0253116.r002

Author response to Decision Letter 0

13 May 2021

We thank the reviewers for their careful reading of the manuscript, and have endeavoured to respond to their helpful suggestions. Please see the attached file for detailed responses.

Attachment

Submitted filename: PLOS R+R response to reviewers.docx

Click here for additional data file. (34.1KB, docx)

Decision Letter 1

Holly Seale

31 May 2021

Government responses and COVID-19 deaths: global evidence across multiple pandemic waves

PONE-D-20-20672R1

Dear Dr. Hale,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Holly Seale

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: I am identified as reviewer #2 on the initial submission. The authors have provided thoughtful replies to the comments that I raised. (Many of the comments I initially raised have been dealt with either with the deletion of the cross-sectional analysis or with the publication of their Nature Human Behavior data paper.)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: Yes: Alexander Tsai, MD

Acceptance letter

Holly Seale

30 Jun 2021

PONE-D-20-20672R1

Government responses and COVID-19 deaths: global evidence across multiple pandemic waves.

Dear Dr. Hale:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Holly Seale

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 File

    (DOCX)

    Click here for additional data file. (395.3KB, docx)
    Attachment

    Submitted filename: PLOS R+R response to reviewers.docx

    Click here for additional data file. (34.1KB, docx)

    Data Availability Statement

    All underlying data are freely available, and continuously updated, on the website of the Oxford COVID-19 Government Response Tracker: https://www.bsg.ox.ac.uk/research/research-projects/coronavirus-government-response-tracker.

    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES