Go slow to go fast: a plea for sustained scientific rigour in air pollution research during the COVID-19 pandemic

Researchers in the air pollution field are well aware that indoor and outdoor air pollution is associated with multiple disorders, including an increase in respiratory infection-related morbidity and mortality [1, 2]. Investigating associations linking air pollution and coronavirus disease 2019 (COVID-19) spread and prognosis of respiratory disease in COVID-19 patients is therefore timely, relevant and plausible. Over a 10-day period, three papers involving original research associating COVID-19 mortality and air pollution were published [3–5]. These publications attracted considerable attention from international news outlets and on social media [6, 7].

Short abstract

Present studies on the role of air pollution and COVID-19 spread, and the prognosis in patients, do not fulfil quality criteria and are not sufficiently informative https://bit.ly/3hGjd73

Researchers in the air pollution field are well aware that indoor and outdoor air pollution is associated with multiple disorders, including an increase in respiratory infection-related morbidity and mortality [1, 2]. Investigating associations linking air pollution and coronavirus disease 2019 (COVID-19) spread and prognosis of respiratory disease in COVID-19 patients is therefore timely, relevant and plausible. Over a 10-day period, three papers involving original research associating COVID-19 mortality and air pollution were published [3–5]. These publications attracted considerable attention from international news outlets and on social media [6, 7].

The first study was performed in the USA and associated COVID-19 mortality in 1783 counties with reported long-term (2000–2016) average air pollution levels (particles with a 50% cut-off aerodynamic diameter of 2.5 μm (PM_2.5) with a ∼1 km² resolution) [3]. Adjustments on the county level were made for 16 variables, including population size, ethnicity, hospital beds, number of individuals tested for COVID-19, weather, and socioeconomic and behavioral variables such as income, obesity and smoking. The study reported that an increase of 1 μg·m⁻³ in PM_2.5 was associated with a remarkably large 15% increase in the COVID-19 death rate. An updated analysis was adjusted for three more confounding factors: days since first COVID-19 case reported, population age distribution, and days since issuance of stay-at-home order for each state. This analysis saw the risk estimate halved to an 8% increase in death rate [8].

The second study used European data and, based on simple correlation analyses, associated long-term (January to February 2020) exposure to nitrogen oxides (NO_x) in the troposphere (resolution ∼7*3.5 km), assessed using satellite data, and absolute numbers of COVID-19-related deaths [4]. The study made use of data from 66 administrative regions in France, Germany, Italy and Spain. In this analysis, no standardisation or adjustment to basic variables such as population size and age distribution were made. The result, which was strongly driven by a few data points from Italy and Spain, was interpreted as an association between NO_x levels and fatal COVID-19 case counts.

The third and smallest study associated levels of NO_x and ozone between 2018 and 2019 obtained from monitoring stations across England with COVID-19 mortality in a mere seven different English regions [5]. Positive associations were seen between levels of nitrogen dioxide and nitrogen oxide and increased COVID-19 mortality and the reported number of cases, without adjustment for population size, age distribution or other confounding variables. A negative association was observed with ozone levels. The authors concluded that their study provided a useful framework to guide health policy in countries affected by the pandemic.

All three so-called “ecological studies” rely on aggregate data, which can suffer from the well-known problem of ecological fallacy, where misjudgements in interpretation occur when inferences about individuals are reasoned from inferences about the group to which the individual belongs [9]. The study design is prone to potential confounding as on an aggregate level, many factors may vary and even co-vary between geographical entities. Given the number of groups (i.e. counties) involved in the first study and the higher quality of exposure assessment, ecological biases are clearly less likely than in the second and third studies. The first study is also methodologically stronger because it is adjusted for a range of confounders at the county level, while in the two other studies, no adjustment for confounding factors was considered.

However, a major issue that is mostly ignored in these studies, resides in the complexity of a potential association between air pollution and COVID-19 morbidity and mortality. In particular, the effect of air pollution on the spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and on COVID-19 prognosis should be distinguished. The dynamics of an epidemic are dependent on transmissibility, contact rates and duration of infectiousness summarised in the basic reproduction number or R₀. This factor may differ between geographical regions and over time. Changes in R₀ can occur over time as a result of the natural history of the epidemic and infection control measures being implemented, as is shown by the levelling off of epidemic curves in many countries where physical distancing is practiced. In particular, the reproduction number can be high in the early stages of an epidemic when disease recognition and control measures are less adequate. Whether air pollution affects the spread of an infectious disease depends on whether the factors that determine the reproduction number (transmissibility, contact rates and duration of infectiousness) are influenced by air pollution. Theoretically, one can imagine that each of these variables can be influenced by air pollution, by changing host susceptibility, mobility and contact patterns, and by the severity and duration of the infection.

To explore whether air pollution influences R₀, high-resolution temporal and spatial data are required, preferably supported by virus sequencing data. This far exceeds the granularity of the data used in air pollution studies and most infectious disease outbreak studies. In particular, the two ecological studies that crudely correlate reported numbers of COVID-19 cases or mortality to regional air pollution levels ignored the time of introduction of COVID-19 in the different areas, did not take into account disease dynamics in any way, and ignored basic epidemiological principles by using inadequate measures of disease frequency [10]. To study the effect of air pollution on the spread of the pandemic in ongoing studies requires knowledge about the time of the virus introduction in the study area(s) and the dynamics of the pandemic in the study area(s). When comparing different study areas, the local outbreak size needs to be accounted for. The authors of the US study were well aware of their study's limitations, relating to outbreak dynamics, and acknowledged that using total population size in a county as a proxy for the local size of the outbreak in a given county, may result in unmeasured confounding through partial adjustment for outbreak size [3]. The updated study probably better adjusted for outbreak dynamics by including days since the first COVID-19 case was reported and days since the issuance of the stay-at-home order for each state, resulting in a considerably smaller regression coefficient for PM_2.5 and COVID-19 mortality. The fact that the epidemic had spread more evenly across the country, also to less urbanised areas, from almost 42% of counties with COVID-19 deaths to almost 78%, might also have contributed to a weaker association with PM_2.5.

A separate US study, associating COVID-19 fatality rates and mortality with PM_2.5 in a three-pollutant model (PM_2.5 adjusted for ozone and NO_x), showed a null association for PM_2.5 with COVID-19 fatality rates and a marginally significant association with mortality [11]. The use of the case fatality rate instead of COVID-19 mortality, and adjustments for the epidemic timing of the introduction and spread of COVID-19 by adjusting for the time the first and 100th COVID-19 cases occurred, probably explains the marked difference in results for PM_2.5 with the other US study. The authors also indicate that control for spatial trends in exposure and residual autocorrelation in exposure between counties may also have contributed to the observed difference in PM_2.5 study results. The same study found positive and statistically significant associations for NO_x. However, it was concluded that NO_x air pollution level may act as a surrogate for urbanicity, despite adjustment for population density.

These studies together highlight that uncovering the effect of air pollution on the spread of SARS-CoV-2 represents an enormous challenge that will require multidisciplinary collaboration and high-quality data. Other approaches, including air pollution time series studies, need to be explored when the appropriate COVID-19 data to perform such analyses become available. Such approaches should also consider the effect of changes in air pollution levels. However, changes in air pollution also occurred after introduction of measures to mitigate the impact of the pandemic. Here, confounding still plays a major role as many additional factors changed at the same time (e.g. physical activity, stress, social economics, medical care visits), introducing other challenges in the data analyses.

In the meantime, multiple other studies have been published, many of which can be found in a useful study repository [12]. However, many of these studies did not take into account COVID-19 outbreak dynamics. This leads to similar biases as described here.

The effect of air pollution on disease prognosis can be studied using more conventional approaches after COVID-19 infection. For instance, by following up confirmed patients over time in different regions with different levels of air pollution. However, even this type of study might be complicated by confounding; for instance, disease severity and testing policies can differ according to treatment centre, and the quality of care delivered by healthcare systems may vary across regions, depending on the size of the local outbreak. These scenarios are more classic examples of confounding for which a range of solutions may be considered. The use of test-negative designs has been proposed, in which risk factors are compared between subjects that test positive or negative to SARS-CoV-2 [13]. Such designs can eliminate some of the aforementioned concerns (access to testing, etc.) but have other limitations, mainly relating to shared risk factors for being tested regardless of the outcome (e.g. presence of respiratory disease).

We certainly do not want to argue that air pollution does not play a role in the COVID-19 pandemic. There is previous evidence from the SARS coronavirus epidemic (though this study does suffer from the same ecological problem as the current studies) [14], similarities with a possible smoking contribution to the pandemic [15], mechanistic evidence involving the angiotensin-converting enzyme 2 receptor in COVID-19 pathology that could be influenced by air pollution [16], and the fact that air pollution has previously been causally linked to the risk of pneumonia and other comorbidities that contribute to COVID-19 deaths [17]. However, determining whether there is a causal effect and obtaining an accurate estimate of the effect requires rigorous and time-consuming research.

A final major point of criticism that we wish to make is that one of the aforementioned papers was accepted for publication after a 1-day review period [4], while the other papers made headlines before the peer-review process based on preprint manuscripts [3, 5, 16]. It is important to ask why this is the way COVID-19 air pollution research is being reviewed and communicated. Research in this field may be of relevance for the spread of infectious disease and disease prognosis, but outcomes from air pollution research are unlikely to lead to changes in the implementation of pandemic control measures and in the clinical care of patients. Instead, studies on the effect of air pollution on COVID-19 epidemiology will have an impact in the longer term. Thus, there seems little reason to abandon the rigorous peer review process and expedite manuscript publication for COVID-19 air pollution studies. Study quality is of great importance; all stakeholders, including the scientific community, policy makers and the general public, are served best by well-conducted and thoroughly appraised studies on air pollution and COVID-19. We need to avoid creating myths that are diverting political and scientific attention from other urgent and actionable questions. The phrase “go slow to go fast” is used in industry to combat the short-sighted fixation on speed while sacrificing product quality and hence utility. Where research does not have an immediate impact on clinical and societal interventions related to the current pandemic, we may want to adopt this philosophy. Air pollution researchers and scientific journal editors have an important role in maintaining calm scientific rigour during this tumultuous time.

Shareable PDF

This one-page PDF can be shared freely online.

Shareable PDF ERJ-01361-2020.Shareable ^{(290.8KB, pdf)}