Policy makers must act on incomplete evidence in responding to COVID‐19

Reducing the transmission of Coronavirus disease 2019 (COVID‐19) is a global priority. Toward this end, public health officials and politicians across the world have been seeking scientific expertise to guide policy. In response, investigators have rushed to share new results on preprint servers, and journals have expedited editorial and peer review processes to publish them. The urgency to define the relevant knowledge base in preventing, diagnosing, and managing COVID‐19 infection and its sequelae has required intense collaboration in evidence generation and synthesis, in order to provide public health officials with authoritative guidance.

Cochrane has responded to the crisis by gathering its community, working closely with the World Health Organization (WHO) and other stakeholders, in developing and publishing several systematic reviews on the effectiveness of behavioural public health measures for reducing COVID‐19 infection. These measures include masks, handwashing, physical distancing, quarantine, contact tracing, screening, and travel restrictions. Because COVID‐19 is still so new, however, these reviews have largely summarized effects on transmission of other viruses in non‐pandemic conditions.

The conclusions of these reviews are similar. None has found robust, high‐quality evidence for any behavioural measure or policy. Each has identified important limitations to their respective bodies of evidence. An updated review of physical interventions by Jefferson and colleagues assesses three commonly recommended interventions: masks, hand hygiene, and physical distancing. They found evidence that masks had limited or no benefit in terms of preventing influenza‐like illnesses or laboratory‐confirmed influenza. However, except for a handful of studies, most of the evidence is from studies examining effects in wearers. An important effect may still lie in how masks reduce transmission of virus to others, which is more difficult to ascertain. Resulting uncertainty in the evidence for public health measures has fed controversies regarding the legitimacy of public health policies involving these measures, with face masks being a special target for criticism.

For each measure, though, lack of evidence of effectiveness is not evidence that the interventions are ineffective. Rather, the details of these reviews show why there may never be strong evidence regarding the effectiveness of individual behavioural measures when deployed, often in combination, in a general population living in the complex, diverse circumstances of individuals' everyday lives. Waiting for strong evidence is a recipe for paralysis. Public health officials must, instead, take measured gambles, based on circumstantial evidence from the reviewed studies and other sources. When protecting the public from harm is the objective, public health officials must act in a precautionary manner to take action even when evidence is uncertain (or not of the highest quality), particularly when the harms and costs of such action are likely limited.

The quality of the signal from any randomized study depends on how consistently its intervention is implemented. Low compliance with the interventions studied in the trials included in the review illustrates the challenge of assessing implementation for behavioural measures intended for everyday use. For an intervention to work, people must get the message, find it persuasive, understand its instructions, and be able to perform the behaviour in their everyday lives. A measure that could make a difference in theory might not do so in practice, if its implementation failed to meet these conditions. Few studies are designed to observe all these factors. Also, the smaller the effect of an intervention is at the individual level, the more likely it will be that biases or other limitations of trials will attenuate or account for it. Based on what we know about the design of two large community‐based COVID 19 trials, one published in the last few days from Denmark and the other still ongoing in Guinea‐Bissau, their results may not move us any further forward in determining the effects of face masks on virus transmission. They may still provide vital information regarding the effectiveness of distribution procedures and self‐reported usage.

Whether a trial detects an effect also depends on what else is going on in the test environment. For example, face masks will be less effective if the surrounding population does not support the behaviour being encouraged in the study group, if masks are not worn properly or social distancing is not practiced (or possible). Conversely, trials studying individual measures will not capture the effects of combined measures. Some of those combined effects may be additive, with large enough cumulative protection to produce important effects, if the measures were studied together. Some of the combined effects may be synergistic, if the behaviours reinforce one another, creating a precautionary culture, wherein they become social norms and personal habits. Contrary, too strong a focus on a single intervention from authorities may reduce adherence to other important measures.

However, while there is reason to believe in the combined effects of multiple behavioural measures, there is not, and may never be, high‐quality evidence from randomized trials on those effects. As a result, public health officials must rely on necessarily incomplete evidence. In making those fateful decisions, they can draw on the trials summarized in Cochrane's systematic reviews. They can also draw on observational studies relating behavioural practices to transmission rates, recognizing the limits to such associative data. And they can draw on basic research, such as highly controlled laboratory studies showing how well face masks reduce the transmission of droplets from coughing and sneezing.

Sound professional judgment is needed to interpret such complex, circumstantial evidence regarding effectiveness. Those inferences require translation into policies that disrupt individuals' lives as little as possible and respect the differences in their circumstances. Finally, those policies must be shared with the public in ways that convey their rationale, benefits, harms, uncertainties and costs, and commitment to the common good.

Feedback on this editorial and proposals for future editorials are welcome.

End Notes

Declarations of interest

KSW is the Editor in Chief of Cochrane and TL is Deputy Editor in Chief; both are full‐time employees of Cochrane. KJ is a member of Cochrane's Governing Board. SW, LB, and MB are members of Cochrane's Editorial Board. The University of Colorado received remuneration for LB's role as Senior Editor for Research Integrity and for the Cochrane Public Health and Health Systems Network. MB received remuneration for his role as Senior Editor for Cochrane Acute and Emergency Care and the Circulation and Breathing Networks. The authors declare no additional interests.

Provenance and peer review

This editorial was commissioned and was not externally peer‐reviewed.