Research into policy: lessons from the COVID-19 pandemic

Abstract

There has been an unprecedented global effort by researchers from many disciplines to obtain and synthesize knowledge to inform policy responses to SARS-CoV-2. While many major advances have been made in generating and applying knowledge on a pandemic caused by a novel pathogen, some things could have been done better, as revealed by the devastating loss of life and economic impact on livelihoods and communities. We reflect on the context in which the pandemic emerged, characterized by underinvestment in public health and growing distrust in institutions, followed by an overview of three broad areas: generation of new knowledge, synthesis of existing knowledge, both what was known prior to the pandemic and what emerged during it, and the challenges of translating knowledge into policy. We also consider areas that were largely overlooked in the research effort. Across all areas, we aim to draw out relevant lessons for future research and public health practice.

Generation, synthesis and application of knowledge

There has been an unprecedented global effort by researchers from many disciplines to obtain and synthesize knowledge to inform policy responses to SARS-CoV-2. What lessons can we learn from this experience?

Some context is necessary. For knowledge to be accepted and used, it must be trusted and trustworthy. Yet the pandemic arose at a time when trust in institutions in many countries was very low. Two seismic political events, the United Kingdom’s decision to leave the European Union and the election of President Donald Trump, involved disinformation campaigns when ‘fake news’ entered everyday conversation. Some populist leaders rejected concepts of scientific inquiry. This had happened before with HIV but now it was facilitated by social media where conspiracy theories spread rapidly. There was also a legacy of failure to invest in public health systems and pandemic preparedness in many countries,^1,2 and especially the capacity to undertake and apply public health research. This affected the entire spectrum of research and experience needed to respond to a pandemic, from basic to behavioural science, and arguable influenced the course of the pandemic and responses too it in many parts of the world.

We look across three broad areas: generation of new knowledge, synthesis of existing knowledge, both what was known prior to the pandemic and what emerged during it, and the challenges of translating knowledge into policy, drawing out relevant lessons.

New knowledge

The genetic code of the virus had been sequenced within a few days of the first cases being identified in Wuhan on 30 December, revealing the emergence of a new SARS-like coronavirus. After six days when the Chinese authorities restricted comment by the scientists involved, the sequence was released and within two days a diagnostic test had been developed.

The next question was how the virus was transmitted. Initial suspicion focused on zoonotic infection linked to wet markets but it was unclear whether there would be person-to-person transmission, indicating potential for a pandemic. This was established on 24 January when a Chinese team published an investigation of a cluster of six patients, five of whom had been in Wuhan.³ Thus, within a month of the first cases being identified, Chinese scientists had characterized the genetic structure of the virus and established its potential for transmission. The lesson is that access to high quality laboratory and epidemiological capacity is a global public good that benefits everyone. Another is that there is no justification for anything that restricts the free flow of scientific knowledge.

Chinese authorities moved rapidly to supress viral spread, with strict lockdowns, although these were complicated by travel over the Chinese New Year. By then, however, the virus had spread abroad. In Italy, the epicentre of the pandemic in Europe, hospitals were being overwhelmed. Now the attention of researchers turned to the pathogenesis of SARs-CoV-2 infection and its treatment. Another important lesson of this pandemic has been the importance of framing problems. From the outset, countries adopted one of two approaches which, at risk of simplification, can be labelled as influenza or SARS models. Many decisions stemmed from that choice, most importantly whether the goal should be to suppress and, if possible, eliminate transmission,⁴ as with SARS or whether it should be treated like seasonal influenza, where the goal has been to mitigate the worst effects because spread is considered inevitable. This also influenced treatment strategies. COVID-19 was seen as primarily a viral pneumonia, like influenza, with effects mostly confined to the respiratory system so treatment should be based on respiratory support, including ventilation, to allow patients to recover. This led to a global scramble for ventilators. Yet it soon became clear that this was a complex multi-system disease involving several pathological mechanisms;⁵ ventilating patients too early was increasing mortality while some supportive measures, such as placing patients prone, were beneficial. However, another lesson from this experience is that, with a few local exceptions, there seemed few mechanisms to disseminate accumulating clinical knowledge.

The early months of the pandemic were characterized by uncertainty about effectiveness of potential therapies. The World Health Organisation initiated the SOLIDARITY Trial, with centres in over 30 countries,⁶ but has struggled with national procedures. It has, however, shown the lack of evidence supporting use of Remdesivir, Hydroxychloroquine and Lopinavir/Ritonavir. The United Kingdom’s RECOVERY trial has, as of March 2021, recruited over 38 000 participants and established the value of Dexamethasone and Tocilizumab. The lesson is that this knowledge could have emerged far earlier if more patients worldwide had the opportunity to participate in a clinical trial.

In parallel, efforts were underway to characterize the human immune response to SARs-CoV-2 infection and design and evaluate vaccine candidates. The development of safe and effective vaccines in under a year has been a remarkable achievement, using a variety of technologies, with traditional approaches based on inactivated viruses to completely new ones using messenger RNA (mRNA). Procedures were accelerated and delays reduced or eliminated. Some mistakes were made, such as a problem with the dosing of the AstraZeneca vaccine, but none that were serious. The lesson from this experience is that new medical products can be brought to market safely and at speed, if sufficient resources are invested and political commitment exists. This offers lessons for the many neglected diseases still seeking effective vaccines or treatments.⁷ The experiences from the pandemic thus far also serve to demonstrate that vaccine development is a crucial but one aspect of a larger system, the other elements including production, supply and cold-chains, communication strategies (including anticipating and addressing vaccine hesitancy), intellectual property rights, workplace regulations and vaccine status documentation and the ethical aspects of this, and global health equity. Research should be directed towards all elements, including conceptualizing challenges and enablers within and between each element.

Another type of new knowledge relates to non-pharmacological interventions designed to reduce transmission, including restrictions on mixing and use of face coverings. These have been especially challenging to evaluate. First, there was little previous evidence to draw on and gaps remained in knowledge of where and how the virus was transmitted.⁸ Second, many such measures are not easily evaluated by randomized trials, given the complexity of the context, the number of variables to consider and control for, barriers to measuring outcomes in a timely and accurate way, and ethical issues. Third, there are many challenges in attributing an intervention to the ultimate outcome of reduced transmission of the virus. This is exemplified by the prolonged debate on face coverings in some countries. Once it was realized that the objective of wearing masks was to reduce transmission to others, the problems became apparent. Thus, while it was possible to allocate individuals to groups asked to wear, or not to wear them, it was not obvious in whom the outcome would be measured. It would be impossible to trace everyone that the trial participants would encounter and potentially transmit the infection to. Moreover, those individuals would likely be exposed to members of both the individual and control groups. Unsurprisingly, the one trial that did seek to answer this question was inconclusive.⁹ Instead, the evidence supporting what is now a widely held consensus that favours face coverings has come from a combination of study designs, including a natural experiment in Germany that used synthetic controls to exploit differences in timing of mask mandates¹⁰ and research from aerosol experts.¹¹ This research has benefited greatly from innovative sources of data, such as the ability to track mobility using mobile phone data¹² and sentiment analysis from social media, with the pandemic providing a major stimulus to advances in this area.

Although many other areas of knowledge have emerged from the pandemic, the final one considered here brings together organizational, management and policy research. Even when effective measures were known, many countries failed to adopt them or did so ineffectively. Many elements of an effective response in a pandemic involve well-established public health methods, such as contact tracing. Yet these were often implemented extremely poorly. Often there was a failure to draw on insights from systems science, recognizing the interconnectedness of the many elements of a response. Thus, responses were often fragmented and failed to make use of existing resources and expertise. Many plans were developed with scant regard for those who had to implement them,¹³ even though there is now considerable evidence on the benefits of co-production of knowledge and the methods that facilitate it.¹⁴ Furthermore, while the value of behavioural science in a pandemic is evident, many countries have underinvested in it, arguably undermining design and implementation of effective responses that take account of people’s lived reality and potential responses to restrictions and information. Sometimes this reflects a very narrow view that separates ‘science’ from policy and operational measures. While tempting, especially where the political environment is partisan, it is a recipe for failure and overlooks the political nature of knowledge.

Synthesis of existing knowledge

The emergence of a new virus with unknown or uncertain characteristics posed obvious problems for clinicians and policymakers initially. Within weeks, a new problem arose, information overload. Studies on different aspects of the pandemic were rising almost as rapidly as the number of cases. An analysis of the first 10 000 papers on COVID-19 estimated that, in June 2020, they comprised 8.3% of all scientific outputs worldwide.¹⁵

Conventional systematic reviews are demanding of personnel and time but, importantly, in a fast-moving situation, they may be obsolete when they are completed. The need for timely syntheses of evidence has been met, to some extent, by rapid evidence reviews. These are typically undertaken in less than five weeks and, while the methods should be transparent and reproducible, they do not include all the stages completed in a systematic review or meta-analysis. They are most appropriate for answering specific policy relevant questions. Examples from the pandemic include ways of ensuring adherence to quarantine,¹⁶ evidence on physical distancing,¹⁷ and the role of face coverings.¹⁸ They can assemble the best available evidence in a timely manner but do require considerable expertise in critical appraisal of evidence, another area where there has often been underinvestment.

The journey from discovery to publication of research is often tortuous and prolonged, with many delays arising from the review and production processes. For some years, this has been circumvented in certain disciplines, such as physics and economics, by use of preprints, papers posted online to allow others to comment before the definitive version is published. This has advantages and disadvantages.¹⁹ For example, preprints reporting an association between smoking and reduced risk of severe COVID-19 infection attracted considerable media attention, despite clear limitations and contrary to decades of evidence on the adverse health impacts of smoking. Another paper, posted in July 2020, suggested ‘that sufficient herd-immunity may already be in place to substantially mitigate a potential second wave’, a conclusion that is self-evidently false. That preprint, as of March 2021, has yet to be published in a peer-reviewed journal.²⁰ Yet it attracted widespread media attention²¹ and was widely cited as evidence against non-pharmacological interventions. Preprints were also used to forward particular agendas.²² As reported by The New York Times, the release of a preprint manuscript estimating a low fatality rate for coronavirus was construed as evidence against social distancing and lockdown measures, and used to question the evidence of the severity of the pandemic.²² It was criticized for its methodological limitations and what was seen as premature promotion to the media.

Notably, the COVID-19 pandemic has unfolded in the era of the rise and dominance of social media, such as Twitter, Facebook, WhatsApp and other similar platforms and applications. The role of social media in the pandemic has two starkly contrasting forms—facilitating the spread of conspiracy theories and mistrust, while simultaneously serving as a channel through which to disseminate updating knowledge and public health messaging. For example, several reports have been published during the pandemic documenting the ways in which social media was used by professional groups and experts to distil and share research updates and public facing advise and explanations, often supported by use of visual graphics.^23–25 However, these accounts also note the risks and challenges associated with the use of these channels including how to assess and maintain information accuracy as well as how to navigate the sheer volume of information produced. Furthermore, social media enables the dissemination of misinformation and conspiracy theories and targeting of certain groups. One report in the US suggested that use of conservative and social media was associated with reporting belief in COVID-19-related conspiracy theories.²⁶ The need to advance our understanding of how to maximize the beneficial role of social media and other online technologies during times of crisis while addressing the harms and detrimental impacts calls for further investigation.

Synthesizing knowledge across academic fields has also proved challenging. Although it should have been obvious that the complex nature of a pandemic, where the virus exploits changes in human behaviour, would only be understandable with a concerted multidisciplinary effort, scientific advisory structures in many countries operated in silos. In particular, early mathematical models were relatively uninformed by understanding of some important influences on transmission, such as the work of care home staff, many in low paid employment and working in multiple facilities. Similarly, a failure to engage with experts in aerosol science delayed the recognition of the characteristics of indoor transmission.

Evidence into policy

It soon became clear that some countries, such as New Zealand, Australia, Vietnam, Finland and Uruguay, were faring far better than others.²⁷ It seems self-evident that there would be scope to learn from their success and it was important to do so, given how existing measures of pandemic preparedness had failed to predict national performance.²⁸ There were several sources of information on what these countries were doing, such as the Oxford COVID-19 Government Response Tracker,²⁹ which scores countries on their responses to the pandemic, and the European Observatory COVID response monitor,³⁰ as well as publications describing country experiences, both positive^31,32 and negative,³³ or reporting comparative analyses.³⁴ Yet some governments struggling to contain the pandemic did not adapt their responses despite evidence of effectiveness that emerged from those countries doing better. In contrast, those promoting policies since proven ineffective did at times look elsewhere, and especially to Sweden, whose initial experience, misleadingly presented as involving an absence of restrictions, seemed to have avoided the very high incidence seen in southern European countries. In doing so, they ignored evidence that mobility in Sweden had fallen sharply³⁵ and its many other national characteristics suggesting that comparison with neighbouring Norway or Finland would be more appropriate.³⁶ Although there is an extensive literature on lesson learning in public policy, one review of comparative studies undertaken during the pandemic found that they tended to exclude some countries that could have offered lessons and often failed to consider transferability.³⁷

Many of those who would have to make policy decisions were confronted with unfamiliar concepts. Although some technical terms, such as R₀, the Reproduction number, would soon enter common parlance, some ideas were difficult to grasp. Among them, perhaps the most problematic was exponential growth. Left unchecked, an epidemic that is growing exponentially may change little for some time but will soon accelerate upwards. Delaying actions to reduce it, even a few days, may make a dramatic difference to the eventual number of individuals infected. A survey in the United States which ascertained whether individuals understood this concept found that those who did not were significantly less likely to support lockdowns.³⁸

Given the economic impact of restrictions, it was perhaps inevitable that those in positions of power who were impacted most would support efforts to present evidence in ways that served their interests, just as is seen in other circumstances where policies threaten corporate interest, such as in relation to tobacco or ultra-processed food. The pandemic was no different, with neoliberal groups supporting the production of what was termed the Great Barrington Declaration,³⁹ a statement that argued against restrictions, drafted by advocates of the concept of herd immunity through natural infection. This concept also attracted widespread support, especially on social media, from individuals arguing, for example, that most reported cases were due to false positive tests, that numbers of deaths were being exaggerated, and that COVID-19 was no more dangerous than seasonal influenza.

The pandemic has also directed attention towards the various mechanisms that govern the role of advisory groups or panels, including who is involved, how advice is transferred and communicated, and what level of public or external scrutiny is present. Future research is needed to investigate structures systematically to facilitate learning from different contexts and regions and to inform understanding of which systems maximize public benefit. By way of example for the purpose of this overview, concerns about perceived weaknesses, such as lack of transparency in official scientific advisory processes and the scale of disinformation being promoted led, in several countries, to civil society groups taking up the challenge. Responding to these concerns, in the United Kingdom an Independent SAGE (drawing on the name of the official Scientific Advisory Group on Emergencies) was created.⁴⁰ Comprising a small team of scientists from different disciplines, it is fully transparent, has produced numerous reports, received input from frontline workers, and holds a weekly media briefing viewed by up to 80 000 people. Its work is used extensively by local government, trade unions and others. Similar bodies have been created in Ireland and Germany. There were other more targeted initiatives, such as the group that developed the John Snow Memorandum,⁴¹ which countered the arguments in the Great Barrington Declaration. Further analysis is needed to understand the impacts of these initiatives and how to optimize the transfer of expertise to policymakers, the media and the public in ways that advance public health goals and how to foster productive debate when expert advice differs.

Issues that were overlooked

As the pandemic progressed, it has become increasingly clear that those already disadvantaged have suffered most. People living in overcrowded accommodation and working in the informal economy were most likely to become infected. Many had underlying health conditions, placing them at risk of severe disease or death. They were most vulnerable to many of the measures taken to limit the spread of infection, such as closures of workplaces. Yet, with a few exceptions, the way in which the pandemic has widened existing inequalities has received little attention and especially so for racial inequalities.⁴² Very few countries even collect data on ethnicity, but those that do have found that it is a very important determinant of outcomes during the pandemic. It is also emerging as an important factor in variations in vaccine uptake. For historical reasons, there is considerable opposition to collecting such data, in some countries. However, this position is no longer tenable.

Although it was clear from the outset that the restrictions necessary to limit transmission would have numerous other consequences for health,⁴³ for example through inability to access essential health care, there was relatively little research on these impacts, with a few exceptions, such as that describing the reduction of primary care attendances for different causes during the first wave.⁴⁴

Little appears to have been done initially to mitigate the impacts of the pandemic and the measures taken to address it on inequalities. This is despite a large body of evidence documenting the ways in which crises, including epidemics, place the greatest burden on the poor and the vulnerable⁴⁵ and the publication, early in the pandemic, of a framework for addressing potential equity impacts.⁴⁶ There were many lessons that could have been learnt from the response to the global financial crisis, for example the importance of active labour market policies and the role of pre-existing precarity, but were not, at least by the public health community and policymakers. Interestingly, the experience of the financial crisis has contributed extensively to proposals for the post-pandemic world from those in the financial sector who, in many respects, are taking the lead on these issues, which will be crucial in managing the aftermath of the pandemic. In Europe, this learning can be seen in the work of the Pan European Commission on Health and Sustainable Development, chaired by Mario Monti, a former Italian Prime Minister and European Commissioner, and calls from current and former central bankers for a green recovery agenda to guide the post-pandemic era.⁴⁷

Finally, in some countries, despite ample evidence to inform impact assessment, there appears to have been inadequate measures taken to prevent or minimize the impact of COVID-19 response strategies on children and young people.⁴⁸ School closures for example have impacted on educational attainment, and such effects have fallen disproportionately on the most deprived, with potentially long-lasting effects in the absence of substantial remedial interventions.⁴⁹

Implications for Europe

Europe has been hit extremely hard by the pandemic. The European Union has agreed a €1.8 trillion recovery plan, including €750 million for the NextGenerationEU fund, a significant share of which will be spent on research and development. It is essential, as it spends these funds, that the lessons from the pandemic are learned.

The achievements of biomedical researchers were remarkable, bringing vaccines to market in record times. Previously, the ability to sequence the genome of the new virus and develop a diagnostic test within days would have been unimaginable. It permitted those developing vaccines to get started. They could do so because of earlier investments, including by the European Union (EU), in innovative vaccine technology, for example in TRANSVAC2. For example, the EU has been a major funder of BioNTech. Yet, in normal circumstances, it would take much time to move forward to exploit these advantages, with researchers struggling to convince funders of the potential of their ideas and to overcome regulatory barriers. In this case, the scale of the crisis was such that funding was made available rapidly, in very large amounts, allowing vaccines to be developed and deployed in record time. However, in retrospect, given that the public sector was bearing most if not all, of the financial risk involved, we should ask why governments and the EU did not seek greater control over the resulting intellectual property, making it easier to influence pricing policies that would maximize affordability of the vaccines worldwide.

However, an effective vaccine programme is much more than this. There have been unacceptable delays in achieving coverage in many countries. There are many reasons but, among them, there seems to have been major gaps in undertaking and applying organizational and behavioural research. Crucially, in many countries, inequalities in uptake, especially in relation to ethnicity, remain unmeasured and thus invisible. Similarly, the application of non-pharmacological interventions in many countries was, at least initially, often uninformed by theory, research and rarely evaluated in practice. The COVID-19 pandemic has reminded us of the importance of having a strong research and expertise infrastructure in place that can respond rapidly in a crisis but this must go beyond the biomedical. As demonstrated by other major public health threats both infectious and non-infectious in nature, such as Ebola and tobacco, respectively, adopting a narrow biomedical lens that focuses on specific forms of evidence and technological solutions while overlooking the political and social aspects of the issue and the risks imposed by vested interests, undermines effective responses, particularly from an equity perspective. The hegemony of biomedical research and perspectives and the tendency to prioritize technological interventions deserves greater attention, including more research into how issues become framed and conceptualized in these narrow terms and the consequent impacts.

There were also many successes in developing new treatments or, rather, in identifying existing treatments that were effective. However, here again, there are lessons that can be learned. The United Kingdom’s RECOVERY trial showed what was possible. Other European countries must now ask why they were unable to do something similar, ideally as part of a pan-European initiative. No one should be denied the opportunity to participate in clinical trials. Nor should they be given treatments already shown to be ineffective, a reminder of the need for better pan-European mechanisms to disseminate clinical knowledge.

The widespread acceptance of weaknesses in preparedness has led to a commitment to create a European Health Emergency Preparedness and Response Authority (HERA). This is a tremendous opportunity but one that faces many challenges. It will be essential that it learns from experiences elsewhere, especially the Biomedical Advanced Research and Development Authority (BARDA) in the United States. It must resist the pressure to become yet another mechanism to provide subsidies to the pharmaceutical industry.

Finally, Europe must now look beyond the pandemic to ask how we can build back better.⁵⁰ This will require policies that foster inclusiveness, investment and innovation, principles established at the 2018 Tallinn Ministerial Conference⁵¹ and being taken forward in the Pan-European Commission on Health and Sustainable Development.⁵² In these, and in other initiatives, the European public health community must ensure that it is always ‘in the room where it happens’.⁵³

Funding

M.v.S. is funded by the National Institute for Health Research (NIHR) Doctoral Fellowship (NIHR3000156) and her research is also partially supported by the NIHR Applied Research Collaboration North Thames.

Disclaimer

The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.

Conflict of interest

M.M. is a member of Independent SAGE and a Commissioner and chair of the Scientific Advisory Board of the Pan European Commission on Health and Sustainable Development, both cited in the text.

Key points

• We must build and maintain trust in, and trustworthiness of, public institutions and ensure integrity of the scientific process and information dissemination.

• Access to high quality laboratory and epidemiological capacity is a global public good that benefits everyone.

• There is no justification for anything that restricts the free flow of scientific knowledge.

• Researchers and practitioners must understand how problems are framed and the implications that follow from this.

• Structures to facilitate rapid sharing of knowledge and experience are essential should be strengthened.

• No patient with a condition where treatment is inadequately understood should be denied the opportunity to participate in a clinical trial.

• The pandemic has shown what is possible with funds and political will; now we must apply the same effort to neglected diseases.

• ‘Selective learning’—overlooking important knowledge and experiences from some countries while adopting ideas from others—should be avoided.

• Researchers must seek answers while looking through an equity lens, asking how their findings can benefit everyone but especially the vulnerable and marginalized.

Additional Content

A video to accompany this paper is available at https://youtube.com/playlist?list=PLv5eq4ZCoNWubJurAJ-7Ht33cjNshLw7R.