Drawing lessons from the COVID‐19 pandemic: Seven obstacles to learning from public inquiries in the wake of the crisis

Abstract

On March 11, 2020, the World Health Organization declared the emerging COVID‐19 threat a pandemic following the global spread of the virus. A year later, a number of governments are being handed the concluding reports of national public inquiries tasked with investigating responses, mishaps, and identifying lessons for the future. The present article aims to identify a set of learning obstacles that may hinder effective lessons drawing from the COVID‐19 pandemic responses. The seven obstacles discussed in this article are: (1) retaining lessons and implementing them effectively, (2) effectively drawing lessons from other countries, (3) the potential for reforms to introduce unanticipated vulnerabilities elsewhere in the system, (4) political pressure, (5) drawing the conclusions from observations, (6) experts versus decision makers, and (7) reforms may not be related to the actual crisis. Exploring these obstacles will be central to future discussions concerning which kinds of responses will set precedent for future pandemics and global health crises.

INTRODUCTION

On March 11, 2020, the World Health Organization (WHO) declared that the global spread of COVID‐19 constituted a global pandemic, following the uncontrolled global reach of the outbreak. The novel SARS‐CoV‐2 coronavirus, having first been identified as a previously unknown form of pneumonia originating in the Chinese city of Wuhan, quickly spread to every country on the planet. Catching many countries by relative surprise, despite its long lead time, the COVID‐19 threat triggered a wide range of emergency measures and responses that varied greatly across and within countries. Operating under considerable uncertainty due to the limited experience of the global health system in responding to creeping crises of this extent, governments were thus forced to monitor the effectiveness of measures and their strategies as they went along, adjusting underway as signals and contingencies emerged (Boin et al., 2020). In this process, governments have also sought to collect vast amounts of data aimed at facilitating intracrisis learning as well as postcrisis lesson‐drawing. In the aftermath, the question of which responses emerge as “best practice” is likely to be contested intensely and has important implications for how the precedent will be set for future pandemic preparedness. The launching of public inquiries in different countries, whose conclusions are expected by decision makers and the electorate to lead to reforms and structural improvements in resilience, is one process that may particularly contribute to setting precedents.

Postcrisis learning constitutes one of the major subjects investigated by crisis scholars (e.g., Birkland, 2009; Deverell, 2009; Stern, 1997). A salient issue in these studies has been the question of how to draw out appropriate lessons—or lessons that may actually enhance resilience over the long term as opposed to lessons that are not directly related to response problems—and then how to make these lessons stick once they have been identified. In the case of pandemics, these lessons would often have to stick for quite some time, which may make lesson retention in this context particularly challenging. The present article aims to identify a set of obstacles that may hinder effective learning from COVID‐19 pandemic responses. By drawing on key insights from the research literature, this viewpoint essay sets out to share some reflections on the prospects for effective and long‐term learning from COVID‐19 inquiries and also reflects on the notion of “best practices” in the context of pandemic response.

This article is structured as follows. Following the introduction, the second section goes on to reflect on seven obstacles to lesson‐drawing that may inhibit long‐term learning from the COVID‐19 pandemic, followed by a concise conclusion.

SEVEN OBSTACLES TO LESSON‐DRAWING THAT MAY INHIBIT LONG‐TERM LEARNING FROM THE COVID‐19 PANDEMIC

Although crises may create favorable conditions for learning, the research literature on the subject demonstrates mixed evidence on the ability of societies to effectively learn from crises and public inquiries (ibid.). Scholars have suggested that “learning after a crisis is rare” (Albright & Crow, 2019, p. 19). Other pressing societal concerns often emerge in the wake of the crisis. This gives us grounds for being only mildly optimistic about the prospects for drawing fruitful lessons during and in the wake of the COVID‐19 pandemic. In the following, we reflect on a number of obstacles for enhancing resilience for future pandemics in the wake of the COVID‐19 pandemic.

Obstacle 1: Retaining lessons and implementing them effectively

Crises unfold over time, depending not only on the temporary duration of the disruptive effects but also due to response efforts. In the context of the COVID‐19 pandemic, for example, accumulated experience and knowledge will continuously change during its life cycle. The timing of the evaluation process is thus of great consequence because conclusions and best practices are likely to be adjusted continuously during the crisis as new evidence comes in and contextual realities, including political sentiments, change.

Crises are characterized by political pressure, uncertainty, ambiguity, and stress ‘t Hart and Boin (2001). At the same time, there is a demand for immediate answers, decisions, and solutions. When it comes to intracrisis learning, there is, therefore, a high risk for ill‐considered conclusions (Moynihan, 2008). The “war for meaning” that is played out as actors try to make sense of the evolving situation will have a large effect on what is seen as relevant, and perceptions of which lessons are relevant may change over time. Also, intercrisis learning is described as fraught and limited. In the aftermath of a crisis, there is a lack of time, resources, expertise, leadership, and organizational capacity to focus on learning. There is also a desire to go back to normalcy, which decreases interest in a recent crisis. Further, drawing the appropriate conclusion is confounded by considerable uncertainty and ambiguity, in turn making it difficult to arrive at the correct causal inferences or explanations. The accompanying politicization of crises may then further inhibit the learning process.

There is also the question of what is learnt, especially which lessons are more permanent and, thus, relevant for future crises. Argyris and Schön (1996) describe two types of organizational learning: single‐loop and double‐loop. In single‐loop learning, the action strategies (i.e., the behaviors) change without changing the governing values (i.e., values, norms, and objectives). Thus, the organization modifies its performance within the existing sets of rules. Moynihan (2009, p. 189) refers to this as what “allows organizations to do the same things better.” In double‐loop learning, the governing values also change. Learning in this way goes beyond simple corrections and also questions and alters the underlying modus operandi (e.g., changes in values, norms, and objectives). For changes and reforms to last, the learning process ought to be comprehensive, conforming to the ideal type of double‐loop learning (Argyris & Schön, 1996). However, Birkland (2009) argues that inquiries after crises normally result in routine lessons‐learned documents that fail to tackle the major structural issues in place, focussing instead on identifying a set of operational matters, which is indicative of single‐loop learning.

Learning from crises also hinges on collaborative or networked forms of learning we may refer to as interorganizational learning and lesson sharing (Moynihan, 2008). Learning within large networks of collaborators and competitors is difficult and is thus a form of learning that remains neglected. Even within a single organization, there is a need to break down barriers between different departments. Overcoming even more barriers is necessary to go one step further to open up and share problems between different organizations (Lagadec, 1997). The different actors probably also have different views of the situation, therefore, the picture of the event at a specific point in time will likely be fragmented (Müller‐Seitz & Macpherson, 2013). In a COVID‐19 context, this becomes particularly manifest at the ministerial level, where the totalizing nature of the crisis necessitates the involvement of all actors across all levels. Implementing a policy that suits every sector is a very challenging process. The temporal dimension of the challenge is also a barrier to the successful implementation of lessons, as lessons learned may fade over time, and it remains unclear whether these will stick until the next pandemic is upon us. After all, pandemics last a long time and produce considerable crisis fatigue. A longing for normality thus jeopardizes lesson retention over the long term.

Obstacle 2: Effectively drawing lessons from other countries

What constitutes a successful response in the middle of a crisis? The number of dead and infected, economic performance or unemployment data often figure, yet fail to take into account circumstantial factors that may also have shaped differences in outcome between countries, such as racism, ageism, gender inequities, homelessness, the state of prisons, and the like.

Drawing lessons from other countries is an essential yet challenging approach to policymaking. For the discussion ahead it is necessary to distinguish between learning and lessons. Learning implies a change in behavior, whereas lessons do not necessarily require a specific change in behavior; they are valued at how they could be implemented and yield a positive outcome in a setting other than the original (Rose, 1991). In the context of drawing lessons from the COVID‐19 pandemic (or drawing lessons from any crisis), one could face different obstacles and barriers for successful implementation of lessons learned in other national policy contexts. Rose (1991, p. 7) defines lesson‐drawing (on a government‐level) as an “action‐oriented conclusion about a programme or programmes in operation elsewhere.” In effect, a lesson focuses upon specific fields of expertise, such as public health. A lesson could be drawn across both time and space, and subsequently be implemented and expected to yield results, either positive or negative, within a given timeframe. This could be described as the lesson's transferability or potential for policy transfer. For policymakers, the concept of lesson transferability is of greater importance than the lesson itself (Rose, 1991). Lessons learned in a country that handled the COVID‐19 pandemic in an acceptable manner do not automatically suggest that another country adopting those lessons will experience the same results.

Furthermore, people interpret situations through their pre‐existing worldviews and analytical assumptions or biases. As different people might understand and interpret unfolding events in distinct ways, lessons will also vary (Levy, 1994). The understanding of a crisis or parts of it always takes the form of one or several crisis narratives (Lidskog, 2018). When new ideas about the world manage to challenge existing policies they have the opportunity to affect the choice of problems to address and, thus, which solutions to implement. COVID‐19 is a case in point, as ideas about which policies and measures are defensible vary greatly between national, cultural, and political contexts.

The reasons for policy failure when implementing best practices observed abroad vary due to such factors as lack of contextual information, incoherent lesson‐drawing leading to policy incoherence, and the inappropriate transfer of less relevant responses (Dolowitz & Marsh, 2000). Policy transfer is also shaped by power dynamics. These factors represent different sources for failure during policy transfer and shed light on obstacles to drawing on experiences from outside contexts. Uninformed transfers may be attributable to the adopting country having insufficient information or knowledge about the original policy and/or the originating country. In other cases, incomplete transfers occur because critical elements of the original policy and conditions for its success in the originating country may not have been successfully copied. Inappropriate transfer can occur as a result of insufficient understanding of differences in local factors like social, political, economic, and ideological differences between the originating and the adopting country (ibid.). In the health sciences, for example, medical science is more or less internationally standardized through the inherently global nature of high‐end research and its professional norms, and yet no two national health systems are alike in practice.

With the rapid spread of COVID‐19, many governments looked to neighboring countries when deciding on how to handle this rapidly developing situation, effectively copying the responses of others in many cases. Most countries have implemented or adopted some sort of strategy to handle COVID‐19 within their respective jurisdictions. Previous research into policy transfer (Stone, 1999) is in line with this observation, suggesting that countries or systems similar to each other (e.g., cultural, ideological), have a better chance of fruitfully transferring policies between one another.

The concept of best practice is in part used as a way of rendering policy successes more “export friendly” so as to export “models” to other countries or systems in the form of “model responses.” In the aftermath of COVID‐19, there will likely be heated debates over which national response strategies should be considered best practice and which normative values should underpin the way we think about pandemics (acceptable socioeconomic cost vs. acceptably low mortality and morbidity). Different aspects will surely be valued in different ways in different countries due to factors such as level of economic development and national political climates. Either way, the response that “wins” this potential debate will likely set precedence for how to respond to such crises for years to come and will perhaps close the window of opportunity for certain alternative approaches, as one type of “model response” is cast as “best practice,” involving a degree of path dependency.

Obstacle 3: The potential for reforms to introduce unanticipated vulnerabilities elsewhere in the system

As a crisis unfolds, an intense pressure to prevent and prepare for similar future occurrences is initiated. This is oftentimes at the expense of other similarly threatening risks (Carley & Harrald, 1997). It is common to emphasize the tendency for authorities to keep “fighting the last war” or center excessively on “avoiding the failure of the past.” Paradoxically, in other words, we strive to prevent the reoccurrence of recent crises while at the same time being acutely aware that the next crisis episode may be radically different. In epidemiological terms, the next pandemic could involve a quite different microbe, rendering some parts of our current knowledge less relevant. Learning processes in this way are prone to introduce new vulnerabilities when lesson‐drawing processes are overly event‐specific, in the sense that reforms made in the wake of a particular event may transform the system in such a way as to leave us ill‐prepared for future events that we know may be of a completely different nature and have new types of challenges.

To describe this dilemma, Lauder (2013) coins the concept of “the reverse fallacy.” His work describes the potential for lesson‐drawing or learning processes to introduce new weaknesses by removing certain organizational functions that, in hindsight, turned out to be more useful than recognized. A basis for this tendency is the well‐known dilemma that success in crisis management work often goes unnoticed, simply because success means that few contingencies arise in one's own area of competency. Further, during times of high uncertainty, while the crisis is unfolding, there is a desire and high pressure for rapid decisions aimed at solving short‐term problems as they surface. However, this tendency threatens to produce long‐term problems and policy or procedural incoherences as specific solutions are made to address short‐term challenges. For example, responses to the current pandemic and the lessons we ultimately draw from it may render us more vulnerable to future pandemics, if they behave in a radically different way but our health systems are excessively reformed for a “new COVID‐19.”

It is also becoming increasingly clear that preparing for a growing array of novel types of events places considerable strain on societies and already crowded agendas. One problem is that accurate foresight and prediction are not available to us. Instead, we must acknowledge and consider the potential for nonlinearity in immediate events occurring before us, while also appreciating the inherently unpredictable nature of the future. In essence, therefore, lesson‐drawing from current historical events will often initiate short‐term thinking—a focus on patching immediately obvious weaknesses while potentially rendering the system ill‐prepared for novel forms of challenges.

An important insight from early disaster research (Quarantelli, 1988) has thus been that planning should be built around general principles, not overly specific or event‐centered lessons. It is likely that maintaining a focus on general principles that may be easily adapted to novel circumstances will leave us better equipped to deal with the unforeseen. In other words, we should ensure that reforms made in the aftermath of the COVID‐19 pandemic render us better prepared for future pandemics and also that implemented changes will not introduce new vulnerabilities elsewhere in the system.

Obstacle 4: Political pressure

The perceived failure or success of a person greatly influences their career, especially in politics and expert‐led decision making. This generates an intense pressure to do something—and preferable something that does not pass by unnoticed. Extreme events, such as the COVID‐19 pandemic, attract media attention and considerable political saliency as well as public interest. Symbols and rituals become important to demonstrate a sense of control (‘t Hart, 1993), but drawing on these inappropriately may also give rise to scandals (McConnell, 2011). How, then, can the need for demonstrating leadership capability to the public be balanced against an overly symbolic and politicized response?

It is not easy to objectively assess the quality of crisis management work. Narratives serving to construct the chain of events are continuously debated and contested (Boin et al., 2009). In this way, disasters hold the potential for triggering political crises in their wake. We can consider this a result of the difficulty of defining the nature of events as history unfolds before us; those with the power of definition will ultimately shape the narrative disproportionately. Such framing contests concern factors like the nature and severity of a crisis, its causes, the responsibility for its occurrence or escalation, and also implications for the future. Different narratives will be framed and reframed and different actors will want to have their frame accepted as the dominant narrative. The “frame contests” thus become an accountability process and potentially a political game aimed at identifying scapegoats. This is potentially a major obstacle to learning.

Lesson‐drawing is an equally contested process as debates about successes and failures are likely to have a central role in the setting of precedent for future, similar crises (Birkland, 2009). Hence, it is not necessarily what takes place “on the ground” that matters most, but whether and how the public is made aware of actions taken, as well as their effects. Ultimately, however, the narrative concerning causal claims is subjective, with a tendency to prioritize visible measures over equally effective but less visible measures. In other words, it is not actions alone that shape the political dynamics of pandemic response, but public perceptions of performance, which also change over time due to, for example, fatigue from complying with officially mandated response measures.

Perhaps the most important implication to keep in mind is that the political space for maneuvering becomes severely constrained (or alternatively broadened) by the narratives simmering in the crisis aftermath. These stories will, in turn, and as we have seen, shape which lessons will ultimately be accepted as model responses.

Obstacle 5: Drawing conclusions from observations

In complex systems, of which society is perhaps the most puzzling type, it is rarely possible to observe direct causal mechanisms at play when many measures are implemented simultaneously. In the case of COVID‐19, social interaction also changes irrespective of measures due to the knowledge of current risks, which further complicates the attribution of effect from individual measures. It thus becomes difficult to conclude that observable outcomes are a direct result of specific measures and to understand the degree to which contextual factors played a role in shaping present outcomes. Although the effects of some interventions can and are indeed being measured, the methodological challenge involved renders the task prone to potentially drawing the wrong conclusions from observations. Another problematic issue is that we often over‐learn and over‐generalize from current crises due to their tendency to dominate attention in the present.

What is considered a successful response is, as we have seen, also not objective. Individuals disagree about the normative foundation of response strategies, such as how to balance priorities concerning safety and freedom. Although it may seem easy to identify relevant factors to judge the success of a crisis response through such indicators as mortality rate or reproduction numbers, the yardstick will always depend on the values and goals of commentators and the public, in turn, based on culture and prevailing meta‐narratives. There are in fact no agreed criteria for evaluating crisis management success for complex crises involving trade‐offs. Whether a response is considered a success or a failure is thus a matter of both “fact” and “perception” (McConnell, 2011).

The standards chosen for the evaluation will always favor some aspects over others and be shaped by the winning causal explanations of the situation. The most salient narrative will be the basis for determining the result of the evaluation and will define what is perceived as relevant lessons (Lidskog, 2018). Determining whether a response to the COVID‐19 pandemic is successful or unsuccessful will likely depend on the frame of the analysis chosen, such as whether the focus is just the spread of the pandemic or if one has a broader socioeconomic or public health focus. Although lessons connected to COVID‐19 may well also have positive spillover effects, such as being beneficial for mitigating seasonal influenza, it remains unclear how postcrisis reforms will prepare us for the next big event.

Obstacle 6: Experts versus decision makers

Crises inevitably put decision makers to the test as they are likely to be confronted with urgent decisions on matters outside their area of immediate expertise or experience. Furthermore, decision makers not only have to manage contingencies as they arise, but they must also generate a public image of relative success. Thus, they must anticipate the direction of criticism and opposition (e.g., response fatigue in the electorate). Typically there is both an operational on‐the‐ground response and a more political/symbolic response occurring in parallel, which demands that crisis communicators are up to the challenge.

Decision making is performed at different levels, both at a strategic level consisting of the political elites or policymakers and at an operational level consisting of people with technical know‐how. In the context of the COVID‐19 pandemic, most countries have witnessed an intense internal struggle between political leaders and expert authorities from public health agencies. Although health authorities are charged with providing expert advice to decision makers, the way in which expertise should be implemented is ultimately up to political leaders.

During periods of crisis, decision makers are inclined to include experts in their decision‐making processes to signal that decisions are sound and evidence‐based. Thus, there is a question of the extent to which decision‐makers base their decisions on expert opinion, as opposed to making politically motivated decisions. Communication is also a key part of this process, providing much‐needed legitimacy and purpose to otherwise unpopular outputs. Relying on expert advice may legitimize decisions and build trust, with the added benefit of potentially mitigating the effects of blame games in the aftermath. At the same time, a strongly expert‐led response may lower the degree to which political leaders experience a sense of control. Expertise may reduce the uncertainty by providing meaning and sense to the event in technical terms, but may also enhance uncertainty if expert advice is mixed or if the evidence base is thin. Consequently, in some cases, expert advice may increase rather than decrease the experience of uncertainty (Broekemaa et al., 2008). Further, the tendency to push accountability away from political leaders and over to nonelected expert authorities is in itself problematic in a democratic sense. In messy crisis circumstances some outcomes may be related to unforeseen contingencies but ultimately end up delegitimizing otherwise sound science. A central concern for decision makers is often how to balance expert input and political leadership; such concerns also have critical implications for the learning process.

Obstacle 7: Reforms may not be related to the actual crisis

As mentioned, crises may be seen as opportunities for change. In hindsight, however, implemented changes are not always possible to trace back to the nature of the crisis in question (Birkland, 2009). Crises generate greater maneuvering space for reform entrepreneurs and may give a strong impetus to otherwise dormant political agendas pushed by interest groups. Problem definition becomes important since it also brings to mind potential solutions. These solutions can be related to the specific response, or they can be pre‐packaged solutions that already existed before the specific event and even not be related to the crisis in question. In the context of COVID‐19, we may well see a wide range of reforms that appear connected, but that, in reality, have little to do with the pandemic or with its consequences. Another risk is that COVID‐19 could become an excuse for opening up opportunities to shift the concentration of power; conflicts could arise if some leaders are unwilling to give up their power in the crisis aftermath.

Public inquiries do not always have the goal of sparking reflection and can rather be seen as a scripted ritual. Birkland (2009, p. 154) argues that “many of these documents and the process that creates them are mere reflections of a group's or interest's preferred social construction of a problem and its ‘target population’.” Similarly, Moynihan (2009) argues that evaluations commonly end up as a very routine product focusing on just operational or tactical matters. Lengthy learning processes are known to be effective tools for mitigating critique and calming opposition. Overly normative learning processes may elaborate at length about what could have been done but offer little advice on how to actually address weaknesses.

CONCLUDING REMARKS

COVID‐19 remains one of the defining crises of our time. As a direct result of its high political saliency, however, drawing appropriate lessons from the COVID‐19 pandemic is difficult because a number of potential “model responses” contending for precedence are likely to emerge in the aftermath, pinning very different countries with different value systems against one another. What constitutes best practices are inherently normative questions and depend on how values interact, making discussions of success and failure highly political. The drawing of lessons is thus complicated by a range of political, economic, and cultural factors, as well as matters of individual preference.

Seven barriers to fruitful learning were identified in this article. The first of these is the challenge of retaining experiences and lessons for a sufficiently long time for them to be useful for a future, similar event. Second, countries generally look to others for inspiration on the kinds of strategic responses available, but drawing lessons from abroad is not a straightforward process. Third, reforms, especially large ones, can sometimes introduce new vulnerabilities into the system by either removing functions that later turn out to have been central or by rendering the learning process too event‐specific. Fourth, politics may get in the way of effective learning due to changes in the political landscape and maneuvering space. Fifth, the wrong kinds of lessons may be identified if the causal mechanisms underlying the learning process have poor validity. Sixth, conflicts and inconsistencies between expert/health authorities and political leaders will likely make accountability questions hard to address. Lastly, the reforms that ultimately are presented might not be clearly connected to the actual crisis in question, due to the time between the crisis and the reform implementation, or because of contradictory agendas steering the nature of reforms. In any case, it is clear that the aftermath of the crisis will be dominated by intense discussions of “model responses” and the kinds of responses that should set precedents for future outbreaks.

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.

Biographies

Kerstin Eriksson is a researcher at RISE Research Institutes of Sweden within the research areas of crisis and disaster management. She holds a PhD from Lund University, Sweden.

Reidar Staupe‐Delgado is a Marie Skłodowska‐Curie Actions Individual Fellow (MSCA‐IF) at Roskilde University, Denmark and an associate professor at UiT The Arctic University of Norway. He holds a PhD in Societal Safety from the University of Stavanger, Norway.

Jørgen Holst is an emergency manager and analyst. He holds an MSc in Societal Security from UiT The Arctic University of Norway where his research centered on learning in organizations.

Eriksson, Kerstin , Staupe‐Delgado Reidar, and Holst Jørgen. 2022. “Drawing Lessons from the COVID‐19 Pandemic: Seven Obstacles to Learning from Public Inquiries in the Wake of the Crisis.” Risks Hazards Crisis Public Policy 13, 165–175. 10.1002/rhc3.12240