Skip to main content
Open Research Europe logoLink to Open Research Europe
. 2022 Aug 26;2:52. Originally published 2022 Apr 29. [Version 2] doi: 10.12688/openreseurope.14566.2

Non-pharmaceutical interventions during COVID-19 in the UK and Spain: a rapid realist review

Pilar Serrano-Gallardo 1,a, Ana Manzano 1, Ray Pawson 1,b
PMCID: PMC10446037  PMID: 37645319

Version Changes

Revised. Amendments from Version 1

As requested by Dr. Kaehne, we have added a new section, "A methodological note on Rapid Review", which contains some additional explanations and key references on the methodology of Rapid Realist Review. This type of review is derived from the Realist Synthesis, developed by Pawson to evaluate the effectiveness of social interventions, which is much more difficult because they vary significantly in their design, management, and implementation; they also vary significantly in their effectiveness in different communities, sub-populations, cultures and time periods; and because they are evaluated in primary studies using different research methods.  The use of complexity theory to conduct this rapid review is justified because the response to the pandemic affected society as a whole. It is a "rapid review" because the primary literature is vast and therefore the reported results reduce and condense the primary materials extracted, encompassing only a set of the most compelling illustrations; furthermore, it was completed in something akin to real time rather than, as is more typical, making retrospective comparisons across the history of a family of interventions; finally, and as with all rapid reviews, the conclusions are suggestive rather than definitive, anticipating the possibility of rival interpretations and with the primary aim being to "open up debate". In addition, we have rephrased "control was never achieved through the application of social controls" to "management of the virus was never fully achieved through social controls".

Abstract

The paper is located at the crossroads of two modern intellectual movements. The first, evidence-based policy, seeks to locate vital information that will inform and improve key policy decisions on such matters as population health, social welfare, and human wellbeing. The second, complexity theory, describes the nature of the social world and perceives human action as persistently adaptive and social institutions as incessantly self-transformative. The first assumes that policies and programmes can achieve sufficient control to meet specific and measurable objectives. The second assumes that social actions are sufficiently capricious so that the society never conforms to anyone’s plans – even those of the most powerful. The unparalleled resources committed to control the unprecedented attack of the COVID-19 pandemic are the epitome of complexity. The long struggle to contain the virus thus constitutes an ideal test bed to investigate this paradigmatic split. The paper undertakes this mission - focusing specifically on the effectiveness non-pharmaceutical interventions and examining evidence from the UK and Spain.

Keywords: COVID-19, non-pharmaceutical interventions, complex systems, counterfactuals, evidence-based policy

Plain language summary

Modern social life consists of millions of interactions as people conduct the routine business of everyday living. We work, we play, we teach, we learn, we shop, we travel, we marry, we touch, we help and seek help … we congregate. Is it possible to subdue all this activity? Is it possible to limit all of these interactions and still maintain society’s basic functions? This extraordinary question became a reality as political leaders and policy makers confronted the unprecedented COVID-19 pandemic. As they awaited the vaccine programme, social control measures became the primary means of protecting public health. The result, throughout Europe, was an unparalleled programme of public restrictions – lockdowns, curfews, border closures, travel bans, the closure of stores, stadiums and schools, the requirement to work from home, rules on social gatherings and distancing, compulsory mask wearing, hand hygiene, and so on. The paper evaluates the effectiveness of these measures, piecing together evidence from scores of primary sources, using the UK and Spain as case studies. The controls provided limited success, rather than a steady decline in infections, rates fluctuated persistently and rebounded through several waves. The paper seeks to identify the reasons for the erratic progress, the core problem being the sheer complexity of the response. Faced with a monumental life transformation, the citizens of COVID responded in unanticipated and unpredictable ways which, with the benefit of hindsight, it is now possible to see more clearly. The paper concludes with some lessons learned, which may improve the future conduct of crisis management.

Introduction

We begin by acknowledging that the unprecedented ferocity caused by SARS-CoV-2, the precipitous transmissibility of the virus, and its ability to mutate genetically. Globally, at the time of writing, the pandemic has resulted in 380,321,615 confirmed cases and 5,689,741 deaths ( WHO Coronavirus Dashboard, Feb 2022). All this despite what is often and quite correctly referred to as the largest programme of interventions and the largest expenditure of public resources outside of wartime ( Financial Times, 2020). It is over two years since the onset of the pandemic in December 2019 and throughout the world, inquiries, formal and informal, have begun on the effectiveness of that response ( Talic et al., 2021).

How will history judge the policies and interventions mounted in the attempt to contain the virus? To be sure, there will be much political finger-pointing and partisan name-calling. The scientific response will be calmer and more considered, but it can be said with some confidence that the vaccination programme will be considered the ‘game changer’. Solid, real-world evidence exists on the effectiveness of a range of vaccines in reducing virus transmission, hospitalisation rates and COVID-related deaths (e.g., GOV.UK, 2021d). No policy is perfect, of course, so it is also likely to be acknowledged, both globally and nationally, that the supply and rollout of the vaccines has been remarkably uneven with significant adverse consequence ( Nature, 2021). It is also clear that there has been a considerable drag on progress due to vaccine hesitancy and intense ‘anti-vax’ campaigning ( Benoit & Mauldin, 2021) as well as grossly inequitable vaccine rollout in poorer countries ( Krishtel & Hassan, 2021).

What is far from clear is how history will judge the pandemic management policies mounted, pre-vaccination, in the first 18 months of the worldwide response. These interventions were variously known as ‘lockdown’, ‘mitigations’ or, more prosaically, ‘non-pharmaceutical interventions’ (NPIs) ( Mendez-Brito et al., 2021). Once again, the enormity of this response must be acknowledged. It consisted of a wholesale attempt at behavioural control across entire populations, which demanded huge adaptations to public services across all social domains – not only healthcare but also social care, education, transport, policing, border control, the economy, etc. ( Dodds et al., 2020).

It is striking, looking back at the early days in the spring of 2020, that it was believed that growing levels of COVID-19 infection could be managed by turning on particular packages of NPIs at particular times ( Zhou et al., 2020). Political messaging over this long period was all about the need to add and occasionally to reduce the array of controls in order to maintain control over current rates of infection. We now know that these mitigation strategies struggled to succeed and that non-pharmaceutical aspirations gave way to pharmaceutical actuality. Such a therapeutic paradigm shift, however, leaves behind an intriguing counterfactual question - without the biological discoveries, would the social control strategies, in the long run, have struggled to succeed?

We will never know of course. Thinking about what has not happened but could have happened is the stuff of ‘counterfactual history’ ( Evans, 2014). Despite the fact that such exercises in ‘altered pasts and alternative futures’ are necessarily conjectural, there are important lessons to be learned by posing the question and this is our task in the remainder of the paper. The first section examines European data relating to transmission curves, which reveal how well and how poorly the virus was brought under control. In particular, we contrast the initial expectations and models of how the virus would behave with data on the rolling averages of transmission over the first 18 months of the pandemic. Wave after wave, public restraints and surveillance strategies were implemented, which resulted typically in a start, stop, start, stop pattern of virus control.

The next part of the paper seeks to explain these fluctuation outcomes. Our basic thesis here is that the first-wave policy response was significantly undermined by its own complexity, and we call on the assistance of ‘complexity’ or ‘systems’ theory to support this hypothesis. Much of the development of these ideas has been conducted at high levels of abstraction in methodological journals and it is important to convey that these system dynamics are in fact routine features of all public policy and healthcare programmes. To this end, we produce a simple typology of seven modes of system complexity, showing how each is deeply embedded in the response to COVID-19.

The paper then proceeds by furnishing more detailed illustrations of each of these dynamics in the virus mitigation strategies (NPIs) implemented in the UK and Spain. We restrict our analysis to two countries in the expectation that the mechanisms described will strike a chord with readers familiar with other policy systems across Europe. All manner of controls, guidance, support, and legislation were unleashed, which generated a tangle of contradictory forces, blocked opportunities, displaced effects, and unintended outcomes. Underperformance was the norm. But was this necessary?

This brings us to our conclusion. The world over, the policy track record of tackling ‘wicked problems’ ( Rittel & Weber, 1974) with complex interventions is not impressive. But complexity should be regarded as a challenge rather than an adversary. Our account reveals many failures, but it does suggest opportunities for learning about crisis management and evidence-based policy. These potential gains are described in the final section.

COVID-19 transmission: expectation and reality

In the first stages of the COVID-19 pandemic, political leaders throughout the world were quick to vouchsafe that they would ‘follow the science’ ( Sasse et al., 2020). The science in question, mathematical biology, was hardly known to the public but in the space of a few months the basic concepts and imagery associated with the modelling of infectious disease became remarkably familiar – particularly the phrase capturing the overall aim of ‘flattening the curve’. According to classic epidemiology the basic transmission curve in a major epidemic takes shape as a disease moves through four groups – the susceptible, the infected, the recovered and the deceased. There is a generic pattern, the bell shape, whereby the number of cases increases exponentially until the proportion of the susceptible has been sufficiently depleted (through recovery or death) so that the growth rate then slows and the number of cases drops eventually so that the epidemic is no longer sustained ( Keeling & Danon, 2009). This ‘natural’ shape of transmission graph is illustrated in the upper section of Figure 1, labelled the ‘number of cases without protective measures’ (known colloquially as the ‘do nothing curve’).

Figure 1. Expectation.

Figure 1.

Source: Pawson (2021). The coronavirus response: boxed in by models. Evaluation, 27(2), 149-167. https://journals.sagepub.com/doi/full/10.1177/1356389020968579.

The policy response, of course, is to ‘do something’ and this results in the array of interventions aimed at mitigating the spread of the virus – hand hygiene advice, provision of protective equipment, installation of additional ventilators in ICU units, recruitment and training of extra critical care staff, the closure of shops, stadiums and schools, social distancing, curfews, lockdowns, travel bans etc., etc. The challenge facing the mathematical modelers is predict the revised trajectory of the disease under the protective measures using statistical estimates of the collective impact of various permutations of the interventions just mentioned. The expectation was that it is possible to calibrate closely the response that a forward programme of interventions would deliver. Implementation of the optimal set of measures would then flatten the curve, thus reducing disease impact so that casualty rates would not exhaust health system capacity ( Pawson, 2021) (see lower portion of Figure 1).

Policy makers in each European country were in receipt of such advice, the recommended batches of measures were implemented, and the predicted models proved fruitless ( James et al., 2021). The underlying statistical estimates of how the virus could be brought under control were simplistic, based on estimates of how people would respond to each potential measures, rather than treating that response as an adapting, self-transforming whole system ( Pawson, 2021). Much of the real reaction, negotiation, fatigue, dispute, confusion and sheer muddle produced by the NPIs is thus lost on the models. We provide evidence for these bold claims in a subsequent section but first it necessary to explore the real shape of COVID-19 transmission. Figure 2 with its mountainous landscapes of infections provides a stark contrast with the aspirations in Figure 1.

Figure 2. Reality.

Figure 2.

Source: BBC News https://www.bbc.co.uk/news/uk-58783591.

This is one of the renowned John Hopkins ‘dashboard maps’ (2021), relating specifically to the pre-vaccination period. It is important to note that the graphs in Figure 2 are not to scale and that only incidence is mapped. Death and hospitalisation rates also fluctuate just as significantly but with a different rhythm. Data collection methodology and operationalisation (and rigour) also vary from country to country. Two common features stare out from the rolling averages and provide the focus for the reminder of the paper. 1) rather than a steady rise and decline in infections, rates fluctuated persistently and rebounded through several waves. 2) management of the virus was never fully achieved through social controls but only began to improve with the arrival of the vaccination programme in late 2021. How can we begin to explain these uncontrolled fluxes?

Applying complexity theory to the COVID-19 response

The thesis here is that NPI policy regimes across Europe were undermined by their own complexity. There has been a significant ‘turn’ towards complexity and systems thinking across the social sciences in recent years ( Williams, 2021) and this orientation features increasingly in policy analysis ( Daviter, 2019), in implementation science ( Braithwaite et al., 2018), and most significantly in healthcare, via the UK Medicinal Research Council’s guidance on the evaluation of complex interventions ( Skivington et al., 2021). Complex systems are self-transformative, driven by an ecology of agents who interact, compete, and adapt. Novel outcomes emerge constantly, generating all manner of unintended consequences. The properties of complex systems are dissected in great detail in the MRC framework (adaptation, emergence, perverse consequences, feedback loops, blockage points and structures, non-linearity, contextuality, tipping points, path dependency, openness, self-transformation, etc.). We simplify this analysis here, identifying seven key modes of system complexity ( Table 1). For each dynamic, we provide a brief discussion of how it was embedded within pandemic crisis management.

Table 1. Modes of complexity.

Mode 1: Disparate
command and
control systems
This mode of complexity occurs in social systems managed under multiple ‘hubs’ and ‘relays’. Most national
responses to the COVID-19 pandemic operate with a measure of regional and administrative independence and
with long implementation chains. Rather than a uniform unfolding of policy a ‘system of policy systems’ develops.
Complexity multiplies according to the connectedness of the networks of control. The more autonomous the
network of hubs and relays, the more likely that inconsistencies, contradictions, and conflicts will ensue.
Mode 2:
Interaction and
emergence
Complex policy systems utilise many different measures. In the case of COVID-19 the controls applied include
hand hygiene, protective equipment, closing of stores, stadiums and schools, rules on social distancing, curfews,
travel restrictions, working from home, etc. The effects of these actions are not additive but interactive. Each
intervention conditions the others, often in unanticipated ways. The combination of measures generates emergent
effects, which may complement each other but also can reduce, compete with, or displace the intended effect.
Mode 3: Policy
discord and moral
disharmony
It is impossible to separate complex interventions from the human agency required for their delivery. Draconian
restrictions on everyday activity meet with diverse responses from different sections of the public, with the
potential to create a moral struggle between those who ‘care’ about complying with controls and those who ‘do
not care’. The wayward actions of notable ‘free riders’ , who flout the restrictions may cause a tipping point leading
to a sharp and unpredictable increase in moral disharmony.
Mode 4:
Contextual
heterogeneity
Complex interventions always create unequal outcomes. The COVID-19 pandemic does not affect everyone
uniformly – with vulnerable groups, such as ethnic minorities, the elderly, those with chronic diseases and
disabilities, those living off the informal economy, etc. being hit the hardest. Some of the measures designed to
reduce infection and death, by ignoring such contextual heterogeneity, had the perverse effect of intensifying
health inequalities and reducing the impact of controls.
Mode 5: Ambiguity
in regulations and
guidelines
Policies begin life as text in the form of regulations or guidelines. In complex, hastily assembled interventions
like COVID-19, some ambiguity in official documentation is inevitable, with unclear pronouncements introducing
further diversity in the public response. Uncertainty arises because of: i) the legal status of the guidance - what is
law and what is merely advisory? ii) ambiguities in the wording or phrasing of the guidance, iii) the remoteness of
official communication channels from everyday behaviour.
Mode 6: Temporal
change in public
attitudes
Public allegiance and adherence to complex interventions vary over time. There is a typical rhythm to public
commitment – often moving from enthusiasm, to acceptance, to routinisation, to fatigue. This progression
is pronounced, as in the COVID-19 restrictions, when unprecedently severe controls are maintained over a
considerable period of time. The pace of change in such motivational patterns is not predictable and not under
the control of policy makers.
Mode 7: Exit and
sustainability
effects
Sustainability represents one of the greatest challenges in healthcare policy – what happens when interventions
wind down? In the case of COVID-19, the expectation was that social controls may be removed once infection
and death rates have diminished sufficiently. The easing of restrictions is itself a significant policy manoeuvre and
carries all of the uncertainties and complexities associated with their introduction. The timing and execution of exit
strategies is thus unpredictable; the relaxation of measures often generate ‘rebound effects’.

Complexity in pandemic management in the UK and Spain

The paper proceeds by furnishing some detailed illustrations of each of these dynamics in the virus mitigation strategies (NPIs) as implemented in the UK and Spain. The methodology used here is based on ‘realist review’ a method of research syntheses which collects together evidence on programme theories or explanatory themes ( Pawson, 2006). Here we present a 'rapid review', highlighting just a few key instances of each mode of complexity ( Saul et al., 2013). For each of the modes of complexity shown in Table 1, examples from the United Kingdom and Spain are uncovered, with the aim of promoting debate. The process uncovered in our analysis go a long way to explaining the halting and intermittent nature of virus control as described in Figure 2. Although the analysis is restricted to two countries, the mechanisms described will strike a chord with readers familiar with other policy systems across Europe.

A Methodological Note on Rapid Review

Realist synthesis (RS) was developed at book length by Pawson (2006). A number of extensions and clarifications followed, most notably a formal account of publication standards by Wong et al. (2013) and a practical guide for information specialists ( Booth et al., 2018). RS is a method specially designed to review the evidence on complex social interventions. Accordingly, it has found much use in public health research, typified by its usage here in evaluating the effectiveness of the unprecedent magnitude, expenditure, duration, and complexity of the measures designed to prevent the spread of the coronavirus infection.

The methodology of evidence synthesis has blossomed in recent decades. The founding approach, known as ‘systematic review’ or ‘meta-analysis’, originated in medical research, and sought to provide a statistical estimate of the mean effect of a particular treatment or therapy across many different clinical trials. The effectiveness of social interventions is much more difficult to capture in this singular fashion - because they vary significantly in their design, management, and implementation; because they vary markedly in their effectiveness in different communities, sub-populations, cultures and time-periods; and because they are evaluated in primary studies that utilise decidedly different research methods. The ‘mean effect’ is meaningless under such a scenario. Accordingly, many new methods of evidence review have proliferated ( Sutton et al., 2019), one of which is realist synthesis.

The purpose of RS is explanatory, it seeks to understand for whom, in what contexts, and in what respects an intervention works and above all why. It assumes there will be successes and failures and seeks to ascertain why. The review sequence can be summarised as follows: 1. Search and ascertain material on the programme theories, the ideas underlying the intention. 2. Seek primary studies that provide a test of those theories and extract the pertinent data. These primary sources may utilise any and all social research approaches. 3. Assess the rigor and relevance of each piece primary research. 4. Synthesise the emerging findings to assess in what respects the programme expectations are met and in what respects they may fall short. 5. Disseminate the findings, with an assessment of their generalisability and policy implications, as well as a statement on the limitations of the synthesis.

These steps are followed in the above account, though it must be emphasized that this is a ‘rapid review’. It is rapid in several senses – I) The primary literature is gargantuan and so beyond any single summary; one 2021 estimate traces more than 87,000 scientific papers in many different languages on the coronavirus pandemic ( Grabmeier, 2021). II) Pawson and Manzano’s contribution to the study were unfunded and the review was relatively under-resourced. III. It was completed in something like real time rather than, as is more typical, making retrospective comparisons across the history of a family of interventions. IV) At paper length, the reported findings reduce and condense the extracted primary materials, covering only a handful of the most compelling illustrations. V) As with all rapid reviews, the conclusions are thus suggestive rather than in any sense definitive. Indeed, they anticipate the possibility of rival interpretations and the primary goal is ‘to open the debate’.

We believe our review is ground-breaking in its use of complexity theory as the lens through which to understand the mixed fortunes of an intervention. Normally review step 1 uses material extracted from the accounts of programme architects and policy makers. The usage of complexity theory is justified in the main text (basically because the response to the pandemic touched every corner of society). But note, as a consequence, that this RS involved a preliminary step of reviewing the complexity literature itself to ascertain which of its many accounts and dimensions would feed into and best support a public health study.

The overall research design was devised by Pawson. He selected and reviewed the material used in the UK section of the study. Serrano Gallardo and Manzano conducted the parallel review on the Spanish response, calling on and inserting further materials from their own experience of evaluating public health programmes.

Mode 1: disparate command and control systems

Spain is a country administratively divided into 17 autonomous communities and two autonomous cities, which have considerable governmental independence, both at the legislative, financial and executive levels. The Spanish Constitution grants the government the power of declaring a ‘state of alarm’ in exceptional emergency circumstances, giving the national government authority to overcome the devolved powers. At the time of writing, it had been used three times during the pandemic: first strict confinement (March to June 2020); confinement of the autonomous community of Madrid (October 2020); and national night curfew (from 11pm-6am) from October 2020 to May 2021.

The bigger picture, however, is that guidelines and mandatory regulations have been issued by the central state and by the regional governments of the 17 autonomies. Accordingly, some autonomous communities followed the actions recommended by government but not others and, significantly, the recommendations were implemented at different times (early vs late adopters). For example, Madrid did not close bars and restaurants at all following the conclusion of the stringent national home confinement in June 2020, despite showing infections and mortality rates well above of the national average ( Instituto de Salud Carlos III. Centro Nacional de Epidemiología, 2021). By contrast, Extremadura (western region bordering Portugal) maintained the longest bar and restaurant closure, achieving the lowest COVID-19 incidence rate in Spain after the third wave ( Manchado, 2021).

In the UK, health policy is also a devolved matter, with England, Scotland, Wales and Northern Ireland having their own mandates. In this instance there is little difference between the chosen package of measures; the only variations being modest disagreements on their optimal timing ( Institute for Government, 2021a). However, the main frictions in control powers in the UK are to be found within specific delivery systems. Figure 3 pictures how the UK Department of Health and Social Care (DHSC) funded but then ceded management responsibility for its ‘Test and Trace’ system to the private sector ( Triggle et al., 2020).

Figure 3. Cooperate responsibilities for UK COVID tracking.

Figure 3.

Source: BBC News https://www.bbc.co.uk/news/health-55008133.

As can be seen more than two dozen agencies are involved, mostly private companies, each with its own priorities and agendas. It provides a classic example of a system with an over-complex command and control structure. Subsequently, the National Audit Office (2020) reported on a catalogue of complications and conflicts. For instance, the freshy appointed contact tracers (to a large extent government funded) were vastly underutilised (4% of contracted time). Targets were missed routinely, the DHSC had assumed that each case transferred to the tracing system would provide 10 to 30 contacts (the actual number was 2.4). Detailed analysis of the Spanish contact tracing system reveals a similarly labyrinthine implementation map with similarly disappointing detection rates ( De Vega, 2020) ( Alonso & Larraz, 2021). Too many cooks spoiled the broth – substantial public resources were wasted.

Mode 2: interaction and emergence

Estimates of the impact of individual prevention measures often proved erroneous, sometimes with deadly effect. All interventions, however well intentioned, depend for their impact on what happens elsewhere in the system. In the first wave of the UK virus, the high risk of hospital acquired infection ( Heneghan et al., 2020) and the urgent need for more hospital space to treat COVID patients led to a programme of discharging elderly patients to care homes. This strategy succeeded in its primary, numerical aim but failed to include a testing programme to accompany transfer and so displaced the problem causing a substantial surge in care home transmission. Thirty-three care home outbreaks in the first week of March 2020 turned into 793 by the end of the month ( GOV.UK, 2020). Increasing COVID-19 space and services in hospitals also led to substantial shortfalls in routine and planned care. Cancer services were significantly affected with a growing backlog for referrals as well as delays and cancellations in first treatments ( Macmillan Cancer Support, 2020).

In Spain, the massive spend on COVID-19 care also syphoned resources from other parts of the healthcare system – but with some subtle differences. Many primary healthcare centres were closed during the first wave, many have reopened since, but in many of these centres, services remained rationed and suboptimal. Once again, other services such as home healthcare were substantially reduced and waiting lists for outpatients and elective surgery increased. During the first wave, the vast majority of nursing home residents who fell ill with COVID-19 in Spain were not transferred to hospitals, but remained in the same nursing homes, which had neither the resources, nor the knowledge, nor the abilities to properly handle the situation. The highest percentage (66%) of deaths from COVID-19 occurred in nursing homes ( Costa-Font et al., 2021). Once again, this phenomenon was not the same throughout the national territory. Excess mortality was registered in 7 of the 17 autonomous communities, which ranged from 10.3% in Castilla-León to 52.5% in Madrid. On the contrary, in 10 autonomous communities there was a decrease in the mortality of elderly people in nursing homes, from 3.8% in Extremadura to 86% in the Canary Islands ( Zunzunegui, 2021).

Across both of our miniature case studies, the implementation of protective measures involves a transfer of resources from other services, with unforeseen and damaging consequence elsewhere in the healthcare system. But this is not a given. It is evident in some rare instances in Spain that opportunities have been grasped for damage limitation.

Mode 3: policy discord and moral disharmony

The public mood and support for government COVID-19 policy follows a similar trajectory in the two nations. ‘Free riders’ are people who gain from the collective effort, without bothering to make an individual contribution. The activities of prominent free riders had a deleterious effect on UK public trust in the management of the epidemic. A longitudinal survey by Fancourt et al. (2020) charts the changes in public trust in the government handling of the pandemic. Starting in May 2020 there was a steep decrease in confidence, which never recovered (until the vaccination programme). This date coincides with the discovery that Dominic Cummings, the Prime Minister’s then closest advisor, had broken lockdown rules with a 500-mile round trip to a family estate. In Spain, the most notorious case involved an entire group of free riders attending a major gala dinner in an enclosed space. Multiple political worthies gathered including the health minister, masks were not worn, and social distancing rules were ignored ( Valdeolivas, 2020).

The fact that such high-profile officials had abstained from collective responsibility ignited a torrent of media abuse in the two countries – ‘one rule for those in charge and one rule for everyone else’. After these high-profile incidents, the negative and lasting decline in public confidence was further exacerbated by crowds of anonymous free riders who gathered in all manner of communal spaces. Trust in the government was lost progressively. According to the Spanish Sociological Research Centre in April 2020, 73% of the population believed that the government was able to manage the pandemic ( Centro de Investigaciones Sociológicas, 2020), whilst in February 2021 only 18.5% held similar views ( Centro de Investigaciones Sociológicas, 2021). Longitudinal polling by Ipsos MORI details a similar substantial decline in the same period in support for the UK government’s handling of the outbreak ( Health Foundation, 2020).

The blame game continued in both countries. In Spain, typically demonised groups such as the youth ( Lledó, 2020) or migrant populations ( Radiotelevisión española, 2020a) were accused of driving up infections and deaths. In the UK, the growing complacency of young people incited both media condemnation (‘the covidiots’) and empathy (‘don’t scapegoat the young’) but, once again, the implications are crucial ( Reicher & Drury, 2021).

The explanation for rising resistance to official advice rests in a phenomenon called ‘risk normalisation’ – small risks become increasingly acceptable over time ( Murphy, 2020). Despite year-long warnings of the savage consequences of COVID-19, most young people had no direct experience of the misery it could cause, many will have noted the limited and sporadic deterrence offered by police, and a few of them may have come across the official reports on the minute death and serious illness rates in their cohort ( Bhopal et al., 2021). Putting such factors together anticipates and provides further explanation of the continuing struggle to sustain lockdown policies.

Mode 4: contextual heterogeneity

National policies often fail to get to grips with local complexity and this is demonstrated in the limited reach of the NPIs in the UK in respect of black and minority ethnic (BAME) communities. Prevalence, mortality, and shielding rates can be pinpointed minutely at the ‘ward’ level and these show the persistent toll of the virus on areas with high proportions of BAME residents ( Otu et al., 2020). Local, ‘soft intelligence’ identifies why these communities fared badly – e.g., collapse of the local ‘cash-in-hand’ economy, significant exposure to ‘fake news’ media, cultural misunderstandings with providers and referral systems, reliance on public transport, social distancing problems with large families in small houses, curtailment of funeral and mourning services, and so on ( West Yorkshire and Harrogate Health and Care Partnership, 2020). Broad-brush, top-down national programming can never counter such deep and locally rooted mores.

It is important to note that Spain already had a high incidence of unemployment even before the 2008 crisis. Inequality in gross income per capita is also high ( Anghel et al., 2018) and the informal economy, estimated at 17.2% of GDP in 2017, is still considered very substantial ( Deléchat & Medina, 2021). This context was nothing but conducive to further exacerbating inequalities in COVID-19. The Spanish national COVID-19 survey revealed how the most affected sectors of the population were those with lower incomes, migrant women and those who worked in the informal economy and in more precarious sectors such as cleaning, home care and nursing homes ( Carlos III Health Institute, 2020). During the first wave (March-May 2020), Barcelona neighbourhoods with lower incomes had a 42% higher incidence of infection than those with higher incomes ( Amengual-Moreno et al., 2020). The crucial point here is that well intentioned national COVID-19 policies do not just reproduce but actually intensify existing inequalities. The poorest communities are more affected by job losses as the economy shrinks ( World Health Organization, 2021). Allas et al. (2020) made an early estimate of UK unemployment risk as follows: ‘The proportion of jobs at risk in ‘elementary occupations’—which employed 3.3 million people in 2019 and include jobs such as cleaners, kitchen assistants, waiters, and bar staff—is around 44 percent. In contrast, the same number for professional occupations—such as computer programmers, project managers, and accountants—is around five percent’. What is more, the UK jobs that remain lower paid (health and social care, transport, shops) are mainly ‘public facing’ and thus carry elevated risks from the virus ( Office for National Statistics, 2020).

Spain has a welfare model that relies heavily on family responsibilities, and different generations co-habit together for longer than anywhere else in Europe. The ‘kinship solidarity model’ characteristic of Southern Europe countries is based on an asymmetrical gender division of work where women’s care is essential for the provision of welfare ( Minguez, 2017). Strict lockdown measures resulted in many of these informal support networks being reduced or suspended, undermining still further the protection afforded to poorer communities ( Amnistía Internacional, 2021).

Mode 5: ambiguity in regulations and guidelines

COVID-19 policy imposes a mass of restrictions on normal behaviour. These restrictions are delivered in the form of official communications on which activities are permitted and which are restricted. Ambiguity and inconsistency in these guidelines are inevitable, with unclear pronouncements introducing further diversity in the public response. Government announcements in both counties blurred the distinction between ‘law’ and ‘guidance’ and ‘exhortation’ in the coronavirus regulations, creating potential confusion among the public. In the UK, the key message in the original government documentation on lockdown read as follows: ‘ What you can and cannot do during the national lockdown. You must stay at home. The single most important action we can take is to stay at home to protect the NHS and save lives. You should follow this guidance immediately. This is the law’ Hickman (2021). The author goes on to point out that much of what is stated in the so-called law is actually public health advice. Many exceptions to the ‘law’ were in fact permitted - shopping for essentials; allowances for childcare bubbles; working where it is "unreasonable" to work from home; medical appointments and emergencies; moving to a new house; daily exercise, etc.

Some of the key concepts in government edicts remained inscrutable. One key task was to restrict social interactions but without forcing individuals and families into isolation. In both countries this task was made manifest in the idea that association was only allowed in ‘bubbles’. Bubbles proved as imperceptible as hot air – with whom, how, and where might they form? Schott (2020) provides a detailed study of ‘graphic confusions’ in UK Government’s COVID-19 official communications. One baffling example concerns a poster explaining the rules on meetings in which the public is permitted a choice: ‘Your household can meet up with one other household indoors or outdoors’ OR ‘You can meet up in a group of up to six people, outdoors only’. The terminological turmoil in this respect is nothing compared to the erratic Spanish rules over the Christmas period 2020 when each of the 17 autonomous community and the two autonomous cities proposed a different welcome. Regulations differed on whether ‘relatives’ or ‘close friends’ were allowed to gather. The recommended boundaries ranged from 6 in Galicia in a ‘single specified bubble’ to 10 in Ceuta in ‘any number of bubbles’ ( Radiotelevisión española, 2020b).

Once again, we glimpse an unintended consequence, inconsistency across different recommendations became a barrier to compliance. Initially in Spain there was much public consternation about the mandatory use of masks outdoors, especially in children’s playgrounds, when there was no such requirement in enclosed closed places such as bars and restaurants ( Royo-Bordonada et al., 2021). The inconsistency of messaging becomes even more problematic when restrictions are turned on and off, and then on and off again. Analysis by Dixon & Roberts (2020) showed that there were two hundred rule changes by as early as September 2020 in the UK. Whilst such tinkering was perfectly understandable it was reflected in measurable levels of public confusion ( Fancourt et al., 2020).

In the last analysis, the sheer impracticality of narrative control over micro social interaction is reflected in the use of some familiar slogans. COVID-19 restrictions extended to most walks of life and exemptions were always included. They are often so potentially compendious, however, that they have to be captured in stock caveats such ‘essential activities’, ‘reasonable excuses’, ‘where necessary’ or ‘force majeure’. Knowing exactly where to ‘draw the line’ thus becomes problematic for officials and the public. As a consequence, public pronouncements also swung gradually to emphasising the importance of ‘behaving in responsible manner’, ‘taking great care’, ‘staying alert’ and so on. Risk management was shifted subtly and arbitrarily onto the individual, with no foreknowledge of whether this new emphasis could be effective ( McVeigh & MacLachlan, 2021).

Mode 6: temporal change in public attitudes

Social interventions often generate an initial surge of enthusiasm with the introduction of innovative ideas (the novelty effect) ( Bracht & Glass, 1968). There is also some pride involved in being in at the beginning of a significant initiative (the showcasing effect) ( Chiesa & Hobbs, 2008). These sensations often dissipate over time as programme activities fade into the background (the routinisation effect) ( Karo & Kattel, 2018). As time continues, policy expectations may become tiresome or even resented (the fatigue effect) ( World Health Organization, 2020). Such a pattern is discernible in the response to COVID-19 restrictions in both countries.

In the UK, the remarkable ‘Clap for Carers’ event in which neighbours stood on their doorsteps clapping their hands and banging pots and pans every Thursday at 8pm, represented a significant, if ‘un-British’, show of public affection for health workers battling against the virus ( Manthorpe et al., 2022). Spaniards were even more spontaneous, clapping in their balconies every day at 8pm ( Cabedo-Mas et al., 2021). But slowly stamina declined, and streets and balconies became empty.

In terms of the medium and long term; there is some evidence of the routinisation effect, when people seek to push back rather than withdraw under restrictions. This process is demonstrated in the significant differences in the numbers of children claiming exemptions in order to attend schools in the UK in the two periods of formal ‘closure’. The Department for Education ( GOV.UK, 2021a) reported that 21% of primary school pupils and 5% of secondary school pupils went into school in January 2021. This compares with 4% of state primary school pupils and 1% from state secondaries who were in school during the closure in the previous year.

A stiffer measure of resistance then follows in both countries. In Spain, according to a March 2021 ISCIII COSMO survey, reported by Instituto de Salud Carlos III (2021) there was a decrease in the frequency of compliance with all preventative measures as pandemic fatigue increased. This decline was particularly discernible in the answers to motivational questions such as ‘I am losing the will to fight against COVID-19’, ‘I am tired of the debates about COVID-19’ and ‘I am tired of hearing about COVID-19’.

Polling by the Scottish Government (2021) showed that in October and November 2020, 4 out of 10 parents of under-18s admitted to adapting COVID-19 guidance to suit their family needs. One example was that 19% agreed that ‘It’s okay for my child(ren) to go into their friend’s house if I don’t go in with them’. The main reasons provided by parents for adapting the guidance were; the mental health of their children (41%), followed by applying common sense (35%), to help improve their own mental health (30%) and to allow them to work (26%).

Mode 7: exit and sustainability effects

It is relatively easy to invent policies that restrict social interaction. But not quite so easy to implement and evaluate them, as we have seen to this point. Uncertainty actually increases when it comes to deciding how and when to withdraw them.

Closing schools, shops, theatres and so on was much simpler to implement than reopening them with capacity limitations, one-way systems, sanitising points, screening and booking systems. In lifting the first lockdown, UK Government advice ( GOV.UK, 2021c) for retailers included: completing a COVID-19 risk assessment, cleaning more often, reminding customers and staff to wear face coverings, ensuring social distancing, improving ventilation, taking part in Test and Trace, turning away people with coronavirus symptoms and attending to staff mental health needs. Additionally, some establishments were expected to keep records of all visitors, to reduce capacity, to manage queues, to erect barriers and screens to protect staff. Identical requirements were placed on many establishments in Spain ( Consejo Interterritorial. Sistema Nacional de Salud, 2021).

These are arduous expectations, especially for small businesses. Very little evidence is available on how well these requirements were met. Data is still gathering on one important consequence – how many small businesses will survive the roundabout of restriction and derestriction? How many temporary small business closures will become permanent? Estimates of UK impact can be found in a Registry Trust report (2021). Big business has proved more reliant under the pandemic but even here there are casualties, one of which impacts on our case comparisons. Nissan made the decision to close its plant in Barcelona with a loss of 2,800 jobs but removed a short-term threat to the 6,700 at its Sunderland plant in the UK ( Euronews, 2020).

These evidential glimmers lead us to our final consideration on complexity. We have argued that the effectiveness of any particular measure depends on its interaction with all other measures (Mode 2). When it comes to lifting COVID-19 restrictions, political and economic interests fight against heath considerations as never before. As a summary statement, this conclusion from the UK Institute for Government report (2021b): serves well:

When and how to start lifting lockdown will present the prime minister and his cabinet with some of the toughest choices they will ever have to make … At the start of the crisis, what was good for public health was also probably in the economy’s long-term interests. As we move into the next phase there is a balance ministers will need to manage – they will be walking a tightrope between the risks of another surge of infections and lasting harm to the economy, people’s lives, livelihoods and prospects.’

Conclusion

In the introduction we posed the question - without the biological discoveries, would the social control strategies, in the long run, have struggled to succeed? We warned that it was impossible to settle such counterfactual questions, that piecing together evidence would be hazardous, but that informed conjecture on the effectiveness of NPIs would be useful in learning about their application in future crisis management.

It is not only the public that suffers pandemic fatigue but so do commentators and policymakers. Much of the preliminary evidence that we have pieced together is in danger of being forgotten. Before the advent of the vaccination programmes, very few European governments felt able to boast of the sustained reduction of the virus and the reasons for this reluctance should not be lost to history ( Goniewicz et al., 2020). Many years ago, Lindblom (1959) characterised policy making as ‘muddling through’ and, alas, this description applies all too well to the efforts to contain the virus.

The boundaries of our analysis should be made clear. Long-term lockdown (extended NPIs) may well work more effectively in countries with authoritarian governments, compliant populations, and mass surveillance systems – though accessing uncensored evidence is difficult ( Thomson & Ip, 2020). New Zealand’s famed exceptionalism, pursuing a zero COVID-19 strategy and implementing an initial stringent and brief lockdown ( Robert, 2020) also has distinctive roots: geographic isolation, easy and immediate border closure, a unitary system of government and a tiny population - the so-called ‘team of five million’ ( Baker et al., 2020). But in complex, liberal democracies, with diverse and disputatious populations, with instant and endless social interaction, with extensive worldwide interconnectedness, and with dispersed administrative systems, the broad picture is that centralised control systems generated outcomes that were partial, short-lived, and indeterminate. Modern European social life is perfectly organised in ways that multiply the microcircuits of disease transmission.

We should not have been surprised. State public policy, as McConnell and Stark (2021) advise, is often imprecise and fractured. Accordingly, the same two-steps-forward, one step-backwards pattern of progress is familiar right across the policy waterfront. Most famously, economic policy over a century has utterly failed to prevent cycles of ‘boom and bust’ ( Quinn & Turner, 2020). ‘Crime waves’ reoccur as prevention interventions falter with offenders adapting deftly to successive countermeasures, whilst fresh opportunities continue to arise ( Sacco, 2005). Studies of comparative, long-term trends in social mobility reveal little more than trendless fluctuation, the so-called ‘constant flux’ pattern ( Erickson & Goldthorpe, 1992). Many foreign aid projects are turned off on because of the ‘Samaritan’s dilemma’, with some recipients choosing it to use the support to improve their situation with others relying on it as a means of survival ( Buchanan, 1975). The very many schemes devised for managing demand for healthcare tend to meet with immediate success only to be met by ‘rebound effects’ due to patients’ steadily rising expectations and the constant advances in treatment technology ( King’s Fund, 2016). In all of these examples the intended effects dissipate and fluctuate as they are met with counteracting mechanisms akin to those analysed above. Complex systems are perfectly designed to achieve the outcomes that emerge.

We arrive at our counterfactual conclusion. Across Europe, in the absence of the vaccination programmes, NPIs would have failed to quell successive virus waves. Countless excess deaths would have resulted until infection rates became sufficient to reach the threshold of herd immunity. One cannot, of course, provide direct evidence for something that did not happen but there is a glimpse of this alternative future in UK data on infection rates that stretch into the vaccination period. Daily case rates in October 2021 reached 44,890, a figure broadly comparable with transmission at the pre-vaccination summit ( GOV.UK, 2021b). This suggests that all the social dynamics of COVID-19 transmission remained active and uncontrolled, the all-important difference being that, thanks to vaccination, death and hospitalisation rates diminished significantly from their previous peaks. And, thankfully, they continue to do so.

Despite this pessimistic conclusion, throughout Europe, official inquiries are underway seeking to learn lessons for the future of pandemic management (e.g., UK Parliament, 2021). We hope that some insights from complexity theory find their way into such post-mortems, for its insights are not automatically negative. Complexity should be regarded as a challenge rather than an adversary. To this end, we conclude with some modest recommendations of our own to improve the design and application of NPIs in epidemics and pandemics.

The first is to widen the pool of evidence considered in the manufacture of policy. Consider the status of the material collected here on the seven confounding mechanisms embedded in complex policy systems. It is clearly not the evidence that found its way into policymaking. Yet it is perfectly valid and highly reproducible evidence – the problem being that it is often dismissed as ‘critique’. We have called on all manner evidence produced from opinion surveys, from administrative records, from document analysis, from local soft intelligence, from media research, and from accounting and audit. This first call mirrors other significant pleas to construct a wider platform for evidence-based policy – c.f. Sridharan and Nakaima (2012) on the need for an ‘ecology of evidence’ and the UK Medical Research Council’s manifesto for ‘methodological pluralism’ ( Skivington et al., 2021).

The second petition is a corollary of the first. The membership of advisory bodies responsible for producing, interpreting, and advising on current evidence should also reflect the substance of the policies and interventions under development. NPIs, as their name suggests, consists entirely of social interventions and organisational reforms – and yet clinicians, modelers and epidemiologists rather than programme evaluators, implementation scientists, policy analysts dominated the ranks of senior advisors in the UK Scientific Advisory Group on Emergencies ( Thacker, 2020). By contrast, and although Spain has a Coordination Centre for Health Alerts and Emergencies and the Interterritorial Council of the National Health System that advises on public health, the Spanish Parliament (Cortes Generales) does not have any permanent legislative scientific and technological advice mechanism to inform debate and the policymaking process ( Aiello et al., 2020). We counsel on a happy medium. Broadly based expertise is the cornerstone of evidence-based policy.

A third recommendation is to ponder alternative models for the conduct of expert committees – minority reports, tribunal systems, open deliberations, adversarial courts, citizens’ assemblies and so on 1 . Practical details vary but the underlying principle is paramount: ‘Creating institutions that establish norms and expectations of legitimate disagreement as part of the process of forming and communicating expert advice would make it easier for experts to stay true to their expertise and harder for politicians to hide their judgments behind the science’.

A fourth suggestion is for increased delegation. European COVID-19 policy making has largely preferred evidence with a macro focus, eying the dashboards for telling perturbations. This inclination generated the stuttering stream of national restrictions, described above, based on overzealous extrapolations of shifts in aggregate data. There should be more emphasis on the micro-circuits of transmission ( Manzo, 2020), seeking and targeting continuous quality improvement within the many sub-processes, logistics and agencies that embody the everyday response to the virus. Total control of a complex system is impossible but there was considerable local learning on infection control in hospitals, care homes, local communities, policing etc. NPIs sought to modify everyday behaviour. Narrative and exemplars can be better than rules and regulations at inspiring change in commonplace routines ( Greenhalgh, 2020).

A final important plea, is for improved institutional memory. It is true but somewhat oversimplistic to assume, that the vital lessons about COVID-19 policy could have been assimilated from knowledge of previous epidemics ( Tsuei, 2020). But there is at least a case for saying that mistakes can always be avoided by not repeating them. We pointed earlier in the paper to the damage that prominent ‘free riders’ can inflict on public confidence in virus control leadership. Astonishingly, in the UK case, this lesson was immediately forgotten as the aforementioned ‘Cummings effect’ was then followed by the ‘Partygate’ scandal ( Foreign Policy, 2022). A similar affront can be observed in the Spanish context. In the same week that the seventh report of the Rafael Campalans Foundation was released, reflecting the disastrous management of the pandemic, especially in nursing homes ( Precedo, 2022), the scandal of the commissions received by the brother of the President of the Community of Madrid has reinforced public displeasure ( Rubio Hancock, 2022). In the last analysis, pandemic policy is in the hands of political leaders and unforeseen consequences always lurk. Lindblom (1959) was correct in assuming that policy making in complex environments always involves ‘muddling through’. But if we read him carefully and focus attention on bespoke evidence on sub-process and local conditions, he was also correct to insist that there can also be a ‘science of muddling through’.

Data availability

All data underlying the results are available as part of the article and no additional source data are required.

Ethics and consent

Ethical approval and consent were not required.

Funding Statement

This research was financially supported by the European Union’s Horizon 2020 research and innovation programme under the grant agreement no. 894303 (Interventions on Social Determinants of Health: a comparative study of returned migrants from the UK and Spain post-Brexit referendum [RETORNO]).

[version 2; peer review: 2 approved]

Footnotes

1 For discussion of these we direct the reader to a paper by Moore & MacKenzie (2020).

References

  1. Aiello E, Melchor L, Oliver E: Spain needs a legislative science advice mechanism. Evidence and Policy.2020. Reference Source [Google Scholar]
  2. Allas T, Canal M, Hunt V: COVID-19 in the United Kingdom: Assessing jobs at risk and the impact on people and places. McKinsey and Company.2020. Reference Source [Google Scholar]
  3. Alonso N, Larraz I: La capacidad de rastreo cae a la mitad en algunas comunidades. Newtral. 2021. Reference Source [Google Scholar]
  4. Amengual-Moreno M, Calafat-Caules M, Carot A, et al. : Determinantes sociales de la incidencia de la Covid-19 en Barcelona: un estudio ecológico preliminar usando datos públicos. Rev Esp Salud Pública. 2020;94:e202009101. Reference Source [PMC free article] [PubMed] [Google Scholar]
  5. Amnistía Internacional España: La otra pandemia - Entre el abandono y el desmantelamiento: El derecho a la salud y la atención primaria en España.2021;89. Reference Source [Google Scholar]
  6. Anghel B, Basso H, Bover O, et al. : Income, consumption and wealth inequality in Spain. SERIEs - Journal of the Spanish Economic Association. 2018;9(4):351–387. 10.1007/s13209-018-0185-1 [DOI] [Google Scholar]
  7. Baker MG, Kvalsvig A, Verrall AJ: New Zealand’s COVID‐19 elimination strategy. Med J Aust. 2020;213(5):198–200.e1. 10.5694/mja2.50735 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Benoit SL, Mauldin RF: The “anti-vax” movement: a quantitative report on vaccine beliefs and knowledge across social media. BMC Public Health. 2021;21(1):2106. 10.1186/s12889-021-12114-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bhopal SS, Bagaria J, Olabi B, et al. : Children and young people remain at low risk of COVID-19 mortality. Lancet Child Adolesc Health. 2021;5(5):e12–e13. 10.1016/S2352-4642(21)00066-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Booth A, Wright J, Briscoe S: Scoping and searching to support realist approaches. In: N Emmel et al. Doing Realist Research. London: Sage,2018. 10.4135/9781526451729.n10 [DOI] [Google Scholar]
  11. Bracht GH, Glass GV: The external validity of experiments. Am Educ Res J. 1968;5(4):437–474. 10.2307/1161993 [DOI] [Google Scholar]
  12. Braithwaite J, Churruca K, Long JC, et al. : When complexity science meets implementation science: A theoretical and empirical analysis of systems change. BMC Med. 2018;16(1):63. 10.1186/s12916-018-1057-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Buchanan J: The Samaritan’s dilemma. In: E.S. Phelps (ed.). Altruism, morality and economic theory. Russel Sage Foundation.1975. 10.7758/9781610446792 [DOI] [Google Scholar]
  14. Cabedo-Mas A, Arriaga-Sanz C, Moliner-Miravet L: Uses and perceptions of music in times of COVID-19: a Spanish population survey. Front Psychol. 2021;11:606180. 10.3389/fpsyg.2020.606180 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Centro de Investigaciones Sociológicas: Barómetro especial de abril 2020. (Estudio n o 3279.).2020. Reference Source [Google Scholar]
  16. Centro de Investigaciones Sociológicas: Barómetro de febrero 2021 (Estudio n o 3309).2021. Reference Source [Google Scholar]
  17. Chiesa M, Hobbs S: Making sense of social research: how useful is the Hawthorne Effect? Eur J Soc Psychol. 2008;38(1):67–74. 10.1002/ejsp.401 [DOI] [Google Scholar]
  18. Consejo Interterritorial. Sistema Nacional de Salud: Actuaciones de respuesta coordinada para el control de la transmisión de COVID-19.2021. [Google Scholar]
  19. Costa-Font J, Jiménez Martin S, Viola A: Fatal Underfunding? Explaining COVID-19 Mortality in Spanish Nursing Homes. J Aging Health. 2021;33(7–8):607–617. 10.1177/08982643211003794 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Daviter F: Policy analysis in the face of complexity: What kind of knowledge to tackle wicked problems? Public Policy Adm. 2019;34(1):62–83. 10.1177/0952076717733325 [DOI] [Google Scholar]
  21. Deléchat C, Medina L: The global informal workforce: Priorities for inclusive growth. International Monetary Fund.2021. Reference Source [Google Scholar]
  22. De Vega L: Rastreadores desbordados: Falla la detección precoz. El País.2020. Reference Source [Google Scholar]
  23. Dixon H, Roberts L: Does anyone know what it means?’ How coronavirus rules have changed 200 times. The Telegraph.2020. Reference Source [Google Scholar]
  24. Dodds K, Broto VC, Detterbeck K, et al. : The COVID-19 pandemic: territorial, political and governance dimensions of the crisis. Territ Politic Gov. 2020;8(3):289–298. 10.1080/21622671.2020.1771022 [DOI] [Google Scholar]
  25. Erickson R, Goldthorpe J: The Constant Flux: A Study of Class Mobility in Industrial Societies. Clarendon Press.1992. Reference Source [Google Scholar]
  26. Euronews: Nissan shuts Barcelona plant as COVID-19 drives car giant into the red. Euronews.2020. [Google Scholar]
  27. Evans R: Altered Pasts: Counterfactuals in History. Little Brown.2014. Reference Source [Google Scholar]
  28. Fancourt D, Steptoe A, Wright L: The Cummings effect: politics, trust, and behaviours during the COVID-19 pandemic. Lancet. 2020;396(10249):464–465. 10.1016/S0140-6736(20)31690-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Financial Times: The new wartime economy in the era of coronavirus.2020. Reference Source [Google Scholar]
  30. Foreign Policy: Is the party over for Boris?2022. Reference Source [Google Scholar]
  31. Goniewicz K, Khorram-Manesh A, Hertelendy AJ, et al. : Current response and management decisions of the European Union to the COVID-19 outbreak: a review. Sustainability. 2020;12(9):3838. 10.3390/su12093838 [DOI] [Google Scholar]
  32. GOV.UK: COVID-19: Number of outbreaks in care homes.2020. Reference Source [Google Scholar]
  33. GOV.UK: Attendance in education and early years settings during the coronavirus (COVID-19) pandemic. Week 3 2021.2021a. Reference Source [Google Scholar]
  34. GOV.UK: Cases in the UK | Coronavirus in the UK.2021b. Reference Source [Google Scholar]
  35. GOV.UK: Working Safely During Coronavirus. (Support for Businesses and Self-Employed People during Coronavirus).2021c. Reference Source [Google Scholar]
  36. GOV.UK: Monitoring reports of the effectiveness of COVID-19 vaccination.2021d. Reference Source [Google Scholar]
  37. Grabmeier J: More than 87,000 scientific papers on coronavirus since pandemic. Ohio State News. The Ohio State University.2021. Reference Source [Google Scholar]
  38. Greenhalgh T: Will Evidence-Based Medicine Survive COVID-19? Boston Review. 2020. Reference Source [Google Scholar]
  39. Health Foundation: What the public thinks – another five things we learnt from our COVID-19 polling.2020. Reference Source [Google Scholar]
  40. Heneghan C, Howdon D, Oke J, et al. : The Ongoing Problem of UK Hospital Acquired Infections. The Centre for Evidence-Based Medicine.2020. Reference Source [Google Scholar]
  41. Hickman T: The continuing misuse of guidance in response to the pandemic. LSE COVID-19. 2021. Reference Source [Google Scholar]
  42. Institute for Government: Coronavirus lockdown rules in each part of the UK.2021a. Reference Source [Google Scholar]
  43. Institute for Government: Lifting the Lockdown. 2021b. Reference Source [Google Scholar]
  44. Instituto de Salud Carlos III: Estudio ene-covid: Cuarta ronda estudio nacional de sero-epidemiología de la infección por sars-cov-2 en España.2020. Reference Source [Google Scholar]
  45. Instituto de Salud Carlos III: Monitorización del comportamiento y las actitudes de la población relacionadas con la COVID-19 en España (COSMO-SPAIN): Estudio OMS.2021. Reference Source [Google Scholar]
  46. Instituto de Salud Carlos III, Centro Nacional de Epidemiología: “Situación de COVID-19 en España a 26 de mayo de 2021” (No. 80). 2021. Reference Source [Google Scholar]
  47. James LP, Salomon JA, Buckee CO, et al. : The use and misuse of mathematical modeling for infectious disease Policymaking: lessons for the COVID-19 pandemic. Med Decis Making. 2021;41(4):379–385. 10.1177/0272989X21990391 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. John Hopkins University and Medicine: COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU).2021. Reference Source [Google Scholar]
  49. Karo E, Kattel R: Innovation and the state: towards an evolutionary theory of policy capacity. In: Policy Capacity and Governance. Palgrave Macmillan, Cham.2018;123–150. 10.1007/978-3-319-54675-9_6 [DOI] [Google Scholar]
  50. Keeling MJ, Danon L: Mathematical modelling of infectious diseases. Br Med Bull. 2009;92(1):33–42. 10.1093/bmb/ldp038 [DOI] [PubMed] [Google Scholar]
  51. King’s Fund: How hospital activity in the NHS in England has changed over time.The King’s Fund.2016. Reference Source [Google Scholar]
  52. Krishtel P, Hassan F: Share vaccine know-how. Science. 2021;374(6566):379. 10.1126/science.abm8724 [DOI] [PubMed] [Google Scholar]
  53. Lindblom CE: The Science of Muddling Through. Public Administration Review. 1959;19(2):79–88. 10.2307/973677 [DOI] [Google Scholar]
  54. Lledó C: ¿Se culpabiliza injustamente a los jóvenes de la transmisión del COVID-19?La Voz de Asturias,2020. Reference Source [Google Scholar]
  55. Macmillan Cancer Support: The Forgotten ‘C’? The impact of COVID-19 on cancer care.2020. Reference Source [Google Scholar]
  56. Manchado S: El caso de Extremadura o cómo pasar en 43 días de tener la incidencia más alta de España a ser la primera comunidad con riesgo bajo.ElDiario.Es.2021. Reference Source [Google Scholar]
  57. Manthorpe J, Iliffe S, Gillen P, et al. : Clapping for carers in the Covid-19 crisis: Carers' reflections in a UK survey. Health Soc Care Community. 2022;30(4):1442–1449. 10.1111/hsc.13474 [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Manzo G: Complex Social Networks are Missing in the Dominant COVID-19 Epidemic Models. Sociologica. 2020;14(1):31–49. 10.6092/issn.1971-8853/10839 [DOI] [Google Scholar]
  59. McConnell A, Stark A: Understanding policy responses to COVID-19: The stars haven’t fallen from the sky for scholars of public policy. J Eur Public Policy. 2021;28(8):1115–1130. 10.1080/13501763.2021.1942518 [DOI] [Google Scholar]
  60. McVeigh J, MacLachlan M: Psychological Governance and COVID-19: A Case Study in Macropsychology.In: M. MacLachlan & J. McVeigh (Eds.), Macropsychology: A Population Science for Sustainable Development Goals.Springer International Publishing.2021;303–316. 10.1007/978-3-030-50176-1_14 [DOI] [Google Scholar]
  61. Mendez-Brito A, El Bcheraoui C, Pozo-Martin F: Systematic review of empirical studies comparing the effectiveness of non-pharmaceutical interventions against COVID-19. J Infect. 2021;83(3):281–293. 10.1016/j.jinf.2021.06.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Minguez AM, Crespi I: Gender equality and family changes in the work-family culture in Southern Europe. International Review of Sociology. 2017;27(3):394–420. Reference Source [Google Scholar]
  63. Moore A, MacKenzie M: Policy making during crises: how diversity and disagreement can help manage the politics of expert advice. BMJ. 2020;371:m4039. 10.1136/bmj.m4039 [DOI] [PubMed] [Google Scholar]
  64. Murphy V: Coronavirus is no longer seen as high risk by many - and that’s undermining control measures.2020. Reference Source [Google Scholar]
  65. National Audit Office: The government’s approach to test and trace in England - interim report.2020. Reference Source [Google Scholar]
  66. Nature: News feature: Six months of COVID vaccines: what 1.7 billion doses have taught scientists.2021. 10.1038/d41586-021-01505-x [DOI] [PubMed] [Google Scholar]
  67. Office for National Statistics: Which occupations have the highest potential exposure to the coronavirus (COVID-19)? 2020. Reference Source [Google Scholar]
  68. Otu A, Ahinkorah BO, Ameyaw EK, et al. : One country, two crises: What Covid-19 reveals about health inequalities among BAME communities in the United Kingdom and the sustainability of its health system? Int J Equity Health. 2020;19(1):189. 10.1186/s12939-020-01307-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Pawson R: Evidence-based Policy: A realist Perspective.London: Sage.2006. 10.1080/15588740802262039 [DOI] [Google Scholar]
  70. Pawson R: The coronavirus response: boxed in by models. Evaluation. 2021;27(2):149–167. 10.1177/1356389020968579 [DOI] [Google Scholar]
  71. Precedo A: El SOS enmudecido’ de las residencias: Un informe concluye que la respuesta al covid fue ‘tardía e ineficaz. infoLibre.2022. Reference Source [Google Scholar]
  72. Quinn W, Turner JD: Boom and Bust: A Global History of Financial Bubbles.University Press,2020. 10.1017/9781108367677 [DOI] [Google Scholar]
  73. Radiotelevisión española: Ayuso achaca los contagios en el sur de Madrid al ‘modo de vida de la inmigración’ y la densidad de población.2020a. Reference Source [Google Scholar]
  74. Radiotelevisión española: Coronavirus Las comunidades autónomas endurecen las restricciones para Nochevieja y Año Nuevo.RTVE.Es.2020b. Reference Source [Google Scholar]
  75. Registry Trust Ltd: The economic impact of the pandemic on SMEs.2021. Reference Source [Google Scholar]
  76. Reicher S, Drury J: Pandemic fatigue? How adherence to covid-19 regulations has been misrepresented and why it matters. BMJ. 2021;372:n137. 10.1136/bmj.n137 [DOI] [PubMed] [Google Scholar]
  77. Rittel HW, Webber MM: Wicked problems. Man-made Futures. 1974;26(1):272–280. [Google Scholar]
  78. Robert A: Lessons from New Zealand's COVID-19 outbreak response. Lancet Public Health. 2020;5(11):e569–e570. 10.1016/S2468-2667(20)30237-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  79. Royo-Bordonada MA, García-López FJ, Cortés F, et al. : Face masks in the general healthy population. Scientific and ethical issues. Gac Sanit. 2021;35(6):580–584. 10.1016/j.gaceta.2020.08.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  80. Rubio Hancock J: Como el Watergate, pero dibujado por Ibáñez.El País.2022. Reference Source [Google Scholar]
  81. Sacco V: When crime waves.Sage: London,2005. 10.4135/9781483328799 [DOI] [Google Scholar]
  82. Sasse T, Haddon C, Nice A: Science advice in a crisis.The Institute for Government.2020. Reference Source [Google Scholar]
  83. Saul JE, Willis CD, Bitz J, et al. : A time-responsive tool for informing policy making: rapid realist review. Implement Sci. 2013;8:103. 10.1186/1748-5908-8-103 [DOI] [PMC free article] [PubMed] [Google Scholar]
  84. Schott B: Britain’s Covid Response Is Graphic Confusion.Bloomberg.Com.2020. Reference Source [Google Scholar]
  85. Scottish Government: Working Paper: Covid-19 Mitigation Measures Among Children and Young People. Coronavirus in Scotland. Publications-Research and Analysis.2021. Reference Source [Google Scholar]
  86. Skivington K, Matthews L, Simpson SA, et al. : A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021;374:n2061. 10.1136/bmj.n2061 [DOI] [PMC free article] [PubMed] [Google Scholar]
  87. Sridharan S, Nakaima A: Towards an evidence base of theory-driven evaluations: Some questions for proponents of theory-driven evaluation. Evaluation. 2012;18(3):378–395. 10.1177/1356389012453289 [DOI] [Google Scholar]
  88. Sutton A, Clowe M, Preston L, et al. : Meeting the review family: exploring review types and associated information retrieval requirements. Health Info Libr J. 2019;36(3):202–222. 10.1111/hir.12276 [DOI] [PubMed] [Google Scholar]
  89. Talic S, Shah S, Wild H, et al. : Effectiveness of public health measures in reducing the incidence of covid-19, SARS-CoV-2 transmission, and covid-19 mortality: systematic review and meta-analysis. BMJ. 2021;375:n2997. 10.1136/bmj.n2997 [DOI] [PMC free article] [PubMed] [Google Scholar]
  90. Thacker PD: Conflicts of interest among the UK government's COVID-19 advisers. BMJ. 2020;371:m4716. 10.1136/bmj.m4716 [DOI] [PubMed] [Google Scholar]
  91. Thomson S, Ip EC: COVID-19 emergency measures and the impending authoritarian pandemic. J Law Biosci. 2020;7(1):lsaa064. 10.1093/jlb/lsaa064 [DOI] [PMC free article] [PubMed] [Google Scholar]
  92. Tsuei SHT: How previous epidemics enable timelier COVID-19 responses: an empirical study using organisational memory theory. BMJ Glob Health. 2020;5(9):e003228. 10.1136/bmjgh-2020-003228 [DOI] [PMC free article] [PubMed] [Google Scholar]
  93. Triggle N, Schraer R, Kemp P: Coronavirus: Inside NHS Test and Trace - how the 'world beater' went wrong.BBC News, 20th November2020. Reference Source [Google Scholar]
  94. UK Parliament: Health and Social Care, and Science and Technology Committees.2021. Reference Source [Google Scholar]
  95. Valdeolivas R: Sin mascarillas y a lo loco: La fiesta más viral de El Español.InfoLibre.2020. Reference Source [Google Scholar]
  96. West Yorkshire and Harrogate Health and Care Partnership: Intelligence and Insight Pack.2020. Reference Source [Google Scholar]
  97. Williams M: Realism and Complexity in Social Science.Routledge,2021. Reference Source [Google Scholar]
  98. Wong G, Greenhalgh T, Westhorp G, et al. : RAMESES publication standards: realist syntheses. BMC Med. 2013;11(1):21. 10.1186/1741-7015-11-21 [DOI] [PMC free article] [PubMed] [Google Scholar]
  99. World Health Organization: Pandemic fatigue: reinvigorating the public to prevent COVID-19: policy framework for supporting pandemic prevention and management: revised version November 2020 (No. WHO/EURO: 2020-1573-41324-56242).World Health Organization. Regional Office for Europe.2020. [Google Scholar]
  100. World Health Organization: COVID-19 and the social determinants of health and health equity: Evidence brief.2021. Reference Source [Google Scholar]
  101. World Health Organization: Coronavirus (COVID-19) Dashboard.2022. Reference Source [Google Scholar]
  102. Zhou G, Chen S, Chen Z: Back to the spring of 2020: facts and hope of COVID-19 outbreak. Front Med. 2020;14(2):113–116. 10.1007/s11684-020-0758-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  103. Zunzunegui V: El exceso de mortalidad por covid-19 en las personas mayores que viven en las residencias de España: Variaciones entre comunidades autónomas.EnR? Envejecimiento en red.2021. Reference Source [Google Scholar]
Open Res Eur. 2022 Sep 5. doi: 10.21956/openreseurope.16244.r29984

Reviewer response for version 2

Axel Kaehne 1

Many thanks to the authors for their thoughtful response to my comments and suggestions on their first version of this extremely insightful and thought provoking paper. Their amendments are appropriate and welcome and their critical response to my suggestions indicate that there is much more common ground between our views in these matters than differences. I note the helpful additional literature they cited which provides a useful context and further support to the argument they put forward. 

I am convinced this paper makes an important contribution to the ongoing debate about the impact of non-pharma interventions in combating the COVID outbreak.

Approved with no reservations.

Is the argument persuasive and supported by appropriate evidence?

Yes

Is the work clearly and cogently presented?

Yes

Is the topic of the essay discussed accurately in the context of the current literature?

Yes

Does the essay contribute to the cultural, historical, social understanding of the field?

Yes

Reviewer Expertise:

Health Services research, Programme Evaluations, Implementation Science

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Open Res Eur. 2022 Jul 18. doi: 10.21956/openreseurope.15727.r29149

Reviewer response for version 1

Frans Leeuw 1

The topic of the study is extremely important: an unparallelled programme of public restrictions, big public finance investments in helping companies, even implementing interventions that even in the year before the first outbreak (2019) were, by the WHO, not considered feasible or important (like lockdowns), mass social protests etc. The realist evaluation approach to this large public challenge has been done carefully, to the point and with a certain distance in perspective. The structure of the paper is excellent. The lessons that are suggested have been underpinned as good as it could be done, but of course time will tell what the results in reality in the future may/ will be.

Is the argument persuasive and supported by appropriate evidence?

Yes

Is the work clearly and cogently presented?

Yes

Is the topic of the essay discussed accurately in the context of the current literature?

Yes

Does the essay contribute to the cultural, historical, social understanding of the field?

Yes

Reviewer Expertise:

evaluation research, behavioral and social studies

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

Open Res Eur. 2022 Aug 5.
Pilar SERRANO-GALLARDO 1

We would like to thank Dr Leeuw for his review and comments on our work. As Dr Leeuw points out, we hope that the complex lessons we propose can be taken into account in ongoing research on pandemic management across Europe and other regions of the world.

Open Res Eur. 2022 May 23. doi: 10.21956/openreseurope.15727.r29151

Reviewer response for version 1

Axel Kaehne 1

This is a fascinating and deeply thought provoking paper examining the effectiveness of the social control measures during the COVID pandemic. The narrative mainly uses evidence from two countries, the UK and Spain and argues that non-pharmaceutical interventions to reduce COVID infections in the population were largely ineffective. In addition, the authors set out in detail why they think measures such as 'lock-downs' were likely to fail in achieving lower infection rates. They utilise a (loosely defined) realist interpretative framework aligned with the 7 different modes of complexity theory. The theory is clearly laid out and the structure of the paper is also clear detailing their response for each mode in turn.

Whilst there is generally little in the paper that I would disagree there are a couple of questions that may, if asked, reveal alternative perspectives which, turn may help widen pool of possible explanations again. 

I will set out some of these below. The authors may find it useful to address/add comments on each of these in the appropriate places their paper.

1) Methods: the authors utilise a realist review approach (page 5) but there is less clarity on how this was done in practice. I think it would be useful for the authors to sketch their realist review approach in a brief separate section to allow readers to gain a better understanding of how the conclusions for each mode were reached. 

2) There are some statements in the paper where findings have perhaps been a bit overinterpreted or interpretative frames of references have shifted imperceptibly. Please consider rephrasing to ensure that initial theses/arguments are not expanded later in the text; example: page 5, paragraph 1: 2) 'control was never achieved...' I think the diagram is not showing that control was never achieved. The Belgium graph clearly shows a different picture and the Austria graph does appear so too. I would suggest to limit the statement to UK and Spain which are also the focus of the subsequent sections (and for which this statement holds true) and reconsider the temporal qualification 'never'.

3) Some statements appear to make political points rather than contribute to the analysis. On page 7, last paragraph, the authors talk about 'transfer of resources' and its impact on health systems. Whilst I agree with the observation, I am not sure this is relevant within the focus of their paper since the aim is to elucidate the failure of social restrictive measures on infection rates. 

4) page 11, third paragraph, second column: the authors put forward an argument about the possibly higher effectiveness of micro-circuits of transmission. However, there is little in their discussion of the seven modes that would not apply to micro-circuits (localised control measures). I am not convinced that the size/footprint is a significant factor with regard to those features of social control measures they discussed. And if it was I would like to see this emerge from their realist review. Aspects of complexity would in my opinion apply to micro-circuits just as they do for macro-circuits. In addition, I am not sure their argument in the seven modes points much to an increased effectiveness of social control measures in micro-circuits, and there may be plenty of counter-examples if one were to look closely perhaps (for example German Federal States with local regulations)?

5) The main thrust of the argument is that the modelling provides poor guidance to effective policy making in this field as static modelling is unable to incorporate feedback loops etc. Yet there is also the possibility to improve modelling as well. The authors should consider mentioning this an option too in their recommendations, starting with localised data aligned with localised COVID responses where this happened. This may still possibly reveal some effectiveness of some social control measures.

Clearly this paper makes an important contribution to the debate and I am looking forward to see a more methodological explanation in future papers perhaps about how we can use realist review in linking policy solutions to complex social interventions such as social restrictions during pandemics.

Is the argument persuasive and supported by appropriate evidence?

Yes

Is the work clearly and cogently presented?

Yes

Is the topic of the essay discussed accurately in the context of the current literature?

Yes

Does the essay contribute to the cultural, historical, social understanding of the field?

Yes

Reviewer Expertise:

Health Services research, Programme Evaluations, Implementation Science

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Open Res Eur. 2022 Aug 5.
Pilar SERRANO-GALLARDO 1

The authors would like to thank Dr Kaehne for the excellence and diligence of their reports; and especially, for spending a considerable amount of their own time in drafting such detailed responses. Dr. Kaehne finds ‘little to disagree with’ in our paper but seeks further clarification on several points. We thus structure our rejoinder around his five queries.

1. As requested, we have added a new coda to the original draft, containing some further explanation and key references to the methodology of rapid realist review. We regard Dr. Kaehne as a ‘critical friend’ of realist methodology, given that he has produced searching commentaries of some of Pawson’s previous writing (Kaehne, 2018; Kaehne, 2021). He shares our Popperian instincts that all empirical research is fallible, and that truth is only obtained by the systematic cross-examination of rival theories. He is the editor of the prestigious Journal of Integrated Care and thus has vast experience in the evaluation of complex interventions. We note, moreover, that interorganizational collaboration in healthcare is ‘complexity epitomised’. Integrated care is thus a topic that is highly conducive and quite familiar with realist review, as represented for instance in papers by Aunger et al. (2021a; 2021b). Having said all this, we disagree with him on the following points.

2. Kaehne thinks we should be more cautious about the extent of our claims on the failure to control the pandemic by means of ‘lockdown’. We heed the point linguistically and have amended a key point in the text (p. 5) to read that ‘management of the virus was never fully achieved through social controls.’ We continue to maintain, however, that real suppression only occurred under the vaccination programme. It is worth looking more closely at the data which underpins the claim about the limitations of lockdown policy. We reproduced a BBC graph which shows the rolling averages of cases in 16 countries. This provides a convenient bird’s eye view and importantly includes evidence on our case studies of the UK and Spain.

Note that these graphs are reproduced from a much larger international data set from John Hopkins University. Alas this was far too large to reproduce in the ORE paper and we maintain that close scrutiny of the big picture provides yet more evidence for our pessimism about COVID social controls. It is also worth noting the precise scope of our proposition. Our claim is that incidence rates fluctuate despite controls. They are irregular, turbulent, disorderly. We are not claiming that there is one profile, that all countries follow the same pathways, that all time-series have the same contours. So indeed, the Belgium and Austria graphs are not the same as those for the UK and Spain. But neither is the Spanish graph remotely the same as the one for the UK. What they have in common is that they are in flux, progress is unpredictable (more on this in point 5). We completely disagree, therefore, that we should limit the scope of our analytic claims to the UK and Spain. We published in ORE in the expectation that readership from a wide range of European countries would recognise the erratic play of the same policy complexities. But the best expression of the actual scope of our analysis is already in the paper’s conclusion ... ‘in complex, liberal democracies, with diverse and disputatious populations, with instant and endless social interaction, with extensive worldwide interconnectedness, and with dispersed administrative systems, the broad picture is that centralised control systems generated outcomes that were partial, short-lived, and indeterminate’.

3. We totally agree with Kaehne’s strictures about keeping political points out of scholarly analysis (Pawson, 2013). We therefore plead ‘not guilty’ to trading in ‘cheap shots’. We see nothing contentious about the claim that the pandemic often involved the transfer of resources from other healthcare provision. We even note that ‘it is not a given’.

4. Kaehne doubts the veracity of our suggestion that pandemic social control measures might be improved if informed by local knowledge of how people react to controls; and implemented by people who are respected in those communities. Once again, we are hardly the first to make this suggestion about the significance of local public health. See the Lancet editorial (2017) and associated papers for important details on how such community interventions have been successfully implemented. Kaehne is obviously right to maintain that such a change of scale is no panacea and that local systems have their own dogged complexities. We are thus content that our paper makes a mere first step in this thesis, namely that most of the COVID control measures consisted of top-down commands that often went unheard, unheeded, and ignored at the local level. Nowadays, people and communities absorb information in completely novel ways. What goes on in these micro-circuits determines the actual level of compliance.

5. Our paper is critical of the use of epidemiological models in predicting the likely advance of the pandemic. We argue that despite its significant presence in key decision centres, mathematical biology provides a poor guide to effective policy making. More recently, other authors have extended the same argument (Corteel, 2022; Greenhalgh and Engebretsen, 2022 ). Kaehne suggests that our conclusions should be moderated, including the possibility that modelling itself could be improved by incorporating ideas from complexity theory on feedback loops, non-linearity, and so on. This debate takes us back to the famous quotation attributed the most famous statistical modeller, George Box: "All models are wrong, but some are useful". Kaehne and I debated this some time ago (Pawson, 2021; Kaehne, 2021). Any influence shaping the contours of a disease can potentially be included as a parameter in a simulation model. With Box we can thus say that it is impossible to predict every possible influence in pandemic systems, but ‘useful’ models are the ones which capture an optimal mix of causal influences. The problem here is that this optimal mix cannot be known in advance. Complex systems are essentially unpredictable, they are composed of and defined by unacknowledged conditions and unintended consequences (Livesay, 1989). Crisis policies are made without knowledge of the full set of circumstances that created the said crisis, and the ensuing policies go onto create conditions that are unintended by their creators.

To be sure, the presence and significance of the unacknowledged and the unintended can be researched, but alas – only retrospectively. It is useful to pursue this argument around a specific example. Hypothetically, it would be possible to include a parameter in a simulation model that predicted the ‘free-rider problem’, and projected the potential negative impact on public opinion, and on rule-following of the activities of prominent rule-breakers. But how would one ‘estimate’ this influence? In the UK case, there is sound evidence cited in the paper that the activities of one such free rider, Dominic Cummings, a prominent government advisor, had a significant negative impact on compliance. Why? It is a long story, but one might summarise it retrospectively by saying that he was a Machiavellian character with undue influence on the PM, who made prepositions public excuses for a preposterously long private car journey when ‘lockdown’ was at its height. The devil was all in the detail. Another free-rider may get off scot-free. A multitude of capricious, sporadic ‘events’ of this ilk shaped the dwindling support for the government restrictions. But it would take a soothsayer rather than a modeller to foresee them and estimate their impact. Hundreds of predictive models were produced throughout the world during the pandemic.  We know, thanks to Box, that they were all ‘wrong’. But it remains important to test the thesis that they actually proved ‘useful’ in managing the response (though this task is, of course, a question that is way beyond the scope of our review). To what extent did modellers sit around the decision-making tables? To what extent did models feed into policy? To what extent did they improve policy? These are vital questions for further research (and the next pandemic).

We conclude by thanking Dr Kaehne for their wise words. Debate on all of these matters will not be settled in a single exchange. Public inquires on the handling of the pandemic are about to be launched across Europe. Let us hope that they are prepared to learn complex lessons.

Additional references.

Aunger, J. R. Millar, and J. Greenhalgh. (2021a). When trust, confidence, and faith collide: refining a realist theory of how and why inter-organisational collaborations in healthcare work. BMC Health Services Research. 21:602. https://doi.org/10.1186/s12913-021-06630-x  

Aunger, J., R. Millar, J. Greenhalgh, R. Mannion, A. Raffety and H. McLeod. (2021b). Why do some inter-organisational collaborations in healthcare work when others do not? A realist review. Systematic Reviews. 10:8 https://doi.org/10.1186/s13643-021-01630-8  

Corteel, M. (2022). Les boîtes noires de la pandémie: Modélisation épidémiologique et pouvoir politique durant la crise de Covid-19.  Multitudes. 86:147.-55.  https://doi.org/10.3917/mult.086.0147

Greenhalgh, T. and  E. Engebretsen. (2022). The science-policy relationship in times of crisis: An urgent call for a pragmatist turn. Social Science and Medicine, 306:115140. https://doi.org/10.1016/j.socscimed.2022.115140

Kaehne, A .(2018) What historiography can tell us about evidence-based medicine and policy: A response to Pawson’s ‘The Realist Foundations of Evidence-based Medicine’. Evaluation. 24(3): 318-24. https://doi.org/10.1177/1356389018781356

Kaehne, A. (2021) From abstract to ideal–The limits of models. A reply to Pawson’s ‘Boxed in by models’. Evaluation. 27 (3): 400-06. https://doi.org/10.1177/13563890211007505 Lancet Editorial. (2017) Public health is local. The Lancet.2(9): e387. https://doi.org/10.1016/S2468-2667(17)30170-6

Livesay, J. (1989) Structuration theory and the unacknowledged conditions of action, Theory, Culture & Society, 6(2):263–292. https://doi: 10.1177/026327689006002005

Pawson, R. (2013) The Science of Evaluation. London: Sage.

Pawson, R. (2021) The coronavirus response: Boxed in by models. Evaluation. 27(2):149-67. https://doi.org/10.1177/1356389020968579

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Data Availability Statement

    All data underlying the results are available as part of the article and no additional source data are required.


    Articles from Open Research Europe are provided here courtesy of European Commission, Directorate General for Research and Innovation

    RESOURCES