Establishing an SEIR-based framework for local modelling of COVID-19 infections, hospitalisations and deaths

ABSTRACT

Without timely assessments of the number of COVID-19 cases requiring hospitalisation, healthcare providers will struggle to ensure an appropriate number of beds are made available. Too few could cause excess deaths while too many could result in additional waits for elective treatment. As well as supporting capacity considerations, reliably projecting future “waves” is important to inform the nature, timing and magnitude of any localised restrictions to reduce transmission. In making the case for locally owned and locally configurable models, this paper details the approach taken by one major healthcare system in founding a multi-disciplinary “Scenario Review Working Group”, comprising commissioners, public health officials and academic epidemiologists. The role of this group, which met weekly during the pandemic, was to define and maintain an evolving library of plausible scenarios to underpin projections obtained through an SEIR-based compartmental model. Outputs have informed decision-making at the system’s major incident Bronze, Silver and Gold Commands. This paper presents illustrated examples of use and offers practical considerations for other healthcare systems that may benefit from such a framework.

1. Background

Infectious diseases can provide a multitude of challenges for health authorities. With uncertainty in demand for treatment and the efficacy of any consequential mitigation, there can be considerable difficulty in the effective planning of future services for both infected and uninfected sections of the population. For many healthcare systems, these challenges are perhaps best represented by seasonal influenza; peaks of which are largely unpredictable in terms of their magnitude and specific timing (Petrova & Russell, 2018). Such uncertainty limits the ability of healthcare providers to accurately assess the need to temporarily procure any additional “escalation” beds or to make corresponding reductions to elective treatments in order to increase available capacity (P. J. Birrell et al., 2020).

With the novelty and virulence of Coronavirus disease 2019 (COVID-19), declared a pandemic on 11 March 2020 (World Health Organisation, 2020), these problems are compounded. High levels of incidence are not confined solely to winter months, with peaks in transmission evident at all seasons of the year (Carlson et al., 2020). A greater severity of symptoms means a greater hospitalisation rate, with this having led to the acute bed base of some hospitals being overwhelmed (Nacoti et al., 2020). The responses to this have involved rationing access to care for infected individuals (Rosenbaum, 2020; Wood et al., 2021), with consequences extending to uninfected individuals through severe disruptions to service for routine and “non-urgent” treatments (Macdonald et al., 2020). Indeed, in the UK, the proportion of patients receiving their first cancer treatment within two months of referral has dropped from 86% in July 2019 to 25% in July 2020 (BMJ, 2020a).

In attempting to mitigate or overcome these problems, a key enabler for local health authorities is the ability to predict, with some reasonable degree of accuracy, the future number of COVID-19 infections, hospitalisations and deaths. For healthcare providers, this can help ensure an adequate number of beds are made available for any impending demand, while identifying periods of relative suppression in order to safely exploit available capacity in working through any elective backlog (Carter et al., 2020). For public health agencies, the ability to predict future “waves” of COVID-19 is important for informing the nature, timing and magnitude of any localised restrictions to reduce spread (Turk et al., 2020), and to ensure that adequate provisions are made regarding the size of temporary mortuary space (Wood et al., 2020).

However, projections of such fidelity have not been within easy reach for many local health authorities. In the UK, much of the early forecasting was performed at no finer granularity than national level (Ferguson et al., 2020). With no alternatives, and in urgent need to estimate hospital capacity requirements, this left local health authorities scrambling to somehow proportion these national trajectories for their own populations. The only option for many systems (including ours) was to pro rata them with some attempt to control for demographic differences between national and local populations. This was not unreasonable with age having been a known risk factor for disease progression and outcome (Jordan et al., 2020). Yet the credibility of such derivations was quickly undermined by diverging accuracy attributable to the localised manner in which the disease was spreading. For instance, transmission in London compared to South West England was far greater than would otherwise be explainable by demographic differences alone (BMJ, 2020b). As the pandemic advanced, forecasts at regional level became available (P. Birrell et al., 2021; Verhagen et al., 2020). But being centrally produced, this has left local health authorities at the behest of the publishers in terms of the output format, update frequency and – perhaps most importantly – decision on which factors underpin the scenario assessed. With locally owned and locally configurable models, individual healthcare systems would have control over these aspects.

To such end, the following principles may be valuable. First, the underlying model should be conceptually appropriate in representing the key dynamics of infection, transmission, disease progression and outcome. Compartmental models of the SEIR variety, temporally estimating the proportion of the population that are Susceptible, Exposed, Infectious and Recovered, have an established history of use in this regard (Kermack & McKendrick, 1927; Silal et al., 2016). Second, the model should be locally configurable. COVID-19 “waves” form locally, and so any simplistic attempts to augment national-level projections may lead to significant under- or over-estimation when applied to individual health systems (Castellani & Caiado, 2020). Third, in informing future projections, considered scenarios should be definable through estimated temporal profiles for parameters relating to contacts and transmission, i.e., how the local population mixes socially and conforms to restrictive measures such as face coverings. In aligning model parameters to these “levers” that are, to some degree, controllable by public health agencies, this allows an evaluation of the efficacy of possible responses such as localised lockdowns. Finally, it is important that the model can, in general terms, be understood sufficiently well to enable effective use, calibration and interpretation by the multi-disciplinary working group that would be required to produce reliable projections under the range of plausible scenarios (more discussion is provided on this in Section 5.2).

This paper details the approach taken in one health system to address these principles in establishing and embedding a localised framework for modelling COVID-19 infections, hospitalisations and deaths. Using a model developed previously within the system (Booton et al., 2021), this paper describes the activities of a cross-system Scenario Review Working Group that has convened weekly since 16 June 2020and whose outputs have been routinely absorbed into local decision-making through the system’s major incident commands (in the UK, these are known as Bronze, Silver and Gold Command).

The remainder of this paper is organised as follows. In Section 2, a summary is provided of the Susceptible-Exposed-Infectious-Recovered (SEIR) model used for generating projections. The framework under which this is used is detailed in Section 3. In Section 4 some example outputs of this process are illustrated. Finally, some concluding remarks are provided in Section 5. Note that while, at the time of writing, the SRWG continues to meet on a weekly basis, the past tense is used hereon for posterity.

2. Model

The Scenario Review Working Group (SRWG) has used the published model of Booton et al. (2021) in order to obtain projections for infections, hospitalisations and deaths. Development of this open-source solution, coded in R, was led by a multi-disciplinary project team at a local university within the relatively early stages of the pandemic. The model had previously been used to provide initial estimates of COVID-19 case trajectories and hospital capacity requirements in the South West of England during the first wave of the pandemic.

The solution uses a variant of the standard Susceptible-Exposed-Infectious-Recovered (SEIR) compartmental model structure, which has commonly been adapted in describing the transmission dynamics of many infectious diseases (Li & Muldowney, 1995), including COVID-19 (Flaxman et al., 2020; Wynants et al., 2020). With reference to Figure 1, susceptible (S) individuals within the population become exposed (E) to the disease at a rate governed by the probability of transmission, the proportion of the population who are infectious, and the number of person-to-person contacts. These individuals then either become symptomatic and infectious (I) or remain asymptomatic (A) yet infectious carriers. Thereon, all the latter and a proportion of the former recover (R) from the disease, while the remainder see their condition worsen to the extent that hospital (H) treatment is required. For these individuals, the resulting outcome is either transfer to intensive care (C), death (D), or recovery (R). Those admitted to intensive care either die (D) or recover (R).

Figure 1.

Schematic representation of the SEIR-based model used by the Scenario Review Working Group (SRWG) to project infections, hospitalisations and deaths. This illustrates the various “compartments” used to represent disease progression among the population. Note that IC refers to intensive care

In order to capture the explanatory importance of person age in COVID-19 transmission dynamics and individual disease progression, the model is stratified according to the eight age groups 0–4, 5–17, 18–29, 30–39, 40–49, 50–59, 60–69 and 70+ years. Age groups were selected in order to appreciate key social contact patterns (for primary, secondary and tertiary education and employment) and clinical outcomes (acute and intensive care length of stay, the probabilities that an infectious individual will require hospitalisation and onward transfer to intensive care, and the probability of death from both such states). The former were parameterised through historical pre-pandemic social mixing rates (Mossong et al., 2008) and those surveyed during first-wave lockdown (Jarvis et al., 2020). Parameterisation of the latter was based upon clinical reports such as those from the Intensive Care National Audit and Research Centre (https://www.icnarc.org/Our-Audit/Audits/Cmp/Reports). Other disease-specific epidemiological parameters assumed unrelated with age were estimated from the literature, including incubation period (5.1 days, Lauer et al., 2020) and the probability of infected cases becoming symptomatic (82%, Mizumoto et al., 2020).

The dynamics of the model are described by a system of (deterministic) ordinary differential equations, with each equation representing the age-stratified inward and outward flow rates for the corresponding modelled compartments, i.e., one such equation for each of the S-E-I-A-R-H-C-D states (see above and Figure 1). Given uncertainty, a number of model parameters within these equations are defined not as exact values but by min-max ranges, for which model fitting is used to obtain a complete parameterisation. To this end, the objective is to minimise the difference between the observed and expected infections, hospitalisations and deaths over the immediate past historical period. This is achieved through Latin hypercube sampling in order to generate a set of random samples across the multi-dimensional parameter space, with selection through maximum likelihood. The frequency by which the model is recalibrated and the capacity in which it is used are discussed in the next section.

3. Framework

This section outlines the scope of the SRWG’s roles and responsibilities, the key information flows it would make use of in discharging its duties, and the format and membership of its regular meetings.

3.1. Scope

The ultimate objective of the SRWG is to reliably predict the future number of COVID-19 infections, hospitalisations and deaths within the local healthcare system. Accordingly, the agenda of SRWG meetings – while pragmatic to the situation and priorities at the time – would typically include the following as standing items. First considered was a comparison of the updated model fits against historical infections, hospitalisations and deaths, i.e., observational data, including the most recent data used for calibration. Any revisions to parameters associated with partially unobservable data and assumptions would also be considered, such as those related to past social mixing and contacts. As part of sense checking, modelled estimates for the current and past values of the effective reproduction number (R) would be assessed against other regional and national estimates.

The second standing item considered the future projections obtained through the model. This would involve any revisions to the specification of the future-state scenario under consideration, in order to incorporate the changing behaviour of citizens and any new locally or nationally mandated interventions. The former could relate to public confidence to re-introduce into societal norms (retail, hospitality), with examples of the latter relating to requirements to wear face coverings or the imposition of travel restrictions.

3.2. Information flows

The flow of information into and out of the SRWG is outlined in Figure 2. For reviewing model and scenario calibration (the two standing agenda items), the group would make use of three types of input. The first of these concerns a daily flow of confirmed cases, bed occupancy and deaths data from the local hospitals. The second contains contacts and movement data, the former based upon the “POLYMOD” (Mossong et al., 2008) and “CoMix” (Jarvis et al., 2020) surveys of people’s movements and interactions (covering pre-COVID-19 and first-wave lockdown conditions respectively) with the latter estimated through the latest Google Community Mobility Reports (https://www.google.com/covid19/mobility/). The third input accounts for any past, present and planned public health interventions and population behaviours, which are retrieved from various confidential and publicly-available sources, e.g., news reports, minutes from the government’s scientific advisory group for emergencies (SAGE), and regular national survey data from the Office for National Statistics (Opinions and Lifestyle Survey: COVID-19 module).

Figure 2.

Outline of inputs to and outputs from the Scenario Review Working Group (SRWG)

SRWG outputs included two summary reports generated automatically upon running the model. One report contained the full results, examples of which are illustrated in Section 4. The other consisted of a short (one page) document gauging model stability through plotting the latest modelled projections alongside those of the previous two model runs. Both documents were received by the healthcare system’s major incident commands (Bronze as main recipient and Silver for information). The model would also output projections in raw data form, which would be used by Business Intelligence for forward capacity planning with acute and community healthcare providers. Bespoke briefing notes would be produced for particular matters of concern on an ad hoc basis.

3.3. Format and membership

Following senior executive level sponsorship, the SRWG commenced weekly meetings from 16 June 2020. Weekly was deemed an appropriate frequency given the rapidly evolving evidence base, national policy and local developments on the ground (Rutter et al., 2020). Meetings were held virtually and were allocated 30 minutes.

The group was chaired by the dedicated Modelling and Analytics team located within the local National Health Service Clinical Commissioning Group (CCG). Public health membership comprised quantitative and non-quantitative representatives from the three constituent Local Authorities within the health system, as well as a regional public health representative. This ensured interpretation of testing data (especially serological analyses) and an interface with local public health decision-making (both in understanding current interventions and relaying projections for escalation). Membership of epidemiologists from the local university, including those who led development on the underlying mathematical model, facilitated explanation of counterintuitive model behaviour and addressed ongoing conceptual appropriateness of the model.

4. Outputs

This section presents three illustrated examples describing how the SRWG has supported the local health system in responding to the pandemic between the months of June and October 2020.

4.1. Estimating the shape of the first wave tail (spring 2020)

Objective: Following nationwide lockdown measures imposed on 23 March 2020 (UK Government, 2020a), cases peaked in the month of April. The question was, how quick would they reduce thereafter? The motivation for this question was, in part, due to a realisation that COVID-19 had consumed large amounts of hospital capacity and that, with non-urgent elective treatments having earlier been cancelled (National Health Service, 2020), there was a sizeable backlog accruing. With reliable projections of COVID-19 admissions and bed utilisation, more effective decisions could be made regarding the optimal balance of elective and non-elective care. Essentially, it would be known just how much more routine treatment could be resumed safely.

Approach: Various national-level relaxations to the first lockdown had already been made by the time of the analysis, including unlimited outdoor exercise and some provisions for inter-household mixing (UK Government, 2020b). With no firm timeline for subsequent relaxations, projections would be derived on the basis of no further change in model parameters concerning social contacts and mixing. On a practical level, this assumed not just that government policy and guidance remained unchanged, but that public behaviour would not alter. That is, citizens would not adjust their interactions with other citizens (such as in response to periods of good weather) nor become complacent regarding precautionary measures (such as the two-metre rule). These simplifying assumptions were necessary at such an early stage, in part given the paucity of information available for model calibration (Rutter et al., 2020), but moreover to gradually build confidence with key “customers” of SRWG outputs (who were hitherto unfamiliar with epidemiological modelling).

Outcomes: Figure 3 contains the modelled projections for daily acute bed demand (number of new COVID-19 admissions required over a 24-h period) and acute bed utilisation (the number of beds in use by COVID-19 patients) for all hospitals within the healthcare system. This shows that both such measures are indeed reducing, but non-linearly with a decreasing rate of reduction. To exemplify, the model projects that median daily bed demand will reduce by 18% of the peak value over the month of June, decreasing to 11% over the month of July. This follows larger reductions of 29% during May. Also included in Figure 3 are the observed values up to the date at which the model was run (10 June). Contrasting these against the projected median is not fully supportive of goodness-of-fit, when assessed via R-squared (0.26 and 0.78), Mean Absolute Percentage Error (0.37 and 0.22) and Mean Directional Accuracy (0.45 and 0.65). Given volatility associated with fairly low counts (especially for acute daily demand), what may provide a more reasonable assessment is a comparison against the projected 95% confidence bands. Upon visual inspection, it can be seen that the majority of observed points lie inside these bands. For acute daily demand this accounts for 57 out of the 86 observations (66%), increasing to 75 (86%) for acute bed utilisation.

Figure 3.

Observed and modelled daily acute bed demand and acute bed utilisation for all hospitals within the healthcare system (indexed given sensitivities). Modelled projections presented as median (solid lines) and 95% confidence bands (shaded areas). The vertical dashed grey line denotes the date at which the model was run (10 June)

4.2. Hospitality reopening and face coverings (summer 2020)

Objective: On July 4 2020 restrictions were lifted across England for much of the hospitality sector, including pubs, restaurants and cinemas, following the reopening of non-essential retail on 15 June (UK Government, 2020c). Social restrictions were also eased, meaning two households could now meet (inside or outside) and people could stay away from home overnight. Robustly estimating the effect of these relaxations on the relevant model parameters had not been straightforward through SRWG meetings during July, especially in advance of the change and in the immediacy thereafter, where supporting data was lacking. The SRWG meeting on 5 August provided an opportunity to review the latest data in calibrating model parameters, as well as accounting for a recent change in legislation requiring the mandatory wearing of face coverings in shops and supermarkets from 24 July (UK Government, 2020d).

Approach: While, by 5 August, there was no evidence of increasing cases following the 4 July relaxations, any delayed effects of community transmission could not be discounted, especially given lags in processing testing data. Google mobility data – then available for most of July – suggested an increase in social contacts from 4 July, and again towards the end of the month. This uptick in late July was also supported by local anecdotal evidence contained in news reports. Given that relaxations were predominantly to the hospitality sector and that advice had not changed regarding schools and home/office working, the SRWG determined that changes should be made only to the “Leisure” parameter relating to social contacts. This was proportionately increased in line with the 4 July rise inferred from mobility data, as well as the SRWG-estimated increase towards the end of the month – which, for parsimony, was assumed to occur from 24 July (the same date as the face covering implementation). The effect of face coverings was captured through a reduction to modelled viral person-to-person transmissibility from 24 July. This was made through a largely subjective assessment using the limited available information regarding efficacy.

Outcomes: Table 1 details the model parameters relating to social contacts. This firstly contains the proportions across the various settings considered in the pre-pandemic “POLYMOD” survey. For modelling during the pandemic, these values are scaled within the periods of time associated with the different interventions imposed by the government. As can be seen, the scaler corresponding with the “Leisure” component, which aligns to the “Retail and Recreation” aspect of Google’s mobility data, increases from 4 July and again from 24 July, for the reasons explained above. As well as affecting projections for hospital demand and utilisation, the impact of this can also be seen through the modelled effective reproduction number, commonly known as R (Figure 4). Following the imposition of nationwide lockdown measures, which reduces R to 0.79 (95% CI: 0.73 to 0.83), the 4 July relaxations are shown to lead to an increase to 0.95 (0.88 to 1.01), with only a modest further increase on 24 July (noting this is a net result of both an increase from greater contacts and a decrease from lesser transmission due to face coverings – with respective contributions approximately 3:1). With R below one, the number of hospitalisations will continue their decreasing trend.

Table 1.

Modelled social contacts, partitioned by the components considered in the pre-pandemic “POLYMOD” survey. Contains original proportions and corresponding scalers estimated for different stages of the pandemic. Note that “other” refers to contacts made at locations other than home, work, school, travel, or leisure, and “multiple” refers to contacts simultaneously made during more than one component (for more information, refer to Mossong et al., 2008)

Component	Proportion of contacts	Scalers on original proportions
		Lockdown	Pre 4 July	4 to 24 July	Post 24 July
Home	30.5%	1.00	1.00	1.00	1.00
Multiple	5.5%	0.67	0.77	0.77	0.77
Work	17.5%	0.36	0.49	0.49	0.49
School	14.5%	0.05	0.10	0.10	0.10
Leisure	13.5%	0.25	0.29	0.52	0.56
Transport	4.5%	0.25	0.33	0.33	0.33
Other	14%	0.67	0.77	0.77	0.77

Figure 4.

Modelled values of the effective reproduction number. Modelled projections presented as median (solid line) and 95% confidence bands (shaded area). The vertical dashed grey line denotes the date at which the model was run (5 August)

4.3. Return to school and a second wave (autumn 2020)

Objective: Schools had been closed to most attendees since the first nationwide lockdown but were reopening at the start of the new academic year in early September 2020. The SRWG meeting on 2 September focused on refining scenario parameter estimates in line with the latest available data and guidance. As well as obtaining “do nothing” model projections, the aim was to understand the effect of local-level mitigations that were expected to be necessary in averting a second wave.

Approach: Since schools had been closed to most attendees since the first lockdown, there was little data available to support the calibration of the relevant social mixing parameters. Initially, as an order-of-magnitude estimate, it was assumed that contacts between pupils would be 50% of that pre-pandemic. On confirmation that earlier plans for “whole year-group bubbles” would be instated (UK Government, 2020e), the SRWG upped this figure to 90%, in order to take account of the much-heightened chances of contact. This value also reflected assessments made within the SRWG that 10% of pupils would not return to school. Modelling revealed an effective reproduction number (R) above one, with which cases would rise exponentially and require mitigation. Aside from reducing the susceptible population through vaccination (not possible at the time), the remaining option to tempering growth would be to impose restrictions on movements and social contacts. In considering this, the SRWG needed to make three assessments – regarding the threshold, nature and duration of the intervention. Given the multitude of uncertainty regarding estimation of these, the accepted approach was to attempt to mimic the localised lockdown imposed on the city of Leicester in July (UK Government, 2020f). For the modelling, this translated into a six-week period of intervention during which reductions in transmission would decrease the effective reproduction number (R) to 0.89 (95% CI: 0.83 to 0.93). The intervention would be made corresponding to the point at which 30 cases per 100,000 is breached (occurring 8 November).

Outcomes: Without mitigation, hospital capacity many multiples of that used in the first wave would be required (Figure 5, upper row). With a locally implemented mitigation, bed demand and utilisation would be reduced to levels below those experienced during the first wave. However, without sustained intervention, the numbers of cases would ultimately continue their ascent. The only means to avert this, aside from vaccination, is to impose repeated lockdowns or to find more permanent ways of living such that contacts and social mixing are reduced to the extent that R remains under one.

Figure 5.

Modelled daily demand and utilisation for acute and intensive care (IC) beds on an unmitigated and mitigated basis (the latter accounting for six weeks of restrictions during which the effective reproduction number, R, is reduced to 0.89). Modelled projections presented as median (solid lines) and 95% confidence bands (shaded areas). The vertical dashed grey line denotes the date at which the model was run (2 September)

5. Discussion

This paper presents both a model (Section 2) and framework (Section 3) for projecting COVID-19 infections, hospitalisations and deaths. Established within a major healthcare system during the first wave of the pandemic, the SRWG has provided weekly results under numerous conditions, key examples of which are illustrated here (Section 4). Modelled results have been used widely for various purposes and have been well received by senior leaders at Bronze, Silver and Gold Command. The principal use of SRWG outputs has been in setting the number of acute beds to open for expected COVID-19 presentations, and thus informing the residual capacity available for elective treatments. With approximately one in five hospital admissions requiring post-discharge intermediate care, outputs have been used to determine the procurement of additional community capacity for “step down” beds and home visits. Projections of deaths have been used by public health teams to understand required mortuary capacity (additional to the fixed amounts available within the hospitals). As well as making projections on a “do nothing” and “do something” basis, the modelling has since been leveraged to actually design that “something”, i.e., the necessary nature, magnitude and timing of a potential intervention. In all cases, it should be noted, the role of the SRWG was not to make formal recommendations but to attempt to provide the best-possible projections to add to the pool of information from which decisions would be made by those with executive authority. It is therefore not possible to forensically isolate the degree to which specific decisions were influenced by particular SRWG outputs.

In this final section, a review is provided of some of the technical limitations, practicalities and considerations for other healthcare systems contemplating such a framework.

5.1. Model limitations

With reference to the first of the four guiding principles outlined in Section 1, namely that “the model should be conceptually appropriate in representing the key dynamics of infection, transmission, disease progression and outcome”, there are a number of matters worth exploring in further detail. First, in assuming a closed population, the model accounts for no immigration or emigration beyond the geographical limits of the local health authority under consideration. While this is a reasonable assumption under lockdown conditions, whereby regional travel is dissuaded or banned, such as during the first wave (UK Government, 2020a), it is limiting at other times. The effect of this could be to underestimate or overestimate transmission, depending on the amount of flow and the relative geographical differences in disease prevalence. Second, the model assumes no nosocomial transmission (hospital-acquired infection). Again, this may be appropriate under lockdown conditions when hospitals have been removed of non-COVID-19 patients, but would be less suitable when elective procedures are resumed and thus susceptible individuals are introduced to a possibly infectious environment (Elliott et al., 2020). This would serve to increase the number of infections beyond that which is modelled. Predictions are exposed to a similar effect from community outbreaks or “super-spreader” events that are not captured in the model – an example of this is evident in Figure 3 (acute bed utilisation, mid-May).

Third, there is no hospital capacity constraint accounted for in the model, thus assuming that limitless acute and intensive care beds could be made available for COVID-19 demand. Under the first lockdown, in which UK hospitals were requested to “postpone all non-urgent elective operations” in order to “free-up the maximum possible inpatient and critical care capacity” (National Health Service, 2020), this may have been an appropriate assumption, but otherwise it would lead to fatality under-estimation due to the consequential occurrence of “capacity-dependent deaths” (Wood et al., 2020). Finally, it is assumed that those surviving infection are immune to future reinfection. At the time of writing, the accuracy of this assumption is simply unknown (BMJ, 2020c), although were reinfection widely possible then the Susceptible compartment (Figure 1) would be larger and thus the number of infections, hospitalisations and deaths would be greater (ceteris paribus).

5.2. Reflections on practical experiences

The presence and management of uncertainty has been a key thread along which the SRWG has operated since its first meeting. In the early stages, uncertainty has manifested through a lack of data required to reliably calibrate the clinical and epidemiological model parameters. As time has passed, these issues have subsided – for instance, early understating of intensive care lengths of stay diminished as the implicit censoring of longer episodes has reduced with time (although this took many weeks and months to percolate through to the data). Uncertainty would next be found in calibrating the many contact parameters associated with the hitherto unknown effect of lockdown relaxations and a return to school. In this regard, a fully data-driven and scientific approach was not always possible, and so assessments would be made on the basis of SRWG consensus following an appraisal of the available information (specifically, a figure for the parameter in question would first be suggested, and incrementally fine-tuned in response to scrutiny and discussion). Such uncertainties have, however, reduced as more is learned from the lived experiences of societal restrictions that have gone before. For instance, in late 2020, with cases rising and local lockdowns being considered, the social contact parameterisation of the first wave lockdown could be duly leveraged. This time, however, uncertainty can be found in potential synergies with winter, a season commonly associated with increased pressures on healthcare systems (Fisher & Dorning, 2016).

In responding to these challenges, the SRWG has benefitted from the following. First, a flexible and reactive agenda in order to ensure emerging issues can be promptly addressed (e.g., newly announced societal restrictions or significant changes in testing or mobility data). Second, a culture prepared to appraise options on a best-efforts (or notionally least-worst) basis; early strives for “perfectionism” had put at risk the ability to provide projections according to the timescales agreed with system stakeholders. And third, an appropriate membership who were, through the relevant networks within their respective organisations, able to ensure that relevant information could be efficiently obtained and relayed to the group. This proved of particular importance for sourcing local information to contextualise the monitored empirical data, e.g., the incidence of specifically localised outbreaks, such as those occurring within certain schools or care homes.

Additionally, the SRWG has benefitted from use of a model which has appeared to provide the appropriate complexity to adequately represent the key epidemiological dynamics while remaining comprehensible to a team ranging in mathematical competence (thus concerning the fourth “design principle” stated in Section 1). The importance of such a balance is well established in applied modelling of public health interventions (Garnett et al., 2011), with particular recognition for epidemiological models used in confronting the COVID-19 pandemic (Holmdahl & Buckee, 2020; Saltelli et al., 2020). Specifically, SRWG function has benefitted from a visualisable description of the model mechanics (Figure 1), as opposed to a “black box” solution, and close working between modellers and non-modellers, including the original model architects.

While, at the time of writing, the SRWG continues to meet, its possible legacy for the wider system is already starting to emerge. Ultimately, the principal activity necessary of such a group is the calibration of model parameters past, present and future. To do this requires the conversion of multi-channel information, including news stories and government briefings, into numerical point estimates as required by the model. Bridging this gap is bridging “the qualitative” and “the quantitative”. In doing so, this has showcased the benefits of working across boundaries, not just organisationally between healthcare, local authorities and academia, but also professionally between planners, analysts and modellers. Embedding such cross-working could lead to enhancements in system capability through the pandemic and beyond.

5.3. Considerations for other healthcare systems

First considered should be whether the healthcare system in question has a sufficient population to meaningfully employ such an approach. Spurious outputs may result if there is insufficient “information” to reliably characterise the model or if the model inadvertently “overfits” to volatilities associated with low-volume data. The healthcare system considered here has a one million population, and it is suggested that application to any lesser population be treated with caution. Second, access to the appropriate data is required. Such data (outlined in Sections 2 and 3) should be of good quality and refreshed at least at the same frequency at which updated model outputs are sought (here, weekly). Third, healthcare systems should have some consideration to the model they use for obtaining projections. The framework presented here is essentially model-agnostic, provided the four outlined guiding principles can be satisfied (Section 1). While other locally employed SEIR-based efforts for modelling COVID-19 have been published (Campillo-Funollet et al., 2020; Renardy et al., 2020), these do not contain open-source code for ready reuse by others. Publishing the open-source code used here allows others to use and adapt the model as they wish (Booton et al., 2021). The final consideration is one of resourcing, and the extent to which the appropriate skills and experiences can be assembled. The healthcare system considered here has had the benefit of close links to epidemiology experts at the local university and a dedicated modelling team within the Clinical Commissioning Group (which has been able to facilitate SRWG discussions through a sufficient grasp of both the practical and academic aspects). Additionally, public health representation has been vital to ensure the latest intelligence could be contributed for parameterisation and scenario analysis, and modelled outputs could be efficiently relayed to those who ultimately held control over any intervention (e.g., localised lockdowns). These may prove critical factors for setting up and running an effective SRWG.

Disclosure statement

No potential conflict of interest was reported by the authors.

Author contributions

RMW conceived the need for the Scenario Review Working Group, determined its format and membership, and obtained sponsorship for its foundation. RMW, ACP, BJM and ALP set the agenda for SRWG meetings, led the discussions, and generated the modelled outputs. RDB and KMT led development on the “SEIR” mathematical model. All authors participated in the SRWG by contributing to model and scenario calibration. RMW wrote the manuscript.