Modelling the effect of first-wave COVID-19 on mental health services

Abstract

During the first wave of the COVID-19 pandemic it emerged that the nature and magnitude of demand for mental health services was changing. Considerable increases were expected to follow initial lulls as treatment was sought for new and existing conditions following relaxation of ‘lockdown’ measures. For this to be managed by the various services that constitute a mental health system, it would be necessary to complement such projections with assessments of capacity, in order to understand the propagation of demand and the value of any consequent mitigations. This paper provides an account of exploratory modelling undertaken within a major UK healthcare system during the first wave of the pandemic, when actionable insights were in short supply and decisions were made under much uncertainty. In understanding the impact on post-lockdown operational performance, the objective was to evaluate the efficacy of two considered interventions against a baseline ‘do nothing’ scenario. In doing so, a versatile and purpose-built discrete time simulation model was developed, calibrated and used by a multi-disciplinary project working group. The solution, representing a multi-node, multi-server queueing network with reneging, is implemented in open-source software and is freely and publicly available.

Graphical abstract

1. Introduction

COVID-19 is a virulent infectious disease caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2). It has spread globally and was declared a pandemic by the World Health Organisation on 12th March 2020 [1]. Since then it has put significant pressure on the health services of even the most economically advanced nations [2]. While authorities had, quite appropriately, centred their immediate response on physical health, the need to manage impacts on mental health has become of increasing concern as the pandemic has evolved.

One of the first articles addressing this was published on 4th February 2020 [3]. Stating that “timely mental health care needs to be developed urgently” the authors reviewed learnings from the 2003 SARS outbreak in making a number of recommendations for COVID-19 patients and affected health workers. Findings thereafter have revealed some of the pressures faced by healthcare employees, with elevated levels of stress and anxiety identified at even the very early stages of the pandemic [4]. Outside of those directly affected by the virus, individuals with pre-existing mental health conditions may place additional demands on mental health services, owing to a “high susceptibility to stress compared with the general population” [5] and “greater risk from social isolation” [6]. Yet, the disruptive and traumatic events associated with how the pandemic unfolds may also necessitate those not previously known to mental health services to seek treatment. Increased loneliness, reduced social interactions, concerns about finances and employment are all recognised factors that may instigate mental health issues through the COVID-19 pandemic [6], [7]. Indeed, evidence from the early stages of the pandemic suggests a rise in the levels of stress, anxiety and depression in the general population [8], [9], particularly in younger people [10]. Adverse consequences, including suicide, may be expected if these conditions cannot be effectively managed [11].

Even before COVID-19, responding to mental health need was not straightforward in the UK, with increasing demand, worsening outcomes, and complex care networks and access points across primary, secondary and tertiary settings [10], [12]. In the healthcare system considered in this study, there are six recognised ‘levels’ at which mental health care may be accessed (Fig. 1). The setting correspondent to the lowest level of need includes various well-being and self-help support initiatives such as social prescribing (Level 1), whereas the setting aligned to the highest level of need consists of specialist secure and forensic services (Level 6). After entering the system, patients transition through the settings according to their increasing or decreasing need, with exit from the pathway through failure to attend, discharge, or death. At the point of referral for any of the services, the patient may have to wait for resources to become available, such as a consultation slot or a bed. This waiting time can have negative consequences for the patient, such as an escalation of need through worsening condition (resulting in the patient reneging in favour of higher-level services) or, in more extreme cases, crisis or suicide [13]. It is these consequences that are of concern given the longer waiting times that may result given the additional demands placed on the service through COVID-19.

Fig. 1.

Description of six ‘levels of need’ for mental health services in the healthcare system studied, detailing the setting, means of access, and type of resource.

This paper reports on the use of modelling in response to pressures posed by COVID-19 on mental health services within a major healthcare system during the first wave. The modelling aims were firstly to project the impact of COVID-19 on a ‘do nothing’ basis, and secondly to assess the optimality of two service-level supply-side mitigations being considered to alleviate negative outcomes. This would inform decisions around where to put what amount of additional resource in order to minimise the extent to which patients would escalate through the system (the general premise being that patients should be treated in the lowest level service where possible).

1.1. Literature review – operational research, mental health and COVID-19

Within the early stages of the pandemic, there have been few examples of studies in which the modelling of mental health services has been considered. In their prospective review of simulation approaches for COVID-19, Currie et al. [14] have acknowledged the potential utility of system dynamics in addressing the matter. System dynamics is a form of continuous simulation in which the rates into and between various ‘compartments’, possibly representing disease state or healthcare service, are modelled. Under the former representation, such a compartmental model is developed by Cardinal et al. [15] in examining whether therapy medium (face-to-face or virtual) affects COVID-19 infection rates for a single community mental health service. However, neither this nor any other COVID-19 study identified at the time of writing applies system dynamics in considering patient flow around a network of mental health services, such as those described in Fig. 1. This holds for discrete event simulation, a commonly-used stochastic alternative to system dynamics that captures patient movement not by continual flows but through the constituent individual events (e.g. patient referral joins queue, patient enters service, patient departs service).

Turning to the pre COVID-19 literature, it is fairly well recognised that the modelling of mental health services has been under-used when compared to aspects of physical healthcare [16], [17]. In two case studies, Smith et al. [18] model the mental health service operations at a prison and a forensic facility using system dynamics, investigating hypothetical scenarios involving changes to demand and number of beds (note that this is a relatively simple system compared to community mental health services). In another study employing system dynamics, Smits [19] considers the outpatient and inpatient services for an adult mental health centre, modelling staffing levels in addition to patient movements. Yet, as duly acknowledged by the author, while system dynamics may afford an insight into the “behaviour of the healthcare system and the reasons that cause changes in system performance”, it is not a tool for objectively quantifying numerical results, such as the distribution of waiting time.

For this, discrete event simulation may be used. Kuno et al. [20] developed a discrete event simulation of the flow of patients with serious mental health illness between acute and residential bedded facilities. Calibrated through patient arrival rate, length of stay, and onward transition rates, the model is used to examine the responsiveness of waiting times to various capacity configurations. A similar method is employed in other mental health modelling studies, regarding the emergency [21] and primary care [22] settings. However, none of these studies consider the effect of waiting time on negative outcomes, such as the afore-mentioned escalation of need or suicide. One exception is the modelling of Koizumi et al. [23], in which the ‘reneging’ of mental health patients from the waiting list of one service to that of another is captured, yet with a number of simplifying assumptions (e.g. exponential lengths of stay) and production in proprietary software (thus limiting re-use). Considering waiting times in mental health is important for several reasons. First, waiting times are variable from service to service and some may be lengthy; second the consequences of waiting can be diverse (people may recover but also may deteriorate, go into crisis or commit suicide) and people vary in how long they are prepared to wait; third, it tends to be monitored as a quality indicator so managing and incorporating waiting times into system management will be desirable for service managers.

1.2. Use of discrete time simulation (DTS)

In seeking to meet the aims of the project, it was clear that flow into and between multiple service and exit nodes would be required to adequately represent patient movements around the mental health system (Fig. 2). Each service node would need to contain multiple service channels in order to account for the many patients that would concurrently receive care in any one setting. In terms of the queueing properties, it can be assumed that there are infinite buffers and no priority classification with patients ‘served’ in the order they arrive, i.e. first-come first-served (FCFS). This is with the exception of patients that renege to another service or exit node, should any waits become excessive. Given the nature of mitigations that may reasonably be assessed through ‘what-if’ scenario analysis, functionality would be required for time-heterogeneity in the arrival and service (length of stay) probability distributions as well as the service node capacities. Thus, in Kendall’s notation [24], the queueing network was essentially a series of $G\left(t\right)\left|G\left(t\right)\right|C\left(t\right)\left|\mathrm{Inf}\right|\text{FCFS}$ queues with reneging. A final model requirement was the need to have versatility with regard to structure and calibration, given the fast pace and evolving scope of the project.

Fig. 2.

High level specification of queueing network required to be modelled, representing service nodes as rectangles, queueing facilities as circles, arrivals and transitions as solid arrows, reneges as dashed arrows, and exit nodes as hexagons.

In modelling this queueing network a discrete time simulation (DTS) was used. This is characteristically similar to the perhaps better known discrete event simulation (DES), insofar as both are stochastic (i.e. appreciative of variability associated with patient arrivals and lengths of stay) and both are temporally derived (based on some notion of clock time). There are architectural similarities too, in that both are conceptualised through a series of replications, with each capturing a number of events over a continuum of time (as generated through different random seeds in order to appreciate the afore-mentioned variability). However, unlike DES, which steps forward individual events in variable periods of time, DTS considers a variable number of events within a fixed period of time. That is, DES is event-driven and operating on an event-step basis, whereas DTS is time-driven and operating on a time-step basis.

While the typical ‘go to’ approach for stochastic modelling would be DES [25], [26], there are particular advantages of DTS which, in this study, relate to the trade-off between computational time and accuracy. For a large mental health system with significant volumes of activity, DES would be highly computationally expensive given the substantial number of individual patient-related events that would be occurring within, say, a day or a week (with such event-steps each requiring a number of computational processes and read/write operations). Whereas, through the appropriate setting of the time-step, these events may be batched within a single iteration of a DTS algorithm without significant loss of accuracy [27]. Additionally, computational timeliness was of particular importance given the ‘shelf life’ of modelled estimates in the midst of a fast-evolving public health emergency. It is arguable that there may be further scope to reduce computational time through adopting a system dynamics (SD) methodology [28]. However, any benefits in this regard were determined to be outweighed by the inherent lack of stochasticity, and thus the inability to produce (meaningful) confidence bands around modelled estimates – a feature all the more important when attempting to address and convey the effect of multiple sources of uncertainty in the early stages of a previously unknown virus [14].

Regarding implementation, most ‘off-the-shelf’ stochastic simulation tools implement a DES method. This includes proprietary software such as Simul8 and Arena and freely available open-source offerings such as Simmer, SimPy and Ciw [29]. In fact, compared to DES, there has been very little interest in DTS methods. Within healthcare, the limited range of examples include the consideration of a single-server queueing process for general practice appointments [30] and the modelling of waiting times for routine outpatient referrals [31]. However, in no such example is a versatile solution made available that meets the modelling requirements articulated at the outset of this section. As such, a purpose-built (and open-source) DTS solution would be developed.

1.3. Remainder of paper

The remainder of this paper is structured as follows. Section 2 covers a full description of the discrete time simulation engine developed, including a permanent link to where the tool can be accessed. Application of the simulation model, including generated outputs, is presented in Section 3. Finally, a discussion of assumptions, limitations and further work is provided in Section 4.

2. Model

Within each replication of the discrete time simulation (DTS), the first task undertaken by the automated computer algorithm is to initialise an event schedule containing information for all external arrivals within the user-defined simulated period of time (inclusive of any ‘warm-up’ period necessary to attain steady-state dynamics for the sought baseline position). Arrivals in this sense are external since they represent demand (i.e. referral or physical presentation) that is new to the system and not originating from within (i.e. exiting one service node and joining the queue for another). With the user-supplied external arrival distributions and associated parameters, the time and location of each simulated arrival can be generated. In order to track individual pathways through the system, and associated performance information such as waiting time, a unique personal identifier is given to each arrival.

Once a replication is initialised, the next task is for the automated computer algorithm to iterate over all time-steps in the simulated period and to perform the corresponding events. As with DES, this is performed in order of advancing clock time, since the state of the system at a future iteration is informed by that of the most current. At each iteration, the event schedule is inspected for events logged to occur within that time-step. The four possible event types, in the order in which they are executed within each iteration, are as follows (noting that at the very first iteration, the sole event type on the event schedule will be an external arrival):

• (a)Service completion – the completion of service at a given service node. The scheduled outcome, whether it be onward transition to another service node or exit from the system, will be executed.
• (b)Service start – the start of service at a given service node. This event type is not scheduled but is conditional on the current numbers within each queue at the time-step in question. Upon starting service, a completion time will be scheduled through sampling from the user-supplied length of stay distribution. Additionally, an outcome will be scheduled through sampling from the user-supplied service node transition probabilities.
• (c)Renege — any currently-occupied queue is abandoned in favour of transition to another service node or exit from the system. This event type is not scheduled but is conditional on the numbers within each queue and their waiting time. The probability of reneging, in addition to the destination, are sampled from the user-supplied functions of waiting time.
• (d)External arrival – new demand not originating from within the modelled system. If there is available capacity at the corresponding service node then service will start at the next time-step.

At the end of the simulated period, the final event schedule, tracking all executed events, is stored. Also stored is a set of derived performance measures tracking occupancy, waiting time, and queue size, noting that only events at time-steps beyond any user-supplied warm-up period are used. These are thereafter aggregated across all performed replications of the simulation, yielding summary metrics at a service node and time-step level, e.g. producing a mean and 95% confidence interval for occupancy at service node X at time Y. Results are provided at a time-step level since the user may supply time-heterogeneous parameters, and thus may be interested in assessing temporal behaviour of the system. The model is set up to accommodate time-heterogeneous values for external arrival rates, service node capacities, and the distribution parameters for lengths of stay. Full details of the model can be found in the Supplementary Material.

In order to run the model, the estimated parameters are recorded and input via .csv template files. Detail on the required parameters is summarised in Table 1. The model is implemented in the statistical programming language R, and is permanently and publicly available, alongside the input template, at [32]. Practically, some global options relating to the number of replications and warm-up period need be configured, and the populated input template is thereafter sourced through execution of the single R script, following which outputs are deposited back into the working directory. Also provided in the electronic repository is a ‘light’ version of the model, which condenses the input template through removing the functionality for time-heterogeneous parameters. This version may be appropriate for novice simulation analysts or for uses in which time-homogeneity can reasonably be assumed.

Table 1.

Model parameters, requiring definition for each service node.

Parameter	Description	Detail	Time-bound
External arrival distribution	Probability distribution for number of external arrivals at a service node within a time-step, and associated parameters. E.g. Poisson with parameter $\lambda$.	Probability mass function.	Functionality for time-heterogeneous parameters.
Length of stay distribution	Probability distribution for length of stay at each service node, and associated parameters.	Either probability mass function or probability density function (latter is rounded to the nearest time-step).	Functionality for time-heterogeneous parameters.
Capacity	Maximum number of service channels that can concurrently be occupied at each service node.	Scalar.	Functionality for time-heterogeneous parameters.
Service node transition probabilities	Probability of transitioning from any one service node to another service or exit node following completion of service.	Probabilities for all ‘to’ nodes must sum to one for a given ‘from’ node.	Functionality for single step-change in parameters.
Renege probability function	Equation in ‘x’ where ‘x’ is time spent waiting, and associated parameters.	Range between 0 and 1. Support on ‘x’ from 0 to infinity.	Time-homogeneous parameters only.
Renege destination transition probabilities	Given renege, the probability of reneging to any of the various service and exit nodes.	Probabilities for all ‘to’ nodes must sum to one for a given ‘from’ node.	Time-homogeneous parameters only.

3. Application

The model (Section 2) was applied to mental health services within a major healthcare system in south west England during the first wave of the COVID-19 pandemic (spring 2020). As part of wider efforts to identify and mitigate pressures posed by the pandemic, the modelling aims were firstly to project the impact of COVID-19 on a ‘do nothing’ basis, and secondly to assess the optimality of two service-level supply-side interventions being considered to alleviate negative outcomes.

3.1. Setting and context

Before detailing model parameterisation and results, it is first worth some commentary regarding the situation ‘on the ground’ at the time of the modelling, in order to more wholly contextualise its application. It should be recognised that conditions for modelling within a live practical environment differ much to those of a dedicated research project, for which sufficient preparatory work can ensure upfront data collection and problem structuring. This is especially the case when in the midst of a global pandemic, where constraints on the opportunity to engage stakeholders or source bespoke data are coupled with the crucial need for timely results. Realistically, these constraints inevitably limit the specific technical quality of modelling that may otherwise be achievable. Yet, this does not mean that such efforts are without value.

For the healthcare system considered here, there was very little information available at the time of the study relating to the ‘do nothing’ impacts of COVID-19 and the possible optimality of supply-side mitigations under consideration (i.e. the modelling aims, as specified in Section 1). Thus, there was a feeling that almost any information was better than nothing — provided, of course, that assumptions and limitations could be clearly articulated. At the very least, this would provide some grounding to the various conversations taking place. And, as is well known by practitioners, the modelling process can actually be just as revealing and insightful as the ultimate model output itself.

In applying the model to the considered system, there were particular challenges additional to those described above. First, there was an incomplete holistic understanding of how the six ‘levels of need’ (Fig. 1) interacted. While individual service providers knew of their interaction with others, there was no complete picture of overarching flows at a system level. Second, at such level, there was incomplete data and no existing modelling to build upon. Finally, as with much of the UK NHS [33], there is little comprehension of modelling approaches, meaning that any efforts to engage stakeholders would have to start the conversation from first principles.

3.2. Parameterisation

To oversee pathway specification, model calibration and scenario analysis, a project working group was set up with representation from across the system’s constituent organisations, including those services aligned to the afore-mentioned six ‘levels of need’ (Fig. 1). Also represented was a breadth of professional experience, with a membership composed of specialist clinicians, general practitioners, data analysts, service managers, and healthcare commissioners. Elicitation of the project working group was primarily through project-wide (virtual) workshops and secondarily through one-to-one follow-ups with project group members who had made themselves available for further discussion.

The first engagement of the project working group was to guide definition of the structure of services to be modelled, between which services patients may flow, and where patients may renege from and to. Since there had been no dynamical modelling of the mental health system pre COVID-19, there was no existing flow schematics to leverage and hence this was a new task to be undertaken. This was the purpose of the first workshop, attended by 31 individuals from across the project working group. While it had first been postulated that the six considered ‘levels of need’ may serve as a structure for modelling flow (i.e. with each level represented by a single service node), it transpired during the workshop that this would be a gross simplification of patient activities. Instead, service nodes would align to the various individual services that comprise the system. Services would, however, only be included from Levels 2 through 5 since Level 1 (Community Support) was deemed too broadly defined, incorporating sources of demand and non-capacity dependent activity (e.g. participation in social groups) and Level 6 (Highly Specialist) was considered too heterogeneous, with complex, bespoke collections of interventions based around small numbers of individuals, and essentially independent of the others. Fig. 3 provides a schematic of the services and flow that were to be modelled.

Fig. 3.

Schematic of mental health services to be modelled, detailing the service nodes (rectangles), patient flows (directed arrows), renege routes (dashed arrows), and exit nodes (hexagons). Service nodes are aligned to the considered ‘levels of need’ (grey-shaded boxes labelled L2 through L5). Note, GP represents general practice, IAPT represents Improving Access to Psychological Therapies (an anxiety and depression service in the UK NHS), and MHP represents Mental Health Provider (an unnamed specialist mental health provider operating within the healthcare system studied). Note also, the exit node Planned End can be accessed from any service point except MHP Triage or Inpatient, and Other can be accessed from any of the service nodes except MHP Triage.

With the structure specified, attention turned to calibrating the model — assigning values to the parameters detailed in Table 1 for each of the service nodes. In basing these parameters, the first consideration was the length of the time-step, for which one day was deemed a natural choice given that modelled patient activities (arrival, discharge, transfer) would realistically be separable by at least such duration. Given a number of practical constraints, including data limitation and project timescales, the parameterisation was achieved through a non-specific mixture of available empirical data, expert opinion, and back-fitting (estimating the remaining unknown model parameters through ‘fitting’ the model such that modelled output measures were aligned to their empirical equivalents, e.g. for occupancy and waiting times). A second workshop, attended by 22 individuals from across the project working group, was used to appraise and refine the parameterisation.

Three scenarios were modelled in projecting performance of the system into the future. The demand profiles (i.e. external arrival rates) were unchanged across these scenarios. They were defined over three periods: first, the immediate pre pandemic months; second, a four month period with subdued demand due to ‘lockdown’ measures (within the first wave); and third, a twelve month period in which both new and any ‘pent up’ demand is released into the system. Demand under the first and second periods were empirically-estimated through available data. Demand under the third period was estimated through analysis performed within the system based on projections of historical demand, a rapid evidence review commissioned from the local university-hosted applied research collaborative [34], and published research concerning the increase in demand for mental health services [35]. Given substantial uncertainty in the COVID-19 forecasts available at the time of the modelling, no ‘second wave’ was assumed in these assessments (note that, following this modelling, a second wave did occur in the UK, peaking in January 2021).

The alternate ways in which services could be configured in meeting this demand was captured across the three scenarios. As a ‘do nothing’ option, the Baseline scenario assumed no change in the service configuration as to the pre COVID-19 period. The remaining two scenarios assumed some intervention based upon those being considered by planners within the healthcare system. Intervention One assumed improved prevention and mitigation in the wider community (Level 1) leading to reduced low-severity demand at primary care (Level 3), as well as uplifts in capacity for psychological therapy (Level 2) and mental health outpatient and community services (Level 4). Intervention Two assumed effective early intervention in community (Level 1) and primary care (Level 3) leading to reduced demand for psychological therapies (Level 2) and reduced lengths of stay (reflecting reduced acuity) at outpatient and community services (Level 4), with capacity reduced on such basis. Both scenarios contained increased flow rates from primary care (Level 3) to specialist mental health services (Level 4). The model parameters and those defining these three scenarios are contained in full at the aforementioned online repository [32].

3.3. Results

Simulated results are provided in Table 2 and Fig. 4, Fig. 5, Fig. 6. In each of these, reference is made to the three previously mentioned periods of time, referred to in the table as ‘Pre’, ‘During’, and ‘Post’ (lockdown) and illustrated in the figures through the dashed vertical lines. Estimated mean and 95% confidence bands are obtained from 100 replications of the simulation model as detailed in Section 2. In interpreting these results, a number of key themes can be identified.

Table 2.

Summary of mental health system performance measures pre, during, and post ‘lockdown’ within the first wave of the pandemic in the UK. Presented results are for the mean over the considered periods, with ‘Pre’ and ‘During’ estimated empirically and ‘Post’ forecast through the simulation model under the considered scenario. Performance is measured by queue size (the number of referrals awaiting commencement of treatment) and wait time (number of days spent in queue before entering service). Change is the percentage difference between ‘Pre’ and ‘Post’ periods.

Service node	Measure (mean)	Baseline scenario				Intervention one		Intervention two
		Pre	During	Post	Change	Post	Change	Post	Change
L2 social prescribing	Queue size	273	150	482	77%	470	72%	475	74%
	Wait time	21	19	22	5%	22	5%	22	5%
L2 IAPT assessment	Queue size	3658	2740	6702	83%	6384	75%	6570	80%
	Wait time	59	60	59	0%	58	−2%	59	0%
L2 IAPT therapy	Queue size	8	8	8	0%	151	1788%	8	0%
	Wait time	1	1	1	0%	12	1100%	1	0%
L3 GP initial	Queue size	152	77	166	9%	167	10%	172	13%
	Wait time	1	1	1	0%	1	0%	1	0%
L3 GP long term low intensity	Queue size	12	8	13	8%	12	0%	12	0%
	Wait time	1	1	1	0%	1	0%	1	0%
L3 GP long term high intensity	Queue size	42	25	46	10%	36	−14%	37	−12%
	Wait time	1	1	1	0%	1	0%	1	0%
L4 MHP triage	Queue size	100	58	109	9%	114	14%	118	18%
	Wait time	1	1	1	0%	1	0%	1	0%
L4 MHP crisis	Queue size	8	5	9	13%	11	38%	10	25%
	Wait time	1	1	1	0%	1	0%	1	0.0%
L4 MHP community	Queue size	519	431	600	16%	36	−93%	1142	120%
	Wait time	34	34	35	3%	1	−97%	38	12%
L5 inpatient	Queue size	3	3	31	933%	43	1333%	92	2967%
	Wait time	1	1	7	600%	9	800%	18	1700%

Fig. 4.

Summary of queue size at each service node modelled pre, during, and post ‘lockdown’ (mean and 95% confidence bands).

Fig. 5.

Summary of service node waiting time as modelled pre, during, and post ‘lockdown’ (mean and 95% confidence bands). Results are omitted for service nodes which display negligible change over the course of these three periods.

Fig. 6.

Summary of service node reneging rates as modelled pre, during, and post ‘lockdown’ (mean and 95% confidence bands). Results are omitted for service nodes which display negligible change over the course of these three periods.

The first of these regards the ability of reneging to mask performance impacts assessed through queue size and waiting time. The large increase in post-lockdown queue size for Social Prescribing and IAPT Assessment occurs across all three scenarios (Table 2), and is as a result of the sharp increase in new demand, both directly to this service node and via GP Initial (Fig. 4). However, it can be seen that waiting times do not bear similar increases (Fig. 5). The reason for this is that the longest-waiting patients are reneging from these queues (Fig. 6), meaning an otherwise lower waiting time for those that do remain. This demonstrates the danger of simply assuming that since waiting times have to a large degree been held, that patients have not suffered as a result of any negative repercussions from reneging. With waiting times often quoted as a measure of service quality, these results underline that, in mental health, this is a more complicated metric than it appears.

The second theme concerns the sensitivity of low-throughput and at-capacity services to increasing demand. The queue size and waiting times for the Inpatient service node are relatively low and steady pre pandemic. But with no slack to absorb the rising demand from other services and no route to renege (Fig. 3), the queue size and waiting times increase substantially (Table 2, Fig. 4). Interestingly, this issue is more pronounced under the two intervention scenarios, since it has been assumed (Section 3.3) that more patients can access these services from primary care (Level 3). The practical implication of a waiting list for inpatient services is that beds may need to be ‘bought’ from other areas in health (who may also be experiencing an increase in demand) or from the independent sector. Additionally, the capacity in all inpatient units will have been reduced by the necessary environmental risk management practices introduced for all organisations during the pandemic.

The third theme addresses the extent to which increased demand can be ‘off-set’ by reduced length of stay. As mentioned (Section 3.2), demand for Level 4 services is increased in both of the intervention scenarios compared to the Baseline. Under Intervention Two, it has been assumed that post-lockdown lengths of stay are reduced by four days (29%) for MHP Crisis and one month (8%) for MHP Community. This is intended to reflect a decrease in the average severity of patients due to an intervention to enhance any primary and community care received prior to arrival. For MHP Crisis, the length of stay reduction is sufficient to ameliorate the increased demand. This can be seen in Fig. 4, Fig. 5 through comparison to Intervention One for which length of stay reductions were not considered. Yet for MHP Community, reducing length of stay by even this large absolute amount is insufficient to absorb the extra demand. While queue size and waiting time stabilise at levels higher than Baseline, they do not grow uncontrollably and this is due in part to reneging (Fig. 6). To prevent this undesirable consequence, a substantially greater reduction in length of stay would be required. Any such efforts should, however, be approached with caution since much effort has already been made (in the UK National Health Service) to reduce lengths of stay, and further efforts may give rise to undesirable consequences, such as lesser clinical outcomes and/or additional pressures on downstream services and suitable community accommodation.

The final theme considered relates to the latent impacts for services with relatively large lengths of stay. Consider MHP Community, for which inward flow increases substantially post lockdown, particularly across the two intervention scenarios. Under Intervention One, capacity is doubled and this is assumed to occur as a one-off step change at the end of lockdown. For some time, it appears that this has succeeded in reducing queue size and waiting times (Fig. 4, Fig. 5), but eventually these measures start to increase, ultimately exceeding pre pandemic levels. The reason for this is the long lengths of stay associated with this service, meaning there is a greater period of transient behaviour before a steady state is reached. This supports the importance of inspecting time series results alongside any numerical summaries (Table 2) that can mask the complexities of such dynamics.

4. Discussion

This article concerns model development and application to quantify the effect of COVID-19 on mental health services according to a ‘do nothing’ baseline and under two mitigating scenarios considered at the time of the study. To ensure value there was a need for timely outputs in informing service planning following the relaxation of ‘lockdown’ measures during the first wave of the pandemic (Section 3.1). To achieve this, a relatively short window of time was available for completing the range of activities typically associated with a modelling project of this nature. As such, there are a number of limitations that should be acknowledged.

4.1. Limitations

A key practical limitation relates to the data used for model calibration. Even without constraints on time it is a difficult task to acquire data from disparate service providers that is complete, good quality, sufficiently granular for stochastic modelling, and which can be consistently aligned to model parameters [36]. This holds especially for UK mental health services, which are known to be data poor [37]. Additionally, there are, of course, many uncertainties associated with a hitherto unknown disease [14]. Despite efforts taken to validate inputs, the calibration achieved here will inevitably reflect these practical realities.

Concerning outputs, a meaningful validation was not possible since complete data did not exist but moreover because over the period concerned there were a vast number of known and unknown extraneous variables impacting upon output metrics. Effective validation is possible when there is a clear baseline and only one or a small number of inputs changed, such that any resultant difference in the output can be purely and credibly assessed and accordingly attributed to the changing input(s). This is not the case during the early stages of a global pandemic. A final practical limitation relates to the coverage of the model (Fig. 3). Due to the inability to gain clarity on how the activities of certain services could be represented within the model, those of voluntary organisations and local authorities were excluded from consideration; these would otherwise feature within Levels 2 through 4.

In terms of model limitations, it is important to acknowledge an assumption that is ordinarily reasonable in models of patient flow, but which is limiting with respect to the application considered here. It is commonly assumed, without contention, that any one patient can be at any one service node at any one time. However, with mental health services, a patient may simultaneously be in receipt of care from multiple services at any one time. For instance, they may be in the specialist service (Level 4) while receiving social prescribing (Level 2). Failing to appreciate this could underestimate capacity sufficiency across service nodes for which this can occur. A possible solution to this problem may exist through the use of agent-based models — an alternative computational modelling paradigm which considers the system as a collection of autonomous decision-making entities (as opposed to the relative constraints of discrete event/time models, which typically concern the more structured progression of entities along some predetermined network of service nodes). Another model limitation relates to patient priority. The model assumes first-come first-served, whereas order of service is realistically dependent on priority levels assigned to the referrals [38].

4.2. Contribution

For the healthcare system under consideration, the modelling has been regarded as a valuable undertaking. It highlighted the potential drawbacks of the two scenarios under consideration, such as increased waiting lists for Level 2 IAPT Therapy and Level 5 Inpatient (Fig. 4) and increased reneging from Level 2 IAPT Assessment and Level 4 MHP Community (Fig. 6). Such insights would simply not have surfaced without this modelling. Ultimately, neither of the two scenarios was adopted in full by the healthcare system, which instead opted for increased proportionate support to capacity at all levels of need and ongoing assessment of demand and capacity sufficiency beyond the first wave of the pandemic. It is not fully possible to comment on the extent to which the modelling has informed such actions; practically there is seldom a one-to-one relationship between model output and decision, especially in such a large and complex healthcare system at such an unprecedented time. Modelling is, at best, one ingredient to factor into the decision-making process alongside various other concerns, such as those of a financial and ‘political’ nature. Within the healthcare system, the exploratory modelling undertaken here has opened the eyes of those not previously exposed to the value of such methods. This has led to the acquisition of external funding to further refine the model for (post) COVID-19 uses, which carries particular importance given the recently-announced Community Mental Health Framework [39] and movement towards Integrated Care Systems [40]. Additionally, it has provoked stakeholder interest in using alternative OR approaches for modelling psychiatric intensive care activities within the system.

More widely in the healthcare service, making the model freely and publicly available enables others to replicate the approach taken here within their own healthcare system, with this paper serving as a blueprint for practically how that may be achieved. This holds especially in the UK, given coding of the model in R and the growing community of R users in the National Health Service [30]. Effective use of this model may reasonably extend beyond the mental health setting, into other healthcare or non-healthcare domains for which there is flow of some denomination along a network of service nodes with or without reneging. Any sought adjustments to the model can be readily made given the open-source implementation.

In terms of advancing the academic literature, at a practical level, this is one of few studies to model a system of mental health services [17], and the first known attempt to do so in light of COVID-19. It is also one of relatively few studies to consider discrete time over discrete event simulation, addressing what is perhaps an undue balance given the computational efficiency advantages. Furthermore, to the authors’ knowledge, this is the first open-source discrete time simulation of a multi-node queueing network with reneging. Finally, for the OR discipline, this study acts as an important demonstration of the versatility of OR methods, showcasing the ability for simulation models to be developed, calibrated, and used at pace. In the authors’ experience, healthcare modelling projects can encounter deadlock with managers failing to recognise the benefits of modelling and analysts hesitating to model due to poor quality or incomplete data. In the case of this study, we were able to compromise sufficiently in order to perform useful modelling in constrained conditions.

4.3. Further work

In terms of model extension, consideration should relate to the afore-mentioned limitations, namely relaxing the constraint of single node service and accounting for patient priority (Section 4.1). The former represents an interesting problem that has to date received little attention in the literature, while approaching the latter could make use of established techniques to more realistically represent queue discipline [41]. In addition to assigning patients to certain priority levels, they may also be assigned to certain ‘groups’ that could be configured upon recognised inequalities, e.g. younger people, whose mental health is known to have been disproportionately affected during the pandemic [10], or healthcare workers [42]. Adjustment of the model in this regard would allow more insightful forecasting and modelling of interventions that can be targeted at such groups.

Investigators should remain aware of how services are adjusting in light of COVID-19 in assessing the ongoing validity of model assumptions and conceptual appropriateness. There are already accounts of how mental health services are changing [43] as well as suggestions for how services should change [44]. This may necessitate adjustments to the model inputs as well as to the model itself (i.e. for which changes to the code would be required). Aditionally, model inputs should be responsive to future demand projections related to any further wave of the pandemic, as well as capturing the potential for associated reductions in capacity due to staff burnout which may occur during prolonged periods of pressure [4].

CRediT authorship contribution statement

B.J. Murch: Conceptualisation, Data curation, Formal analysis, Investigation, Methodology, Software, Visualisation, Writing – review & editing. J.A. Cooper: Investigation, Writing – review & editing. T.J. Hodgett: Data curation, Writing – review & editing. E.L. Gara: Investigation, Writing – review & editing. J.S. Walker: Investigation, Writing – review & editing. R.M. Wood: Conceptualisation, Formal analysis, Investigation, Methodology, Resources, Supervision, Writing – original draft.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following is the Supplementary material related to this article.

Data and material

Model code and data used for this study is available at https://github.com/nhs-bnssg-analytics/simulation-dts-renege.