Examining the effect of interventions in emergency care for older people using a system dynamics decision support tool

Abstract

Background

Rising demand for Emergency and Urgent Care is a major international issue and outcomes for older people remain sub-optimal. Embarking upon large-scale service development is costly in terms of time, energy and resources with no guarantee of improved outcomes; computer simulation modelling offers an alternative, low risk and lower cost approach to explore possible interventions.

Method

A system dynamics computer simulation model was developed as a decision support tool for service planners. The model represents patient flow through the emergency care process from the point of calling for help through ED attendance, possible admission, and discharge or death. The model was validated against five different evidence-based interventions (geriatric emergency medicine, front door frailty, hospital at home, proactive care and acute frailty units) on patient outcomes such as hospital-related mortality, readmission and length of stay.

Results

The model output estimations are consistent with empirical evidence. Each intervention has different levels of effect on patient outcomes. Most of the interventions show potential reductions in hospital admissions, readmissions and hospital-related deaths.

Conclusions

System dynamics modelling can be used to support decisions on which emergency care interventions to implement to improve outcomes for older people.

Key Points

• Outcomes for older people attending Emergency and Urgent Care settings are poor.

• A range of emergency care frailty interventions have been shown to improve outcomes, but deciding which intervention to develop is not straightforward.

• System dynamics modelling offers a low-risk approach to exploring and providing evidence-based support to decisions on which emergency care interventions to implement.

Background

Emergency and Urgent Care (EUC) is a major international issue, especially for older people, for whom frailty attuned care pathways might improve outcomes [1], knowing which model of care to implement in different settings can be demanding. This paper describes a user-friendly decision support tool that enables clinicians and planners to assess the potential impact of five evidence-based interventions for people aged 75 or older attending their Emergency Department (ED).

The tool uses a computer simulation approach—system dynamics (SD)—to represent the patient journey from ED via (potentially) a hospital stay through to discharge and then possible readmission or reattendance in ED. Patients were classified into five 5-year age bands to take into account age-related differences in outcomes. The modelled outcomes include mortality (hospital-related, and community), length of stay, readmission and reattendances. The tool allows users to run ‘in silico’ experiments without the need to test service developments on the ground.

The aim of this paper is to describe the genesis of the SD tool and offer insights as to how the tool might be helpful in practice.

Methods

Computer simulation is widely used in business and industry to test process or service redesign ideas on a computerised version (a model) of the real-world system before incurring the risks and costs of implementation. SD takes a high-level, strategic view, and depicts the interactions between the different parts of a system over time [2]. It has been used to model patient flow at an aggregate level [3–6], support policy decisions [7] and to demonstrate how small changes in one part of the NHS can have unexpected impacts on other parts [8, 9].

The tool was sense-checked by a wide range of professional healthcare experts, and the outputs validated against hospital metric data collected as part of the study and against Office for National Statistics (ONS) data (see Supplementary Appendix S1).

Selection of interventions

The selection of interventions to be modelled was informed by a systematic review of reviews [1], which reviewed the type, effect size and quality of the evidence for the interventions. A brief description of each intervention is summarised in Table 1, with more detail in Supplementary Appendix S2. The evidence associated with each intervention has been included in the tool’s user interface, along with the supporting literature. The documented effect sizes were incorporated in the SD model and used to determine both the immediate outcomes and any knock-on consequences.

Table 1.

Interventions included (see Supplementary Appendix S2)

Intervention	Description
PRE-ED
Proactive Care	Primary care led population risk stratification programme involving nurse-led comprehensive geriatric assessment (CGA), care planning and coordination [10–12]
Hospital at home (HaH)	Holistic care provided for people with urgent care crises in their own homes [13]
In-ED
Geriatric Emergency Medicine (GEM)	Consultant geriatrician led CGA in the ED [14–18]
Front door frailty (FDF)	Nurse-led CGA plus community in-reach and rehab teams to avoid admission [15], [17–21]
POST-ED
Acute Frailty Unit (AFU)	Geriatrician led CGA delivered in short stay areas for admitted patients. [22]

The tool allows the user to select which age group(s) are eligible to receive the chosen intervention, and the hours for which it will be operational: for example, 9 am–5 pm on weekdays, 24/7, or any user-specified times. This then determines the number of patients who receive the intervention, based on the proportion of ED arrivals in that age group and time period, enabling users to get a more realistic idea of the potential impact of their selected intervention.

Data

The model parameters were derived from a linked data analysis of routine healthcare data for the entire Yorkshire and Humber (Y&H) region of the United Kingdom (population 5.5 million) using data from the CUREd research database [22,23], a large, linked database comprising healthcare information for approximately one-tenth of United Kingdom’s population. The database links NHS 111 calls, ambulance incidents, Accident & Emergency, Admitted Patient Care episodes and provider spell datasets, combining over 23 million linked patient episodes of care from April 2011 until March 2017. The CUREd dataset makes it possible to track each patient from their initial emergency call, any conveyance to the ED, their ED attendance, through ED discharge or hospital admission, and ED re-attendance.

The data analysis informed the following parameters (Supplementary Appendix S1, Table A2) for each age band:

• Number of patients in ED and in hospital.

• Average daily number of ED attendances.

• Average daily number of emergency admissions via ED.

• Average daily number of emergency admissions not via ED, e.g. direct admissions to specialty services.

• Average length of stay.

• Average daily number of in-patient deaths.

• Average proportion of patients who re-attend ED within 30 days of discharge.

The model drew upon several other data sources for the model parameters and initial patient population levels:

• ONS mortality statistics (2019) [24] and population estimates for the Yorkshire and Humber region (mid-2019) [25].

• The number of care home residents in the Yorkshire and Humber region has been estimated from the Care homes market study [26] as the information is not readily available. The 2017 Care homes market study states that 11,300 care homes provided care for 410,000 residents. Recent estimates of the number of care homes in the Yorkshire and Humber region suggest 1,453 homes [27], which would lead to approximately 52,719 residents in the area.

• The number of care home deaths has been estimated from ONS data that look at the number of deaths within the care sector [28].

The model

The model follows a cohort of older people for 1 year and the metrics are updated daily as patients move through the system. The user interface allows the user to select the hospital setting most similar to their own, based on three hospital ‘archetypes’—large, medium and small. Users can either enter their own values of each parameter, if they know them, or simply use the default values provided with the tool. The model then simultaneously runs two scenarios: a baseline (do nothing) and an intervention. The results, which include several key hospital metrics such as the average number of patients attending and discharged from ED each day as well as hospital mortality figures, are then displayed. Each graph shows the chosen metric over a year under the baseline scenario, compared with the selected intervention scenario.

The SD model and its user interface were developed in the simulation software AnyLogic (version 8.7.3). The technical development of the model is reported separately [29] and the validation tests are shown in Supplementary Appendix S1.

Stakeholder engagement

Stakeholder engagement was conducted throughout all stages of development to ‘sense-check’ the emerging findings and comment on the usability of the user interface:

• The research team (clinicians from primary care, emergency and geriatric medicine) reviewed the model monthly over 3 years.

• An independent study steering committee (also including clinical and methodological experts) provided high-level oversight and gave a strong steer on which intervention scenarios to include, considering the level of supporting evidence.

• A series of four, 1-hour-long, external stakeholder events (three aimed at clinicians and commissioners and one at patients and carers, totalling around 40 individuals) considered the structure and usability of the user interface and results screens.

• The tool was also demonstrated at two national NHS measurement classes attended by 60 attendees from 20 NHS organisations. The attendees included service managers, improvement and transformation leads, clinical directors, consultants and specialty doctors from frailty and emergency care departments within NHS Trusts, commissioning groups and local councils.

The stakeholder events and NHS measurement classes gathered feedback from potential end-users of the tool. One key element of their feedback was to have a tool that could be used in any area of NHS England. The NIHR study and initial development of the decision support tool had focused on the Yorkshire and Humber area but following the feedback from the stakeholder events, a generic version of the tool was developed (with a slightly different user interface) for use in any of the Integrated Care Boards (ICBs) of NHS England.

Patients and the public were involved in the design of the programme of work, and in reviewing the development of the SD model. In a separate workstream, patient/carer interviews elicited what matters to older people with urgent care needs, and we attempted to bring these findings into the model development wherever possible.

Tool availability

A link to the tool will be provided on the NHS Future platforms website, alongside user guides (all free of charge).

Results

This section presents some illustrative results (Table 2) for five selected intervention scenarios for a hypothetical large hospital in the Yorkshire and Humber region. For each intervention, the hospital parameters have been adjusted according to the risk ratios cited in the review of reviews and the outcomes are compared with the baseline ‘as-is’ scenario. The parameters used in the baseline scenario are given in Supplementary Appendix S1, Table A2.

Table 2.

Scenario analysis—the five interventions compared with the baseline, applied to a hypothetic large hospital in the Yorkshire and Humber region

Intervention	Opening times	Admissions		Older people in hospital		Nursing home admissions		Readmissions		Hospital deaths
		Empirical evidence estimates	SD model estimate	Empirical evidence estimates	SD model estimate	Empirical evidence estimates	SD model estimate	Empirical evidence estimates	SD model estimate	Empirical evidence estimates	SD model estimate
Proactive Care	24 hours a day	Negligible	Negligible	Not reported	Negligible	Negligible	Negligible	Not reported	< 0.5% reduction possible due to daily variation	Negligible	Negligible
HaH	9 am-5 pm, weekdays	Not reported	<1% reduction, i.e. 50 fewer admissions annually	Not reported	Negligible	Reduction (RR = 0.58) applied at 6 months	2.4% reduction, i.e. 60 fewer admissions annually1.	Larger proportion of people living at home at 6 months (RR = 1.05, CI = [0.95, 1.15])	1.5% reduction, i.e. 70 fewer readmissions annually	Slight reduction (RR = 0.98)	< 1% reduction, i.e. five fewer deaths annually.
GEM	9 am-5 pm, weekdays	(2.6–19.7%) reduction	2.6% reduction, i.e. 450 fewer admissions annually	Conflicting reports about LoS	1.5% reduction, i.e. 330 fewer patients in hospital annually.	Limited evidence	Negligible	Reduced (RR = 0.74)	9.3% reduction, i.e. 450 fewer readmissions. (The evidence suggests a 26% reduction when applied to all older patients but as the service is not operational 24/7, leading to a target population of 9.6%)	Data not included in reviews	1.5% reduction, i.e. 11 fewer deaths annually
FDF	8 am-8 pm, everyday	Fewer admissions (risk ratio = 0.9)	4.9% reduction, i.e. 850 fewer admissions annually.	No clear evidence	3.2% reduction, i.e. 730 fewer patients in hospital annually.	Fewer admissions to nursing homes (RR = 0.75)	6% reduction, i.e. 150 fewer annually	Reduced readmissions (RR = 0.95)	2% reduction, i.e. 100 fewer readmissions annually. (The evidence suggests a 5% reduction when applied to all older patients but as the service is not operational 24/7, a smaller percentage of patients are targeted).	Reduced hospital related mortality (RR = 0.92)	7% reduction, i.e. 50 fewer deaths. (Due to the reduced hospital related mortality and the reduced hospital numbers)
AFU	24 hours a day	Not included	3.5% reduction, i.e. 610 fewer admissions annually.	Length of stay could be increased by ½ day	1.6% reduction i.e. 360 fewer inpatients annually.	Not included	7% reduction, i.e. 175 fewer patients discharged to long term care home annually.	Fewer readmissions (RR = 0.78)	22% reduction, i.e. 1,000 fewer readmissions annually	Reduced hospital related mortality (RR = 0.86)	15% reduction, i.e. 110 fewer deaths annually

Baseline scenario

For the baseline scenario, the model estimates the following outcomes for patients aged 75 and above in a hospital of this size:

• 73 ED attendances per day, of which 48 lead to an admission.

• 16 emergency admissions direct to wards, per day.

• 14 readmissions within 30 days (11 from their own home and 3 from care homes) per day.

• 750 deaths in hospital per annum.

Interpreting Table 2

The five intervention scenarios are listed in the first column of Table 2. The typical operating hours associated with the intervention were used and are given in the second column. Each of the columns labelled ‘Empirical evidence estimates’ describes the effect size documented in the literature. The columns labelled ‘SD model estimate’ show the impact of implementing the chosen intervention for 1 year in that particular hospital setting. These are shown as percentage changes, as well as an indication of how many hospital admissions/readmissions/deaths would be avoided annually. For example, under the hospital at home scheme, operating between 9 am and 5 pm on weekdays, the model suggests 50 fewer admissions and 70 fewer readmissions from older patients during the year.

The results from this illustrative experiment suggest there is potential to reduce hospital admissions and readmissions, leading to fewer older patients in hospital and hospital-related deaths. In terms of hospital admissions, FDF and AFU appear to offer the greatest potential in reducing numbers. The AFU offers the most noticeable reduction in hospital readmissions (22% fewer, which could lead to an annual reduction of 1,000 readmissions), whereas FDF sees a marked reduction in hospital inpatient numbers (a 3.2% reduction, which could lead to 730 fewer patients in hospital, annually). AFU and FDF interventions also potentially offer larger reductions in the number of hospital-related deaths and admissions to long term care facilities. For example, an AFU intervention could result in 15% fewer deaths (approximately 110 per year).

It is also worth highlighting that some of the interventions have negligible impact. There is a benefit to including these, as it may prevent people from trying schemes that could prove not to be effective. Finally, we note that several of the services are only operational at certain times, either between 9 am and 5 pm on weekdays or between 8 am and 8 pm each day. This suggests further potential improvement, as services that extend their opening hours would see larger impact on their admissions, readmissions, etc. Using the tool, the user can extend the opening hours in their virtual scenario and see what effect it has on their hospital metrics. For example, if the GEM intervention (with a predicted 2.6% reduction target in hospital admissions) was to extend its hours to a 24/7 service, the tool estimates that there would be approximately 1,200 fewer admissions during the year and a similar reduction in the number of readmissions when compared to the baseline scenario. If, however, the opening hours cannot be extended but a larger effect size can be achieved (e.g. 19.6% instead of 2.6%), the tool estimates that there could be approximately 1,500 fewer admissions and 45 fewer hospital related deaths. This ability to consider different opening hours/target populations may prove useful for clinicians, commissioners and planners undergoing improvement projects or developing business cases to improve their care for older patients.

Discussion

To our knowledge, this is the first reported development and validation of a decision support tool (using peer-reviewed published evidence) focusing upon service for older people with urgent care needs. The tool can help clinicians, service managers and commissioners identify what model might best suit their specific setting and gauge the impact of the service on not just immediate short-term outcomes (admission vs. discharge from ED), but the impact on the wider health and social care system, over 1 year.

Strengths and limitations

Key strengths of the SD decision support tool are that it uses robust evidence to create the scenarios, an integrated dataset reflecting the whole of the EUC pathway, and extensive stakeholder engagement to ensure that it is both user-friendly and a realistic representation of the system. The model results can also be easily exported into Excel if needed.

The SD model adopts a whole system perspective, looking at the patient’s journey from their ED attendance, through their discharge and possible readmission over a simulated year of operation. By considering the whole system, the model can illustrate the connections not reflected in the empirical evidence. For example, in the HAH intervention, the literature does not provide evidence on the number of hospital admissions or the number of older patients in hospital beds. However, the SD model estimates the impact on both metrics. In the GEM scenario, the evidence on hospital numbers, nursing home admissions and hospital related deaths is limited—but these can be estimated in the SD model.

A final, very important strength of the tool is that we are able to consider the knock-on effects that some interventions may have further downstream in the patient pathway or in the future. For example, in the AFU intervention scenario, the evidence suggests that with the intervention increasing a patient’s length of stay by 0.5 days, there should be more patients in hospital. However, the reduction in patients readmitted leads to an overall reduction in hospital numbers.

We were not able to include frailty measures into the model, as these were not routinely embedded into the CUREd dataset. Although it would have been possible to capture Hospital Frailty Risk Scores (HFRS) for the admitted cohorts [30], this would not be available to include in the whole system model. We only used interventions that have been reported in evidence reviews detailing aggregate effect sizes; emerging care models, such as pre-hospital frailty services, offer promise, but the effect sizes for these interventions have not been aggregated. We were unable to model jointly delivered interventions, such as FDF in combination with AFU, as only separate interventions have been reported and we do not know how they interact or whether their effects are additive. We were unable to report upon person centred metrics as these were not included in the CUREd data.

The model does not explicitly take account of time of day, day of week or month, but uses the averages for these parameters over the full 6 years of data in the CUREd dataset, thereby smoothing out any daily or seasonal variation.

Finally, the model does not include an estimate of the staffing resources needed to provide a service and would need to be considered separately.

Relationship with wider literature

Current literature on emergency care models tends to report the impact of a single (albeit perhaps complex) intervention on a single cohort of individuals, and their associated outcomes in a linear manner [10–21,31]. In this study, the SD decision support tool permits an understanding of such interventions by taking a whole systems perspective that also incorporates temporal impacts on patients and therefore services: for example, seeing how a reduction in hospital readmissions may affect the number of patients discharged into care homes. Such an approach perhaps better mirrors the real-world impact of interventions in complex systems.

Few studies describe the ‘dosing strategy’ of the intervention (i.e. the proportion of people who might receive the intervention, when the service’s opening hours and patient eligibility criteria are considered). By including a consideration of the services’ opening times, we can provide perhaps more evidence-based estimates of the impact of interventions.

In each of the studies [10–21,31], the effect sizes for the chosen hospital metrics (admissions, readmissions, length of stay, hospital-related mortality), are typically given in terms of a risk ratio showing a summary estimate for the level of reduction observed. However, using a whole-system approach gives results that at first sight may feel somewhat counter-intuitive, to clinicians who are used to seeing summary estimates, but it does provide a more realistic estimation of what might be achieved with one scenario compared with another.

Implications for practice

The main aim of this SD decision support tool has been to enable any hospital to examine the benefit of a chosen ED intervention on their older population presenting at ED, without necessarily going through multiple different service development cycles. Clinicians and hospital planners can see the effect of the five interventions on their hospital setting and associated metrics. The user interface allows the user to easily enter their own data or use that contained within the tool and to view the graphical results produced. This whole system modelling might be especially relevant to the emerging ICBs as they consider their population health management approaches for older people. As more evidence-based interventions become available the tool can be adapted to include their effect sizes.

This SD model is but one tool required to enable and enact service developments. A knowledge of the evidence base, improvement methodology, understanding the policy context and financial levers and leadership are all necessary [32].

Implications for research

Future iterations of the SD model might be further developed to incorporate frailty measures as these become more widely represented in underpinning datasets [33, 34], Patient Reported Outcome Measures adapted for emergency care settings (in development), and an increased range of service options. Future research is needed to develop and test this tool for use in other acute hospital settings.

Conclusions

System dynamics modelling coupled with emergency care data can be used to support decisions on implementing emergency care interventions to improve outcomes for older patients. The decision support tool can support clinicians, service managers and commissioners to identify what EUC model might best suit their specific setting and gauge the impact of the service over 1 year.

Declaration of Conflicts of Interest

None declared.

Declaration of Sources of Funding

This work was supported by the NIHR, grant number (17/05/96), Identifying models of care to improve outcomes for older people with emergency and urgent care needs—funding applies to S.C., T.E., S.B., C.B., S.M.M., L.P., D.E., A.S., L.M.. S.C. and J.V.O. were also funded under NIHR Doctoral Research Fellowship 300901, 2020–2023, The development and validation of a patient-reported outcome measure for older people with frailty and urgent care needs. L.P. also receives(d) funding to undertake evidence reviews across the HS&DR and PHR programmes for NIHR. T.E. also received funding from the University of Southampton to further disseminate the decision support tool through a series of workshops to members of the British Geriatrics Society (BGS) community. L.M. is also funded by the Spanish Ministry of Science, Innovation and Universities (PID2019-104319RB-I00). S.C. is paid as clinical lead for the Acute Frailty Network, a national frailty improvement collaborative (NHS Elect), and receives research funding from NIHR for work on urgent care for older people. S.B. and T.E. received funding from NIHR for work on Frailty Dynamics.