New integrated care models in England associated with small reduction in hospital admissions in longer-term: A difference-in-differences analysis

Highlights

• •Integration policies are often seen as strategies to reduce avoidable hospital activity.
• •“New care models” were piloted in England under the “Vanguard” initiative.
• •We evaluate the efficacy of the Vanguard programme in reducing hospital utilisation.
• •The Vanguard initiative slowed the rise in emergency admissions observed in England.
• •The reduction occurred mainly in care home sites, at the programme’s end.

Abstract

Closer integration of health and social care services has become a cornerstone policy in many developed countries, but there is still debate over what population and service level is best to target. In England, the 2019 Long Term Plan for the National Health Service included a commitment to spread the integration prototypes piloted under the Vanguard `New Care Models’ programme. The programme, running from 2015 to 2018, was one of the largest pilots in English history, covering around 9 % of the population. It was largely intended to design prototypes aimed at reducing hospital utilisation by moving specialist care out of hospital into the community and by fostering coordination of health, care and rehabilitation services for (i) the whole population (‘population-based sites’), or (ii) care home residents (‘care home sites’).

We evaluate and compare the efficacy of the population-based and care home site integrated care models in reducing hospital utilisation. We use area-level monthly counts of emergency admissions and bed-days obtained from administrative data using a quasi-experimental difference-in-differences design.

We found that Vanguard sites had higher hospital utilisation than non-participants in the pre-intervention period. In the post-intervention period, there is clear evidence of a substantial increase in emergency admissions among non-Vanguard sites. The Vanguard integrated care programme slowed the rise in emergency admissions, especially in care home sites and in the third and final year. There was no significant reduction in bed-days.

In conclusion, integrated care policies should not be relied upon to make large reductions in hospital activity in the short-run, especially for population-based models.

1. Introduction

In many countries, the increasing number of older patients with multiple chronic health conditions imposes significant pressure on the use of health and social care services. Evidence suggests that many emergency admissions to hospitals may be avoided with appropriate actions [[1], [2], [3]]. Closer integration across and between health and social care settings has become a cornerstone of the policy response in many developed countries [[4], [5], [6], [7], [8], [9]].

However, integration has been a somewhat nebulous concept with multiple aims [10]. For instance, there are a variety of health and social care settings within a given system that could theoretically be ‘integrated’, for a variety of population groups, and in a variety of ways. Early models of integrated care tended to focus on single disease management models, or those predicted to be at ‘high-risk’ of hospitalisation with case management approaches [11,12]. These approaches rarely met intended aims, with some increases in patient satisfaction but little effects on health, or utilisation/cost measures except some small increases plausibly because of increased identification of unmet need [13,14].

More recently, the trend has been towards broader whole population-based approaches [15], implementing a variety of interventions across population groups, tending to focus on scaling up prevention-based approaches. There have been well-known successes using this population-based approach, most notably Kaiser Permanente in the US [16], spawning a movement towards Accountable Care Organisations (ACO) [17]. However, it is not clear if the ACO findings [18] will translate to very different settings outside of the US. Gesundes Kinzigtal in Germany is the best-known example of the population-based approach in Europe, but has proven difficult to study rigorously given the lack of comparable data and its unique setting [19].

So far, however, very little attention has been paid to integrated care models that sit between the very focused early disease management/’high-risk’ approach and the very broad population-based approaches outlined above. For example, approaches targeted to populations living in specific institutional settings, such as care homes. The high average utilisation of care, the very defined population and the standardised setting might theoretically aid implementation and scale-up.

Mixed method studies have highlighted many contextual tensions across the distinct objectives of cost-saving and of providing extended access and improved service quality to patients [13]; and the challenges to implement complex structural transformations that take time to be operationalised before generating `expected’ results [20]. Evaluation of integrated care initiatives has varied but has generally been criticised for being too short to allow full implementation or have an effect on outcomes. A large umbrella review of systematic reviews and meta-analyses identified the majority of study follow-up period lasted only up to a year [4]. However, the lag between effects of implementing integrated care initiatives and effects on outcomes is likely to vary depending on the population group targeted. For example, to affect hospital utilisation, focussed interventions on a targeted population (e.g. care home residents) who are vulnerable to high levels of health care should be more responsive in keeping people out of hospital in a shorter period than targeting the general population with prevention-based approaches.

In 2014, NHS England outlined plans to develop `new care models’ [21], supporting the creation of prototypes for integrating health and social care services [22] to be subsequently spread across England [6]. Introduced in 2015 and financed until March 2018, the Vanguard `New Care Models’ was a major innovation programme aiming to design and test prototypes for integrating health and social care services [22]. The scale of the Vanguard piloting programme was large: between 2015 and 2018, NHS England invested about £389 million on supporting vanguard sites to develop and evaluate new care models [23]. This covered a population of around 5 million – around 9 % of the entire population in England. The declared aspiration of the Vanguard programme was the development of `locally driven’ prototypes, which if successful could be rapidly spread across England.

Fifty local areas were selected across England to act as `vanguards’ for the five care models proposed: 14 multispecialty community providers (MCP); nine primary and acute care systems (PACS); six enhanced health in care homes (ECH); along with eight urgent and emergency care networks (UEC) and 13 acute care collaborations (ACC). MCP and PACS were similar population-based models aimed at moving specialist care out of hospitals into the community, by fostering a closer integration of General Practitioner (GP), hospital, community and social care services [24]. They were not specifically prescribed to be targeted to specific groups of patients but rather flexible to be adapted to the needs of the local population. ECH focused upon care home residents, mainly older people, and aimed at improving the quality and the coordination of health, care and rehabilitation services by promoting collaborations of the different institutions involved: the NHS, local authorities, the voluntary sector, carers and relatives [25]. MCP, PACS and ECH vanguards focused upon improving the integration between primary, secondary, community and social care, with the aspiration to provide more integrated care in the community [26]. UEC and ACC are different kinds of models, focused upon the better organisation of care provided by hospital and emergency services [27].

The initial objectives for Vanguard integrated care pilots were wide-ranging, reflecting the broad objectives often expressed by those supporting the implementation of a more integrated approach to health and social care [28]. The programme initially supported ‘bottom up’ innovation, with Vanguards encouraged to set their own objectives across a broad range of potential outcomes, guided by a number of loose evaluation questions set by NHS England. However, by the second year of the programme the official policy objective had shifted to the very narrow aim of reducing hospital activity, with funding for the final year contingent upon Vanguard sites achieving a reduction in emergency admissions and hospital bed days. This shift from broad to narrow objectives was experienced as problematic by those involved [27], but it was justified centrally by the argument that, after two years of operation, Vanguards should be ‘earning their passage’ by reducing hospital admissions (p. 47) [29].

Evidence so far from the Vanguards is emerging and mixed. Internal analyses from NHS England [30] are said to show that growth in emergency admissions in the 23 MCP/PACS Vanguards was around 6 % lower than in the rest of England, but the underlying analysis upon which this figure is based has not been published. The Improvement Analytics Unit (a partnership between NHS England and The Health Foundation) published detailed quantitative analysis of outcomes from six individual Vanguards [31], showing mixed results, with some Vanguards associated with increased emergency attendances and admissions to hospitals, whilst two of the Care Home Vanguards showed a reduction in these measures.

This paper is the first independent national evaluation of the efficacy of the three major (of the five proposed) Vanguard prototypes (MCP, PACS and ECH) in meeting the official programme objectives of reducing emergency admissions and total bed-days in hospital. Using national administrative data and a difference-in-difference analytical design we found that the Vanguard initiative slowed the rise in emergency admissions observed in England, especially in care home sites and in the third and final year of the programme. However, the introduction of these Vanguard prototypes was not associated with an overall reduction in total bed-days.

2. Materials and methods

2.1. Data

We obtained monthly counts of emergency admissions and bed-days from NHS England. These were aggregated by NHS England from patient-level Secondary Uses Service [32] data held in their national commissioning data repository. The experimental sites (Vanguards) were defined by GP practices that were active in England in March 2018 in the 9 PACS and 14 MCP and 6 ECH sites (see Supplementary Appendix 1) and were mapped by NHS England. We combined the two population-based models, PACS and MCP sites, in the same treatment group (from this point termed integrated care sites) as our associated fieldwork suggested that the two models were not significantly different in terms of their organisation, objectives and scope [24,27]. The care home (ECH) sites formed the other experimental group.

Activity in GP practices not in integrated care and care home sites was aggregated up to the level of the corresponding Clinical Commissioning Groups (CCGs). The unit of observation is therefore a set of practices that are part of a Vanguard site in the same CCG (31 integrated care sites and 6 care home sites), an entire CCG where there is no Vanguard, or a set of practices not part of a Vanguard site in a CCG where there is a Vanguard site (196 sites in the control group used for the main analysis). Our full time horizon is 60 months: 24 months before (April 2013–March 2015) and 36 months after the introduction of Vanguard (up to March 2018).

We use two outcome measures: 1) Emergency admissions are those with a `specific acute’ treatment function code. Better integrated care in the community might plausibly affect (preventable) emergency route into hospital, less plausibly elective admissions; 2) Total bed-days, including stays after both emergency and elective admissions. Better integration between hospital and community care might plausibly enable quicker discharge from the relatively expensive hospital setting, which can potentially apply to all admissions.

Monthly counts were matched to general practices’ registered population counts, made available through NHS Digital and aggregated to the level used for the analysis. To account for different population sizes, we analysed the emergency admissions rate per 1000 persons and total bed-days rate per 100 persons. Descriptive statistics are provided in Appendix 2.

2.2. Statistical analysis

We compared average (population unweighted) monthly outcomes observed in Vanguard integrated care and care home sites with the outcomes observed in non-Vanguard sites, before and after their introduction (the `intervention’). We smooth the monthly series by plotting estimates from local linear regressions [33].

To examine the net effect on outcomes (Y) attributable to the intervention, we used a least squares regression with two-way (i.e. site and period) fixed-effect estimator and a difference-in-differences analytic design [34,35]. We estimate variants of the following model:

$$Y_{it}={\alpha }_i+{\gamma }_t+{\beta }_j{D}_{it}+X_{it}\delta +{\epsilon }_{it}$$

for i = 1, 2,. . ., N and t = 1, 2,. . ., T where ${\alpha }_i$ and ${\gamma }_t$ are site and time (month) fixed effects, respectively. Site fixed-effects account for determinants of outcomes that differ across areas but do not change over time. Month indicators account flexibly for changes over time that are common across areas. The advantage of the two-way estimator strategy is that the area and month effects automatically control for a wide variety of difficult-to-observe factors that might be related to differences in observed outcomes between the Vanguard and non-Vanguard groups.

The index j identifies three groups: the control group of non-vanguard sites (j = 0) and the two treated groups comprising the population-based sites (j = 1) and the care home sites (j = 2). $D_{it}=j$ indicates if the programme j is active in sites i in the period t; otherwise $D_{it}=0$. The impact of the Vanguard programmes on outcomes is captured by the parameters ${\beta }_1$ and ${\beta }_2$. These are estimates of the net effect of the population-based models and of the care home models, respectively.

To account for factors that vary over time within area, we also control ($X_{it}$) for area-level population structure as the monthly proportion of population by age-groups (0–24; 25–64; 65 and older). To account for the skewness of the rates and the presence of zeros, all outcomes were transformed using the inverse hyperbolic sine transformation [36]. We used robust variance estimators to account for clustering of outcomes within CCG. All the analysis were performed in STATA/MP 14.2.

The identification of the net effects of the intervention on outcomes relies on the standard `parallel trends’ assumption [37]. That is, without intervention, time trends in the outcomes would have been parallel in the Vanguard and non-Vanguard groups. We tested whether the trends in the log-transformed outcomes were parallel in the pre-intervention period, by estimating area-fixed effect multivariate models with a linear time trend interacted with the treatment indicators.

The Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC) goodness-of-fit statistics were used to decide on the best model specification to the treatment of missing data and the pre-intervention observational time-window to be used (see Supplementary Appendix 4). The analysis under a balanced sample that discarded 12 areas with missing data for the outcomes or covariates in some months, obtained the best (lowest) AIC/BIC and is our main analysis.

We estimated the average effect over the whole post-intervention period and then separate effects for each post-intervention period comprising the first financial year (from April 2015 to end of March 2016), the second (from April 2016 to end of March 2017) and the third (from April 2017 to end of March 2018) following the intervention.

Our main analysis estimated the average net effect of the intervention over the population of all ages. We also conducted age-group specific analysis to assess whether the effect varied across age groups.

We checked the robustness of our findings to the potential for selection bias into the Vanguard programme by re-running the main model over restricted pre-intervention periods and non-Vanguard groups (see Supplementary Appendix 4).

We also tested the robustness of our findings to different definitions of the control group in two ways. We firstly excluded the parts of CCGs not in Vanguard models in the CCGs that were not covered by the Vanguard model in the CCGs that contained a Vanguard. This is because the non-participants in the same areas are unlikely to be suitable “controls” if they have purposefully selected out or if they experience positive or negative spillover effects. We also generated a control group more “similar” to the intervention group via propensity score matching so that, conditional on observable differences between comparator and intervention groups, the propensity of treatment is similar for both groups. We estimated the propensity to be included in the control group via a population weighted probit model group membership against area-level covariates, measured in 2015, that are independent from the intervention. These include average population proportions by age-groups (0–24; 25–64; 65 and older) and the CCG area-level Index of Multiple Deprivation [38], to account for the significant differences in these dimensions reported elsewhere [23]. We restricted control group membership to CCGs found to be more “similar” to the intervention groups and re-run our main models. However, the specification with the full control group provided the best goodness-of-fit statistics and were therefore reported in Section 3.

3. Results

Emergency admissions and total bed-day rates were lower in the control group than in the treated sites in the pre-intervention period and remained so in the post-intervention period (Fig. 1). While care home (ECH) sites follow a more similar pre-intervention trend with respect to the control groups, there is some evidence that population-based (PACS/MCP) sites were experiencing higher emergency admissions rates in the first three months of observation (April–June 2013) and just before the Vanguard call (November–December 2014). However, the null hypothesis of parallel trends was not rejected at conventional statistical levels for the Vanguard population-based (p-value = 0.739) and care home (p-value = 0.817) sites.

Fig. 1.

Emergency admission and total bed-days rates in the Vanguards and non-Vanguard sites, before and after the Vanguard introduction.

Notes: the lines represent locally weighted regressions (bandwidth = 0.2) of the raw monthly outcomes averaged by groups (scatter points) on time (pre- post-intervention). The grey area identifies the period between January 2015 (when the NHS invited individual organisations and partnerships to apply to become Vanguard) and March 2015 (when population-based and care home sites were selected).

Population-based sites experienced the highest total bed-day rates, followed by care home and by non-Vanguard sites. Despite the fact that pre-intervention trends on total bed-days rates differed slightly, in particular for care home sites just before the Vanguard call, the null hypothesis of parallel trends could not be rejected at conventional levels for the Vanguard population-based (p-value = 0.658) and care home (p-value = 0.181) sites.

In the post-intervention period, there is clear evidence of an increasing trend in emergency admissions rates among non-Vanguard sites, especially in the third year of the Vanguard programme, but the rates still remained lower than those observed in Vanguard sites. Emergency admissions rates increased in Vanguard population-based sites but remained stable in Vanguard care home sites. On the other hand, overall total bed-days rates declined slightly in non-Vanguard sites but remained almost unchanged in population-based and care home sites.

Table 1 provides the difference-in-differences estimates. Over the three-year period of the Vanguard programme, care home sites experienced an overall significant relative net reduction in emergency admissions of about 4.2 % (95 % CI: −8.2 % to −0.2 %). The net reduction in emergency admissions occurred mainly in the third year following implementation (−6.5 %, 95 % CI: −12.4 % to −0.6 %). For population-based sites, a significant net reduction (−3.1 %, 95 % CI: −6.2 % to −0.1 %) was found only in the last year of the Vanguard programme, not on average across the post-period. We did not however, find any significant net effect on total bed-days rates attributable to the integrated care and care home Vanguard schemes.

Table 1.

Difference-in-differences estimates of changes in emergency admissions and total bed-days rates attributable to the Vanguard integrated care and care home initiatives, by different post-implementation time windows.

	1st–3rd year	1st year	2ndyear	3rd year
Monthly emergency admission rate (per, 000 population)
Population-based	−0.004	0.011	0.015	−0.031**
	[−0.032, 0.024]	[−0.021, 0.043]	[−0.018, 0.047]	[−0.062, −0.001]
Care home	−0.042**	−0.018	−0.035	−0.065**
	[−0.082, −0.002]	[−0.050, 0.014]	[−0.076, 0.007]	[−0.124, −0.006]
Monthly total bed-days rate (per, 00 population)
Population-based	0.011	0.020	0.017	0.001
	[−0.017, 0.039]	[−0.012, 0.053]	[−0.019, 0.053]	[−0.029, 0.030]
Care home	0.013	0.019	0.013	0.008
	[−0.006, 0.032]	[−0.008, 0.046]	[−0.013, 0.039]	[−0.013, 0.029]

Notes: Figures represent estimated ${\beta }_j$ parameters of site and time fixed-effect specifications based on the balanced sample (223 sites: 31 population-based sites, 6 care home sites and 186 control sites) that use the full 24-months before implementation (see sub-Section 2.2 for details). Columns 2, 3 and 4 reported estimated net impact computed using a restricted post-intervention sample comprising 12, 12–24, 24–36 months after introduction. 95 % confidence intervals in brackets. Significance levels: * p < 0.10, ** p < 0.05, *** p < 0.01.

The analysis by age-group (Fig. 2) provided further insights. As expected, we find no significant net effects on emergency admission rates among the youngest segment of the population. For care home sites, we find overall (purple bar) a significant relative net reduction in emergency admission rates among the adult (−5.3 %, 95 % CI: −10.4 % to −0.2 %) and older population (−3.4 %, 95 % CI: −6.7 % to −0.1 %), in particular in the third year following implementation (red bar) (adult population: −7.8 % 95 % CI: −14.8 % to −0.7 %; older population: −6.4 %, 95 % CI: −10.6 % to −0.2 %). A net reduction on emergency admission rates emerged also in population-based sites in the third year following implementation among the adult (−3.3 %, 95 % CI: −7.1 % to 0.5 %) and older (−3.7 %, 95 % CI: −7.9 % to 0.5 %) populations.

Fig. 2.

Difference-in-differences estimates of changes in emergency admissions and total bed-days rates attributable to the Vanguard population-based and care home models, by age groups.

Notes: Bars represent the magnitude of the estimated ${\beta }_j$ parameters (horizontal lines identify 95 % associated confidence intervals) of site and time fixed-effect specifications based on the balanced sample (223 sites: 31 population-based sites, 6 care home sites and 186 control sites) that use the full 24-months before implementation (see sub-Section 2.2 for details) by age-groups. See Supplementary Appendix 5 for age-group specific time trends on outcomes and pre-intervention trend parallelism tests. ${\beta }_j$ parameters are provided in Supplementary Appendix 6

We found a significant net increase in total bed-day rates among the adult population in the first year following the introduction of the Vanguard population-based models (+4.4 %, 95 % CI: 0.6%–8.1%). We also found a significant net increase in total bed-days rates attributable to the Vanguard care home models for the same age group in the second year following implementation (+4.0 %, 95 % CI: 1.0%–6.9%).

4. Discussion

The main components of the Vanguard `New Care Models’ programme for the three prototypes we focussed on (MCP/PACS and ECH) were largely intended to reduce hospital utilisation by moving specialist care out of hospital into the community and by fostering coordination of health, care and rehabilitation services for care home residents.

4.1. Principal findings

Using national administrative data and a difference-in-difference analytical design we found that the Vanguard initiative slowed the rise in emergency admissions observed in England, especially in care home sites and in the third and final year of the programme.

Specifically, Vanguard sites experienced higher emergency admissions and total bed-days rates than non-Vanguard sites in the two years prior to their introduction. After their introduction, non-Vanguard sites experienced a sharp increase in emergency admissions, especially since 2017. Over the three years of the programme, a significant relative net reduction of −4.2 % in emergency admission rates was found only in the care home sites, with the bulk of the effect occurring among the adult and older population. The magnitude of the net reduction in the emergency admission rates in care home sites increased over time and became statistically significant in the third year after implementation. In that year, a significant reduction in emergency admission rates of −3.1 % also emerged for the population-based sites.

However, the introduction of the Vanguard prototypes was not associated with an overall reduction in total bed-days.

4.2. Limitations

Our study has four main limitations. Firstly, our estimates are valid under the assumption that the outcomes would have followed the same trend in the Vanguard and non-Vanguard groups in the absence of the programme. Under that assumption, the (conditional) difference we observe in the evolution of the outcome between those groups is attributable to the Vanguard integrated care programme. We found evidence of parallel trends prior to implementation in the Vanguard and non-Vanguard groups. We also undertook a series of analyses to assess the robustness of our findings to the treatment of missing values, the length of the pre-intervention period used and the definition of the control group. While the estimated net effects (reported in the Appendix 4) were only marginally influenced, the goodness-of-fit statistics worsened compared to our main analysis. Nonetheless we cannot rule out the possibility that confounding events, unrelated to the programme implementation, have affected one group but not the other in the post-implementation period. It remains possible that other coincident innovation programmes might have influenced our results.

Secondly, residual confounding may still occur should historic differences not fully identify whether the intervention sites are more or less likely to embrace the intervention. There were in excess of 260 [23,39] expressions of interest for becoming a Vanguard prototype, and only 50 were successful. Including unsuccessful applicants in the control group might have introduced bias in the analysis but we were unable to identify unsuccessful Vanguard applicants. Unobservable factors (omitted from the vector of controlling characteristics) might also have biased our estimated net effects.

Thirdly, our focus was on two outcomes: emergency admissions and total bed-days rates. These outcomes were chosen because the types of Vanguards analysed were typically expected to involve an emphasis on prevention and hospital admissions avoidance, and emergency admission and bed-days rates became the main outcome measures assessed by the programme in the second and third years [23,27]. We recognise that Vanguards, like other integration initiatives, were initially set up with a much broader range of objectives, but it has not proved possible to robustly evaluate Vanguards against these as the data are not available and local objectives were rarely set in such a way as to be evaluable [40].

Finally, leading on from this, our analysis aimed at assessing whether the programme as a whole had a measurable impact on service use (i.e. hospital utilisation). We were not able to link the outcomes to specific initiatives within the Vanguards because the initiatives implemented were heterogeneous and poorly defined [27]. It is therefore difficult to know what the drivers of the improvements in outcomes that we have shown might be. For example, funding for the final year of the Vanguard programme was only available if progress could be shown against measures of emergency admissions, and this may have driven improvements in coding alongside efforts to reduce admissions. Our qualitative work has shown that this linkage did lead some Vanguards to discard some of their initiatives which were not directly focused upon admissions [27], and it may be that this sharper focus drove improvements. Further research is required in order to explore in more depth whether improvements in outcomes derive from specific service innovations or whether they are driven by a more general improvement in collaboration and communication between sectors consequent upon engagement in a pilot such as this [41].

4.3. Strengths in relation to other studies

The 2019 NHS Long Term Plan announced that “[f]ollowing three years of testing alternative models in the Five Year Forward View through integrated care ‘Vanguards’ […], we now know enough to commit to a series of community service redesigns everywhere” (p.13) [6]. Evidence of reduced emergency admissions attributable to the population-based (PACS and MCP) programmes underpinning this statement was taken from an internal NHS evaluation. We have used a dataset that is very similar to that used by the NHS team. In Supplementary Appendix 3, we replicated their results closely, using the same approach as they had taken of analysing rolling 12-months moving averages on raw outcomes indexed to the month of intervention. Our analysis is more standard in considering how the programme affected the level of hospital utilisation, includes a more extensive statistical analysis, and extends to the care home (ECH) scheme, showing these to be most effective on the emergency admissions measure over a three year period.

In addition to these national studies there have been evaluations of individual Vanguards. A study conducted as part of the national new care models evaluation included an evidence synthesis of locally-commissioned evaluations [40]. The synthesis summarised a significant grey literature of 108 local evaluation reports. It concluded that many local areas struggled to design and carry out robust evaluation designs, with particular problems in collecting relevant data [23,40]. Robust evaluations of some individual Vanguards have been undertaken by The Improvement Analytics Unit have found mixed results [31]. Their evaluation of four specific care home sites has shown significant reductions of emergency admission from care home residents in three sites (Rushcliffe, Wakefield and Nottingham City) but inconclusive results for the other site (Sutton) [42]. Our overall evaluation of all care home sites shows that on average there is a relative net reduction in hospital admissions for this model of integration, more beneficial than a population-based model over the three years follow-up.

4.4. Meaning of the study

Providing more integrated care for patients is not straightforward. In this paper we have tested the NHS England Vanguard pilots (PACS/MCP and ECH) against their official objective (reducing hospital utilisation) and shown that, whilst there were modest reductions in emergency admissions in Vanguard sites, these were seen mostly in the pilots which targeted elderly people living in care homes. The results suggest that focused interventions on a concentrated population of care home residents who are vulnerable to high levels of emergency admissions are likely to be most effective in keeping people out of hospital over the time period analysed [2,43]. Moreover, our results clearly show no net changes in the first year and relative reductions in emergency admissions only became significant after three years. This suggests that, even in a relatively straightforward organisational context where initiatives can be applied to whole resident population, achieving desired results takes time. Moreover, absolute emergency admissions were not reduced, but the effect was driven by the vast increase in emergency admissions in the control group (up to intervention group pre-intervention levels), which was contained in the intervention models. Finally, the modest net reduction in emergency admissions which we have shown was achieved with the help of considerable additional funding and a dedicated support programme for Vanguard sites. Our results do not assess the cost-effectiveness of this approach to integrating care.

4.5. Policy implications & future research

The recent NHS Long Term Plan announced the “commit[ment] to a series of community service redesigns everywhere” (p. 13) [6], spreading the innovative practices piloted with the Vanguard initiative. The way in which the spread will be operationalised is still far from clear and not well understood. Our research sheds lights on the effects of such initiatives on those sites involved in the programme. It cautions that the effect is likely to be heterogeneous among sites and initiatives and unlikely to be detectable in the short-term. This suggests that those seeking to introduce similar initiatives in other policy contexts should take a longer term view and refrain from judging policy success too soon. It also suggests that the approach taken by the Vanguard programme – of making funding contingent upon the achievement of measurable impact against a narrow set of prescribed metrics within two years – was probably mistaken. Vanguard sites received national support and considerable additional funding, and it remains to be seen if the beneficial impacts will be replicated without this funding and the associated support.

Contributors and acknowledgements

MM did the econometric analysis, and led the design and the writing of the paper. MS contributed to the design, oversaw the econometric analysis, and contributed to writing the paper. KC contributed to the design of the study and the writing of the paper. JW, AC and CT advised on the design and contributed to the writing of the paper. CT contributed to the paper in his previous role as the head of Operational Research & Evaluation Unit for NHS England. The authors are grateful to the wider team members for their suggestions in all phases of the research.

Funding

This paper is based on independent research commissioned and funded by the NIHR Policy Research Programme the new models of care Vanguard programme in England: national programme evaluation, PR-R16-0516-22001. The views expressed in the publication are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health and Social Care, arm’s length bodies or other government departments.

Declaration of Competing Interest

None.

Appendix A. Supplementary data

The following is Supplementary data to this article: