Impact of a primary care partnership programme on accident and emergency attendances at a regional hospital in Singapore: a pilot study

INTRODUCTION

Crowding[1] at accident and emergency (A&E) departments occurs whenever the demand for emergency medicine services exceeds the available resources. It can compromise the quality of care to patients, which may, in turn, have an adverse impact on their health outcomes.[2,3,4,5] It has also been reported that A&E crowding is detrimental to morale,[6] productivity[3,7] and training[8] of A&E staff. Over the years, this mounting problem has affected the healthcare systems of many countries, including developed countries such as Australia, the United Kingdom and the United States.[9,10,11,12] Given the prevalence of A&E crowding across the world, studies that aim to understand the causes of A&E crowding or identify measures that can potentially alleviate the problem[13,14,15,16,17,18] have attracted extensive interest from the research community.

One widely reported mitigating measure to reduce A&E crowding[17,18,19,20] involves the management of A&E demand by patients whose medical conditions do not warrant an A&E visit; this is commonly known as 'inappropriate attendance' (IA). Patients who contribute to IA at A&E are usually those with non-emergency health problems that are unlikely to require admission and can be managed by primary care physicians (PCPs). The presence of such patients in a crowded A&E may hinder healthcare providers from attending to other patients who require emergency medicine service in a safe and timely manner.

Several studies have evaluated the effectiveness of various measures aimed at reducing A&E utilisation, including those pertinent to IA reduction.[17,18] In a systematic review,[17] these IA reduction measures were broadly categorised into three types, namely primary care linkage, A&E diversion and financial penalties or cost-sharing. Primary care linkage entailed programmes[21,22,23,24,25] aimed to reduce IA by strengthening linkages of target population to primary care network and/or providing care coordination of A&E attendees with PCPs. In contrast, A&E diversion involved measures aimed to direct patients who visit the A&E inappropriately away from the A&E either before or at the time of A&E triage.[26] Lastly, financial penalties or cost-sharing programmes[27,28,29] involved modifications of co-payment schemes for visits to A&E and other healthcare service providers such as PCPs. Overall, results on the effectiveness of the aforementioned IA reduction interventions were limited and mixed.

In Singapore, GPFirst programme was introduced at the start of 2014 in the eastern part of the island. Patients who consulted any of the 192 private general practitioners (GPs) participating in the programme received a subsidy of SGD 50 for the A&E consultation fee if they were subsequently referred to the A&E of Changi General Hospital (CGH), the only public hospital in the eastern part of Singapore. In addition, the same patients who were referred to the A&E via GPFirst would receive an automatic priority classification at the A&E and be attended to sooner than patients with less urgent medical conditions. GPFirst distinguishes itself from the existing IA reduction measures in two ways. First, unlike other reported primary care linkage-related measures,[21,22,23,24,25] which involve extension of the operating hours of GP clinics and targeting specific patient groups such as elderly patients, those with anxiety or without insurance, GPFirst was designed to encourage a broader group of patients with mild to moderate conditions to visit a GP instead of presenting themselves at the A&E as self-referred patients. Second, GPFirst had a financial component in the form of a subsidy of SGD 50 on A&E consultation fees for subsequent GPFirst referral to the CGH A&E; this subsidy, which is about the bill size of a GP visit payable by a patient in Singapore, ensures that the patient would be not too financially compromised if they have to visit the A&E after being seen at a participating GPFirst clinic.

METHODS

This was a retrospective cohort study that analysed variables extracted from the administrative databases of the A&E at CGH, Singapore. The study population included all CGH A&E self-referred attendances occurring between 1 January 2011 and 31 December 2014. Self-referred A&E patients who arrived by ambulances or were personnel from the Singapore Armed Forces (SAF) were excluded from our study, as they were not the target population of the GPFirst primary care partnership programme. It must be noted that all 18-year-old male Singapore citizens and permanent residents are required statutorily to undergo two years of compulsory service in the uniformed services, with the majority of them serving in the SAF. Existing financing policy allows SAF personnel to visit the A&Es of public hospitals in Singapore without requiring them to pay the A&E attendance fees. No ethics approval was required, as the study was an evaluation of a new healthcare programme using de-identified data.

We used interrupted time series (ITS) analysis to test whether the roll-out of GPFirst has any effect on self-referred attendances at CGH A&E. The ITS design has been reported as one of the strongest evaluative designs,[30] as it allows for study of the intervention effect while controlling for pre-intervention trends.[31] Moreover, ITS analysis can enhance the likelihood of detecting a change resulting from an intervention because it accounts for the full range of longitudinal data instead of collapsing all data to single before-and-after intervention time points. Like most other quasi-experimental designs, the primary limitation of ITS analysis is the possibility that factors other than the intervention have affected the targeted outcome, and are not fully accounted for by the pre-intervention trend. Nevertheless, this limitation can be minimised by the inclusion of a control group or control outcome,[30] so that there are both a before-after comparison and an intervention-control group comparison. As it was not operationally feasible to find another public A&E in Singapore with self-referred patients who shared similar profiles (i.e. demographics and health-seeking behaviour) as those attending the CGH A&E to be the control group in this study, we employed time series forecasting,[32] which is a well-established forecasting technique to estimate the control time series that would represent the monthly self-referred attendances at CGH A&E in the absence of GPFirst. With this control time series and the actualised self-attendance records in the first year of GPFirst roll-out, we could then evaluate the impact of the GPFirst programme on self-referred A&E attendances. To this end, it must also be highlighted that the aforementioned forecasting technique was chosen based on a comparative study, which aimed to determine the time series forecasting technique that yielded the best accuracy in forecasting monthly A&E attendances in the next 12 months. Details of this study are summarised in the Appendix.

RESULTS

Our comparative study showed that an auto-regressive integrated moving average (ARIMA) was most accurate among the three investigated methods in forecasting monthly A&E self-referred attendances in 2013, based on the past 24-month records. Therefore, ARIMA was employed to forecast the monthly A&E self-referred attendances in 2014 based on the actual monthly attendance records from 2011 to 2013 [Figure 1]. It was evident that the forecasted monthly attendances in 2014 were higher than the actual monthly attendances. On an annual basis, actual self-referrals of the A&E declined by 8.7% in 2014. This decline was 1.5 times higher than that observed in 2013 [Table 1]. Moreover, the actualised attendance reduction in 2014 relative to 2013 was significantly more than what was forecasted in 2014 by 278 patients per month or 3,336 patients per year [Table 2]. Given the non-overlapping 95% confidence interval (CI) of actualised and forecasted monthly reduction in self-referred attendances in Table 2, it appeared that the introduction of GPFirst could have contributed to a significant drop in the actual A&E self-referred attendances in 2014, with the magnitude of annual decline in the same year being sharply higher than what was observed in the preceding one year.

Figure 1.

Graphs shows the monthly accident and emergency department self-referred attendances from 2011 to 2014 with forecasted attendances using an auto-regressive integrated moving average (ARIMA) in 2014.

Table 1.

Annual reduction in self-referred attendances at the accident and emergency department.

	n (%)
	2013	2014
Annual attendances	93,381	85,264
Annual decline in attendances	5,717 (5.8)	8,117 (8.7)

Table 2.

Mean monthly reduction in self-referred attendances at the accident and emergency department in 2014 relative to 2013.

	Monthly mean (95% confidence interval)
Actualised reduction in self-referred attendances in 2014	676.4 (514, 839)
Forecasted reduction in self-referred attendances in 2014	398.3 (379, 417)

DISCUSSION

It is apparent from Figure 1 that there was a general trend of declining monthly actual A&E self-referred attendances year to year from 2012. This decline could potentially be attributed to both external factors and new hospital-initiated measures. One such measure was the introduction of publicity brochures[33] within the CGH A&E premises in 2013 to raise awareness among A&E patients on the availability of GP clinics in the neighbourhood of the A&E. The effect of such a measure, together with other external factors that came into play prior to 2014, would have been accounted for when the monthly A&E self-referred attendances in 2014 were forecasted by ARIMA. To the best of our knowledge, GPFirst was the only new hospital-initiated programme in 2014 that would have a direct impact on A&E self-referred attendances. Moreover, in an unpublished report by Ministry of Health (MOH), which compared the self-referred attendances across all adult public A&Es, the annual decline of these attendances in the CGH A&E in 2014 was more than 3.5 times higher than that of all other A&E departments combined. Thus, the current available evidence appeared to suggest that the significantly higher annual reduction in actualised attendances in 2014 compared to the forecasted reduction in the same year could potentially be attributed to the roll-out of GPFirst in 2014.

To date, there have been reports in the existing literature of financial penalties or cost-sharing programmes[27,28,29] whose primary goal was diverting patients with non-emergency conditions away from the A&E. Essentially, these programmes impose additional financial burden on participating patients whenever they visited the A&E by requiring them to make a co-payment. The results on the effectiveness of these programmes in reducing A&E utilisation have been mixed. In contrast, the GPFirst programme was designed to alleviate the financial burden of patients who were referred to the A&E after they had visited a participating GP clinic by offering a subsidy of SGD 50 in A&E consultation fee for their visit to the CGH A&E. Instead of imposing a financial penalty to discourage patients with non-emergency conditions from visiting the A&E, our results appeared to suggest that the offer of a financial subsidy could be an effective intervention to nudge more patients to visit a GP first, since these patients were less likely to feel financially penalised if they were subsequently referred to the A&E.

The impact of GPFirst on A&E self-referred attendances not only depends on the programme design (which includes a quantum of subsidy and benefits of referral by participating GP clinics), but also hinges heavily on awareness of the programme among the public. Given the limited publicity campaign of the GPFirst programme in its first year, it was likely that the full impact of this scheme on CGH A&E self-referred attendances might not have been realised within the first year of its roll-out. Thus, it is crucial that more effort is put into improving awareness of GPFirst among residents within the catchment area of CGH A&E, particularly those who are likely to visit the A&E with non-emergency conditions. A recent study has reported that factors associated with attendances for non-emergency conditions among A&E patients included gender, ethnicity, referral source, hours of visit, nationality and history of being frequent emergency department visitors.[34] The same study also found that the odds of attending the A&E with non-emergency conditions were higher among attendees who were younger, self-referred or who contributed to at least one attendance for non-emergency conditions in the previous year. With our understanding of the profiles of residents at higher risk of visiting the CGH A&E for non-emergency conditions, subsequent publicity efforts for GPFirst could be rationalised to primarily target the higher-risk residents to increase their awareness of the scheme so that they will less likely contribute to self-referred attendances at A&E for non-emergency conditions in future.

In conclusion, the GPFirst programme is a novel IA-reduction measure that distinguishes itself from existing measures by incorporating both primary care linkage and a financial component to nudge more patients with mild to moderate conditions to visit a GP instead of using the A&E as their first stop for care. Our pilot study of GPFirst offers invaluable insights on the programme's potential impact. Current available evidence seems to suggest that GPFirst could have contributed to a significant reduction in A&E self-referred attendances at CGH in its first of year of implementation. If targeted publicity campaigns can be organised to widen awareness of this scheme among the public living within the CGH A&E catchment area, a greater reduction in A&E self-referred attendances could potentially be achieved.

Financial support and sponsorship

Nil.

Conflicts of interest

How CH is a member of the SMJ Editorial Board, and was thus not involved in the peer review and publication decisions of this article.

APPENDIX

Comparative Study of Forecasting Techniques

Time-series forecasting is a well-established research field used across various sectors, which uses past and present values of one or more time series to predict future values of the time series of interest. The popular techniques for time-series forecasting include simple moving average, exponential smoothing (ETS), Auto Regressive Integrated Moving Average (ARIMA), Seasonal ARIMA (SARIMA), artificial neural networks (ANN). Since these techniques are likely to differ to in their accuracy of forecasting A&E attendances, and it is not possible to evaluate the accuracy of all time-series forecasting techniques in existing literature, we decided to evaluate the A&E attendance forecasting accuracy of only three commonly reported techniques in this paper.

This first technique is ARIMA which is a class of model that captures a suite of different standard temporal structures in time series data. These temporal structures include those pertinent to dependent relationship between an observation and some number of lagged observations. ARIMA is a widely used technique for forecasting time-series data in the healthcare sector and beyond. In this study, we used auto.arima function within R to develop the ARIMA model. This package returns the best ARIMA model based on either Akaike Information Criterion (AIC) or Bayesian Information Criterion (BIC) and it auto-computes the model parameters (i.e. p, q and d) as appropriate for the corresponding AIC/BIC. The second technique to be evaluated in our study is ETS. The latter generates forecasts-based formulae that weighs recent observations more heavily than the past ones. ETS has been applied extensively across fields like aviation, healthcare, finance, telecommunications, electronics, etc. In our study, we employed the exponential smoothing algorithm which provides a state space framework for exponential smoothing with proper underlying stochastic formulation. This algorithm accounts for the full suite of exponential smoothing methods and chooses the one that best fits the data. The final forecasting technique used for evaluation in this study is ANN. The latter can model any type of relationship in the data with through data mining with no prior assumptions about the model. The most popular neural network models for time-series forecasting is the feedforward network model. In this work, extreme learning machines (ELM) was employed for fitting and forecasting the time series data. Prior study has reported that this ELM method was thousands of time faster than the traditional feedforward network learning methods like back propagation algorithm.

In this comparative study, the monthly A&E self-referred attendance data from January 2011 to December 2013 was first employed to evaluate the accuracies of three selected methods in forecasting A&E attendances. The forecasting method that performed the best in forecasting ahead the 12 monthly A&E attendances in 2013 based on past 24-month attendance records with lowest error would then be employed to predict the 12 monthly A&E attendances in 2014 (i.e. control time series) when the aforementioned primary care partnership program was first introduced. The accuracy of each of the above three forecasting methods was measured by computing two commonly used performance metrics, namely the mean absolute percentage error (MAPE) as well as root mean square error (RMSE). MAPE allows direct comparison of forecast accuracy across multiple time series while RMSE is the standard deviation of the residuals and measures how spread out these residuals are from the line of best fit. If and are the observed and forecasted monthly attendances at time t respectively, then the MAPE and RMSE over a period of interest when there are n equally spaced observation time points are defined as below:

RStudio was used in all forecasting computations in this study.

Results

Using the monthly A&E self-referred attendances in 2011 to 2012, the MAPE and RMSE of each of aforementioned three forecasting techniques in projecting the monthly attendances in 2013 were summarized in Supplementary Table I. Evidently, ARIMA outperformed the other two techniques, as illustrated in Supplementary Figure 1. The monthly self-referred attendances forecasted by ARIMA were closer to respective actual attendances when compared to ETS and ANN. Several studies which involved forecasting of A&E attendances (34–36,53) have reported MAPE in the range of 2 to 17&. Since the MAPEs of the three selected forecasting techniques were closer to the lower end of the aforementioned range, we concluded that the accuracies of the selected forecasting models were comparable to those of the better ones in the existing literature. With MAPE of ARIMA being smallest among the three selected forecasting techniques, it was subsequently employed to forecast the monthly A&E self-referred attendances in 2014, when GP First was first rolled out.

Supplementary Table I.

Performance of the three forecasting methods

Forecasting Method	MAPE (%)	RMSE
ARIMA	3.0	301
Exponential Smoothing (ETS)	3.2	296
Artificial Neural Networks (ANN)	5.0	450

Supplementary Figure 1.

Forecasted monthly self-referred A&E attendances in 2013 using three forecasting techniques ARIMA, ETS, ANN.