Statistical tools used for analyses of frequent users of emergency department: a scoping review

Abstract

Objective

Frequent users represent a small proportion of emergency department users, but they account for a disproportionately large number of visits. Their use of emergency departments is often considered suboptimal. It would be more efficient to identify and treat those patients earlier in their health problem trajectory. It is therefore essential to describe their characteristics and to predict their emergency department use. In order to do so, adequate statistical tools are needed. The objective of this study was to determine the statistical tools used in identifying variables associated with frequent use or predicting the risk of becoming a frequent user.

Methods

We performed a scoping review following an established 5-stage methodological framework. We searched PubMed, Scopus and CINAHL databases in February 2019 using search strategies defined with the help of an information specialist. Out of 4534 potential abstracts, we selected 114 articles based on defined criteria and presented in a content analysis.

Results

We identified four classes of statistical tools. Regression models were found to be the most common practice, followed by hypothesis testing. The logistic regression was found to be the most used statistical tool, followed by χ2 test and t-test of associations between variables. Other tools were marginally used.

Conclusions

This scoping review lists common statistical tools used for analysing frequent users in emergency departments. It highlights the fact that some are well established while others are much less so. More research is needed to apply appropriate techniques to health data or to diversify statistical point of views.

Strengths and limitations of this study.

• First overview of statistical tools used in frequent users analysis.

• Follows a well-defined methodological framework in an extensive body of literature.

• Quality assessment is not performed in a scoping review.

• Studies in other languages than English or French might have been missed.

Background

Emergency department (ED) ‘frequent users’ are a sub-group of ED users that make repeated, multiple visits during a given amount of time. Though there is no consensus about definition for frequent users, thresholds in the literature range from two to more than 10 ED visits per year,^{1 2} while the most common one is more than four ED visits per year.^{1 2} Frequent users represent a small proportion of ED users but account for a large number of visits.^3–5 They often display complex characteristics such as low socioeconomic status combined with physical and mental health issues.⁶ As such, their ED use is considered suboptimal,⁷ as the best strategy would be to identify those patients at an earlier stage in their health problem trajectory, in order to treat them more efficiently.⁸ Furthermore, frequent users’ visits may lead to overcrowding in EDs and decreased quality of care.² Identifying factors that best describe those users and predict their ED use is therefore an essential task to improve ED care as well as frequent users’ health problems. Adequate statistical tools are needed to that end. Although they are numerous, no literature review has been published yet about statistical tools used for analysing ED frequent users. Therefore, the aim of our study was to draw up a list of statistical tools used in identifying variables associated with frequent use or predicting the risk of becoming a frequent user.

Methods

In order to list the statistical tools used in describing variables associated with and prediction of frequent ED use, we conducted a scoping review. We followed the 5-stage methodology of Arksey and O’Malley⁹ adapted by Levac et al.¹⁰ The methodological framework of a scoping review allows ‘mapping rapidly the key concepts underpinning a research area and the main sources and types of evidence available’,¹¹ thus allowing us to identify gaps in the literature and future research opportunities.

Stage 1: Identifying the research question

We defined our research question as follows: What statistical tools are used in the identification of variables associated with frequent ED users and in their prediction?

Stage 2: Identifying relevant studies

We searched PubMed, CINAHL and Scopus databases in February 2019, using search strategies developed with the help of an information specialist (see the online supplementary appendix for the complete search strategy). Keywords included variants of ‘frequent users’, ‘emergency departments’ and ‘statistical tools’.

There were no restriction regarding the population age or sex, health conditions, study period or country.

Stage 3: Study selection

Articles written in French or in English were included using the following criteria:

• The study must focus on frequent users of EDs (studies focusing on re-visits or on frequent visits other than in EDs were excluded).

• The study must have an explicit definition of frequent users, such as four visits in 1 year (reviews were excluded).

• The study must use at least one statistical tool that is classified as inferential (not descriptive, as defined by The Cambridge Dictionary of Statistics¹²), such as hypothesis tests, regression models, decision trees or others.

• The study’s objectives must include identifying variables associated with frequent use or predicting the risk of becoming a frequent user.

We collected 4534 potential abstracts (figure 1). Of those, 32 were duplicates and 4344 were excluded by an investigator (YC) after reading the title and the abstract. At this stage, studies were discarded if it was explicit from the title and the abstract that they were unfit for the scoping review (for instance studies about frequent use of inpatient services, systematic reviews, etc). In case of uncertainty, studies were kept for complete reading. Then, YC and FRH or ID independently evaluated the remaining 158 full text articles, of which 109 matched the above criteria. A third evaluator was consulted in case of discrepancy. Reasons for exclusion were: not in French or English (1), duplicate (3), systematic review (4), no inferential statistics (5), no explicit definition of frequent users (5), focus not on ED (14), no description or prediction of frequent users (17). A reference search among the references of the 109 included articles yielded five relevant articles. Thus, 114 articles were included in this study, of which YC, ID and MB examined the full texts.

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram. ED, emergency department.

Stage 4: Charting the data

YC, MB and ID independently extracted the corresponding data. Reported characteristics were the first (two) author(s), the publication year, the study location, the population, the frequent users’ definition, the objectives, the sample size and the statistical tools used concerning the research question.

Stage 5: Collating, summarising and reporting the results

The results are reported via a content analysis.¹³

Patient and public involvement

Patients or public were not involved in this study.

Results

The studies’ main characteristics are presented in table 1. Out of 114 studies, 65 were conducted in the USA, 17 in Canada and 8 in Australia (figure 2). The various statistical tools were classified into four main categories: regression, hypothesis testing, machine learning and other tools.

Table 1.

Main characteristics of the 86 included studies

Authors, year and country	Population	Frequent user definition	Study main objectives	Study cohort size	Statistical tools used
Aagaard et al 201415 Denmark	Psychiatric	≥5 visits per year	To identify predictors of frequent use of a psychiatric ER.	8034	Logistic regression
Adams et al 200016 Australia	Adults with asthma	≥2 visits per year	To identify whether factors other than severity and low socioeconomic status were associated with this disproportionate use.	293	Logistic regression
Ahn et al 201891 Australia	General population aged≤70 years	≥4 visits per year	To examine the characteristics of frequent visitors to EDs and develop a predictive model to identify those with high risk of a future representations to ED among younger and general population.	170 134	Maximum likelihood monotone coarse classifier algorithm Logistic regression Mixed-effects model
Alghanim and Alomar 201517 Saudi Arabia	All	≥3 visits per year	To determine the prevalence of frequent use of EDs in public hospitals, to determine factors associated with such use, and to identify patients’ reasons for frequent use.	666	Χ2 test Logistic regression
Alpern et al 201418 USA	All	≥4 visits per year	To describe the epidemiology of and risk factors for recurrent and high frequency use of the ED by children.	695 188	Negative binomial regression Logistic regression Generalised estimating equations
Andren and Rosenqvist 1987113 Sweden	All	≥4 visits per year	To follow a cohort of heavy ED users with regard to changes in medical and psycho-social profiles and ED use and to identify predictors for a maintained high use of ED services and the relationship between changes in access to social networks and utilisation of medical care services.	232	Decision trees Linear regression
Andrews et al 201892 USA	Medicaid enrollees with addiction	≥2 visits during a 2 year-period	To examine whether the number of outpatient addiction programmes accepting Medicaid in South Carolina counties is linked to repeat use of the ED for addiction-related conditions.	2401	Logistic regression
Arfken et al 200419 USA	Psychiatric	≥6 visits per year	To identify risk factors for people who use psychiatric emergency services repeatedly and to estimate their financial charges.	74	Logistic regression
Batra et al 201783 USA	Women	≥3 visits per 3 months	To use population data to identify patient characteristics associated with a postpartum maternal ED visit within 90 days of discharge after birth.	1 071 232	Logistic regression Receiver operating characteristic curve
Beck et al 2016105 UK	Mental health	≥3 visits in 3 months	To statistically identify characteristics associated with a shorter time to re-attendance and a higher number of overall ED admissions with a Mental Health Liaison Service referral.	24 010	Cox regression Negative binomial regression
Bieler et al 201220 Switzerland	All	≥4 visits per year	To identify the social and medical factors associated with frequent ED use and to determine if frequent users were more likely to have a combination of these factors in a universal health insurance system.	719	Wilcoxon rank-sum test Logistic regression
Billings and Raven 201321 USA	All	≥3 visits per year ≥5 visits per year ≥8 visits per year ≥10 visits per year	To examine whether it is possible to predict who will become a frequent ED user with predictive modelling and to compare ED expenditures to total Medicaid services expenditures.	212 259	Logistic regression
Birmingham et al 2017124 USA	All	≥4 visits per year	To characterise frequent ED users, including their reason for presenting to the ED and to identify perceived barriers to care from the users’ perspective.	1523	t-test Χ2 test Wilcoxon rank-sum test
Blair et al 2018112 UK	Children	≥4 visits per year	To describe the sociodemographic and clinical characteristics of preschoolers who attend ED a large District General Hospital.	10 169	Χ2 test Poisson regression Mann-Whitney U test
Blonigen et al 201782 USA	Veteran psychiatric	≥5 visits per year	To identify patient-level factors associated with ED use among veteran psychiatric patients and to examine factors associated with different subgroups of ED users including ‘high utilisers’.	226 122	Χ2 test Zero-truncated negative binomial regression Logit regression
Boyer et al 201122 France	Psychiatric	≥6 visits per year	To examine characteristics of frequent visitors to a psychiatric emergency service in a French public teaching hospital over 6 years.	1285	Logistic regression
Brennan et al 201423 USA	Psychiatric	≥4 visits per year	To assess the incidence of psychiatric visits among frequent ED users and utilisation among frequent psychiatric users.	788 005	Kruskal-Wallis test Mann-Whitney U test Logistic regression
Buhumaid et al 201524 USA	Psychiatric	≥4 visits per year	To evaluate demographic factors associated with increased ED use among people with psychiatric conditions.	569	Logistic regression
Burner et al 201884 USA	People with diabetes	≥3 visits per 6 months	To describe characteristics of patients with poorly controlled diabetes who have high ED utilisation, and compare them with patients with lower ED utilisation.	108	Logistic regression
Cabey et al 201425 USA	All	90th percentile	To define the threshold and population factors associated with paediatric ED use above the norm during the first 36 months of life.	16 664	Non-parametric distribution fit Logistic regression Bootstrap Clopper-Pearson method
Castner et al 201526 USA	People with psychiatric and substance abuse diagnoses	≥3 visits per year	To stratify individuals by overall health complexity and examine the relationship of behavioural health diagnoses (psychiatric and substance abuse) as well as frequent treat-and-release ED utilisation in a cohort of Medicaid recipients.	56 491	Logistic regression
Chambers et al 201327 Canada	Homeless	90th percentile	To identify predictors of ED use among a population-based prospective cohort of homeless adults in Toronto, Ontario.	1165	Logistic regression
Chang et al 201428 USA	Psychiatric	≥4 visits per year or ≥3 visits during two consecutive months	To identify the patient characteristics associated with frequent ED use and develop a tool to predict risk for returning in the next month.	863	Χ2 test Logistic regression
Christensen et al 2017107 USA	All	≥4 visits per year	To determine the patient characteristics and healthcare utilisation patterns that predict frequent ED use (≥4 visits per year) over time to assist healthcare organisations in targeting patients for care management.	13 265	Zero-inflated Poisson regression Receiver operating characteristic curve
Chukmaitov et al 201229 USA	People with ambulatory care-sensitive conditions	≥4 visits per year	To study characteristics of all, occasional and frequent ED visits due to ambulatory care-sensitive conditions.	4 914 933 (number of visits)	Logistic regression
Colligan et al 201630 USA	Medicare beneficiaries	≥4 visits per year	To examine factors associated with persistent frequent ED use during a 2 year period among Medicare beneficiaries.	5 400 237	Logistic regression Wald test
Colligan et al 201796 USA	Medicare beneficiaries	≥4 visits per year	To examine factors related to frequent ED use in a large, nationally representative sample of Medicare beneficiaries.	5 778 038	Χ2 test Analysis of variance Logistic regression Wald test
Cunningham et al 201797 USA	All	95th percentile ≥10 visits per year	To compare frequent and infrequent ED visitors' primary care utilisation and perceptions of primary care access, continuity and connectedness and to examine primary care utilisation and perceptions as predictors of ED use.	1113	t-test Χ2 test Logistic regression
Das et al 201731 USA	Children with asthma	≥2 visits per year	To explore the predictability of frequent ED use among children with asthma using data from an EHR from one medical centre.	2691	Wilcoxon rank-sum test Χ2 test LASSO logistic regression Regularised logistic regression Decision trees Random forests Support vector machines
Doran et al 201333 USA	All	2–4 visits per year 5–10 visits per year 11–25 visits per year ≥25 visits	To identify sociodemographic and clinical factors most strongly associated with frequent ED use within the Veterans Health Administration nationally.	930 712	Logistic regression
Doran et al 201432 USA	All	≥3 visits per year	To examine patients’ reasons for using the ED for low-acuity health complaints, and determine whether reasons differed for frequent ED users versus non-frequent ED users.	940	Logistic regression
Doupe et al 20124 Canada	All	≥7 visits per year	To identify factors that define frequent and highly frequent ED users.	105 687	Logistic regression Receiver operating characteristic curve
Fernandes et al 200334 Brazil	All	≥3 visits per year	To identify characteristics related to poor disease control and frequent visits to the ED to apply appropriate clinical management.	86	Χ2 test Logistic regression
Flood et al 201785 USA	Children	≥4 visits per year	To identify factors associated with high ED utilisation among children in vulnerable families.	2631	Χ2 test t-test Logistic regression
Freitag et al 200577 USA	People with chronic daily headache	≥3 visits per year	To examine the characteristics of chronic daily headache sufferers who use EDs and identify factors predictive of ED visits.	785	Wilcoxon rank-sum test t-test Χ2 test Poisson regression Negative binomial regression Logistic regression
Friedman et al 200978 USA	People with severe headache	≥4 visits per year	To determine frequency of ED use and risk factors for use among patients suffering severe headache.	13 451	Markov chain Monte Carlo imputation Logistic regression
Frost et al 201735 Canada	All	≥3 visits per year	To determine whether machine learning techniques using text from a family practice electronic medical record can be used to predict future high ED use and total costs by patients who are not yet high ED users or high cost to the healthcare system.	43 111	Logistic regression
Girts et al 2002114 USA	People with a diagnosis of psychosis	≥2 visits per 6 months	To develop a predictive model of ED utilisation for patients where a diagnosis of psychosis could be identified from a claim associated with a medical service provider visit.	764	t-test Linear regression
Grinspan et al 201536 USA	People with epilepsy	≥4 visits per year	To describe (1) the predictability of frequent ED use (a marker of inadequate disease control and/or poor access to care), and (2) the demographics, comorbidities and use of health services of frequent ED users, among people with epilepsy.	8041	Χ2 test Logistic regression Regularised logistic regression Elastic net logistic regression Decision trees Random forests AdaBoost Support vector machines Receiver operating characteristic curve
Gruneir et al 201893 Canada	Nursing home residents	≥3 visits per year	To describe repeat ED visits over 1 year, identify risk factors for repeat use and characterise ‘frequent’ ED visitors.	25 653	Logistic regression Andersen-Gill model
Hardie et al 2015108 USA	All	≥4 visits per year	To describe frequent users of ED services in a rural community setting and the association between counts of patient’s visits and discrete diagnoses.	1652	Poisson regression
Hasegawa et al 201437 USA	People with acute asthma	≥2 visits per year	To examine the proportion and patient characteristics of adult patients with multiple ED visits for acute asthma and the associated hospital charges.	86 224	Χ2 test Kruskal-Wallis test Logistic regression
Hasegawa et al 201476 USA	People with acute heart failure syndrome	≥2 visits per year	To examine the proportion and characteristics of patients with frequent ED visits for acute heart failure syndrome and associated healthcare utilisation.	113 033	Χ2 test Kruskal-Wallis test Negative binomial regression Linear regression
Hasegawa et al 2014102 USA	People with chronic obstructive pulmonary disease	≥2 visits per year	To quantify the proportion and characteristics of patients with frequent ED visits for acute exacerbation of chronic obstructive pulmonary disease and associated healthcare utilisation.	98 280	Χ2 test Kruskal-Wallis test Logistic regression Negative binomial regression Linear regression
Huang et al 200338 Taiwan	All	≥4 visits per year	To characterise frequent ED users and to identify the factors associated with frequent ED use in a hospital in Taiwan.	800	Χ2 test Logistic regression
Hudon et al 201639 Canada	All	≥3 visits per year	To identify prospectively personal characteristics and experience of organisational and relational dimensions of primary care that predict frequent use of ED.	1769	Mixed-effects logistic regression
Hudon et al 20175 Canada	People with diabetes	≥3 visits for three consecutive years	To explore the factors associated with chronic frequent ED utilisation in a population with diabetes.	62 316	Logistic regression Decision trees
Hunt et al 20063 USA	All	≥4 visits per year	To identify frequent users of the ED and determine the characteristics of these patients.	49 603	Logistic regression
Huynh et al 2016103 Canada	People with substance use disorders	≥4 visits per year	To assess the characteristics of individuals with substance use disorders according to their frequency of ED utilisation, and to examine which variables were associated with an increase in ED visits using Andersen’s model.	4526	Χ2 test Analysis of variance Negative binomial regression Generalised estimating equations
Kanzaria et al 201786 USA	Adults aged 18–55 years	≥4 visits per year	To examine the persistence of frequent ED use over an 11-year period, describe characteristics of persistent versus non-persistent frequent ED users, and identify predictors of persistent frequent ED use.	173 273	Logistic regression
Kerr et al 200540 Canada	Injection drug users	≥3 visits during the two past years	To examine rates of primary care and ER use among injection drug users and to identify correlates of frequent ED use.	883	Χ2 test Wilcoxon signed-rank test t-test Logistic regression
Kidane et al 201898 Canada	Patients who received oesophagectomy	≥3 visits per year	To evaluate healthcare resource utilisation, specifically ED visits within 1 year of oesophagectomy, and to identify risk factors for ED visits and frequent ED use.	3344	t-test Wilcoxon rank-sum test Fisher exact tests Logistic regression
Kim et al 2018125 Canada	All	99th percentile	To describe patient and visit characteristics for Canadian ED highly frequent users and patient subgroups with mental illness, substance misuse or ≥30 yearly ED visits.	261	t-test Wilcoxon rank-sum test
Kirby et al 201041 Australia	People with chronic disease	≥3 visits per year	To explore the link between frequent readmissions in chronic disease and patient-related factors.	15 806	Χ2 test Logistic regression
Kirby et al 201142 Australia	All	≥4 visits per year	To identify the factors associated with frequent re-attendances in a regional hospital thereby highlighting possible solutions to the problem.	15 806	Kruskal-Wallis test Χ2 test Logistic regression
Klein et al 2018126 USA	Adults who present to the ED repeatedly for acute alcohol intoxication	≥20 visits per year	To describe frequent ED users who present to the ED repeatedly for acute alcohol intoxication and their ED encounters.	325	Difference in proportions test
Ko et al 201543 Taiwan	All	≥4 visits per year	To describe the distribution of the frequency of ED visits among ED users in 2010 and to evaluate the association of frequent ED use with various patient characteristics.	170 457	Logistic regression
Ledoux and Minner 200644 Belgium	Psychiatric	≥4 visits per year	(1) To provide a naturalistic evaluation of patients repeating admissions in a psychiatric emergency ward (distinguishing between occasional repeaters and frequent repeaters), (2) to identify patients' characteristics that predict repeated use of a psychiatric ER and (3) to propose adapted treatment models.	2470	Mantel-Haenszel test Analysis of variance Logistic regression
Lee et al 201894 USA	Persons with systemic lupus erythematosus	≥3 visits per year	To identify lupus erythematosus patients who persistently frequented the ED over 4 years.	129	t-test Χ2 test Fisher exact test Logistic regression
Legramante et al 201645 Italy	All	≥4 visits per year	To evaluate and characterise hospital visits of older patients (age 65 or greater) to the ED of a university teaching hospital in Rome, in order to identify clinical and social characteristics potentially associated with ‘elderly frequent users’.	38 016	t-test Logistic regression
Leporatti et al 201646 Italy	All	90th percentile ≥3 visits per year	To describe the characteristics of patients who frequently accessed accident and EDs located in the metropolitan area of Genoa.	147 864	Zero-truncated negative binomial regression Logistic regression
Lim et al 201447 Singapore	People with asthma	≥4 visits per year	To describe the characteristics of frequent attenders who present themselves multiple times to the ED for asthma exacerbations.	155	t-test Χ2 test Mann-Whitney U test Logistic regression
Limsrivilai et al 201748 USA	People with inflammatory bowel diseases	75th percentile of the annual medical charges	To identify predictive factors readily available in a standard electronic medical record to develop a multivariate model to predict the probability of inflammatory bowel diseases-related hospitalisation, ED visit and high total charges in the subsequent year.	1430	Receiver operating characteristic curve Logistic regression
Lin et al 2015104 USA	Homeless people	≥3 visits per year	To examined factors associated with frequent hospitalisations and ED visits among Medicaid members who were homeless.	6494	Χ2 test Analysis of variance Negative binomial regression
Liu et al 201349 USA	People with mental health, alcohol or drug-related diagnoses	≥4 visits per year	To determine whether frequent ED users are more likely to make at least one and a majority of visits for mental health, alcohol or drug-related complaints compared with non-frequent users.	65 201	t-test Χ2 test Logistic regression
Mandelberg et al 200050 USA	All	≥5 visits per year	To determine how the demographic, clinical and utilisation characteristics of frequent ED users differ from those of other ED patients.	43 383	Logistic regression Survival analysis
Mann et al 201651 Canada	People with chronic pain	90th percentile	To investigate the role of chronic pain in healthcare visits and to document the frequency of healthcare visits and to identify characteristics associated with frequent visits.	1274	Logistic regression
Mann et al 201795 Canada	People with chronic pain	90th percentile	To describe factors associated with high clinic and ER use among individuals with chronic pain.	702	t-test Logistic regression
McMahon et al 201852 Ireland	All	≥4 visits per year	To examine the characteristics of the frequent ED attenders by age (under 65 and over 65 years).	19 310	Χ2 test Logistic regression
Meyer et al 2013109 USA	Prisoners with Human Immunodeficiency Virus	≥2 visits per year	To characterise the medical, social and psychiatric correlates of frequent ED use among released prisoners with HIV.	151	t-test Χ2 test Poisson regression
Milani et al 201653 USA	People with multimorbid chronic diseases	≥4 visits per year	To examine the association between multimorbid chronic disease and frequency ED visits in the past 6 months, by sex, in a community sample of adults from northern Florida.	7143	Breslow-Day test Logistic regression
Milbrett and Halm 2009110 USA	All	≥6 visits per year	To describe the characteristics of patients who frequently use ED services and to determine factors most predictive of frequent ED use.	201	Χ2 test Mann-Whitney U test Poisson regression
Moe et al 2013121 Canada	All	95th percentile	To develop uniform definitions, quantify ED burden and characterise adult frequent users of a suburban community ED.	14 223	Χ2 test Mann-Whitney U test
Mueller et al 201654 USA	Children with cancer	90th percentile ≥4 visits per year	To (a) evaluate patient and ED encounter characteristics of frequent ED utilisers among children with cancer and (b) quantify healthcare services for frequent ED utilisers.	17 943	Χ2 test Logistic regression
Nambiar et al 201755 Australia	Adults who inject drugs	≥3 visits per year	To describe demographic factors, patterns of substance use and previous health service use associated with frequent use of EDs in people who inject drugs.	612	Negative binomial regression Logistic regression
Nambiar et al 2018106 Australia	Adults who inject drugs	≥3 visits per year	To describe characteristics of state-wide ED presentations in a cohort of people who inject drugs, compare presentation rates to the general population and to examine characteristics associated with frequent ED use.	678	Negative-binomial regression Generalised estimating equations
Naseer et al 201887 Sweden	Older adults	≥4 visits during a 4-year period	To assess the association of health related quality of life with time to first ED visit and/or frequent ED use in older adults during 4 year period and if this association differs in 66–80 and 80+ age groups.	673	Cox proportional hazard model Logistic regression
Neufeld et al 201656 Canada	All	≥4 visits per year	To describe factors predicting frequent ED use among rural older adults receiving home care services in Ontario, Canada.	12 118	Χ2 test Logistic regression
Neuman et al 2014117 USA	All	≥4 visits per year	To compare the characteristics and ED health services of children by their ED visit frequency.	1 896 547	Mantel-Haenszel test Receiver operating characteristic curve Generalised linear mixed-effects models
Ngamini-Ngui et al 2014118 Canada	Patients with schizophrenia and a co-occurring substance use disorder	≥5 visits per year	To assess factors associated over time with high use of EDs by Quebec patients who had schizophrenia and a co-occurring substance use disorder.	2921	Generalised estimating equations
Norman et al 201657 USA	All	≥4 visits per year	To clearly define and describe characteristics of frequent EMS users in order to provide suggestions for efficient and cost-effective interventions that address the healthcare needs of these users.	539	Logistic regression
O’Toole et al 200779 USA	Substance users	≥3 visits per year	To identify factors associated with 12 month high frequency utilisation of ambulatory care, ED and inpatient medical care in a substance-using population.	326	t-test Χ2 test Logistic regression
Palmer et al 201458 Canada	All	≥4 visits per year	To determine if having a primary care provider is an important factor in frequency of ED use.	59 803	Χ2 test Wilcoxon rank-sum test Logistic regression
Panopalis et al 201059 USA	People with systemic lupus erythematosus	≥3 visits per year	To describe characteristics of systemic lupus erythematosus patients who are frequent users of the ED and to identify predictors of frequent ED use.	807	One-way analysis of variance Logistic regression
Pasic et al 200580 USA	Psychiatric	2 SD above the mean number of visits ≥6 visits per year ≥4 visits in a quarter	To examine the sociodemographic and clinical characteristics of high utilisers of psychiatric emergency services.	17 481	Χ2 test Logistic regression
Paul et al 201060 Singapore	All	≥5 visits per year	To determine factors associated with frequent ED attendance at an acute general hospital in Singapore.	82 172	Χ2 test Logistic regression
Peltz et al 2017101 USA	Medicaid-insured children	≥4 visits per year	To describe the characteristics of children who sustain high-frequency ED use over the following 2 years.	470 449	Χ2 test Wilcoxon signed-rank test Logistic regression
Pereira et al 201661 USA	All	≥5 visits per year	To develop machine learning models that can predict future ED utilisation of individual patients, using only information from the present and the past.	4 604 252	Decision trees AdaBoost Logistic regression
Pines and Buford 200662 USA	People with asthma	90th percentile ≥3 visits per year	To determine socioeconomic and demographic factors that predict frequent ED use among asthmatics in southeastern Pennsylvania.	1799	t-test Χ2 test Logistic regression
Quilty et al 201663 Australia	People without chronic health conditions	≥6 visits per year	To determine the clinical and environmental variables associated with frequent presentations by adult patients to a remote Australian hospital ED for reasons other than chronic health conditions.	273	t-test Χ2 test Fisher exact tests Logistic regression
Rask et al 199881 USA	All	≥10 visits per 2 years	To describe primary care clinic use and emergency ED use for a cohort of public hospital patients seen in the ED, identify predictors of frequent ED use, and ascertain the clinical diagnoses of those with high rates of ED use.	351	Χ2 test t-test Logistic regression
Rauch et al 2018115 Germany	All	≥3 visits per year	To examine (1) what ambulatory care sensitive conditions are linked to frequent use, (2) how frequent users can be clustered into subgroups with respect to their diagnoses, acuity and admittance, and (3) whether frequent use is related to higher acuity or admission rate.	23 364	Χ2 test t-test Linear regression Non-negative matrix factorisation
Sacamo et al 2018111 USA	Persons with substance use	≥2 visits per 6 months	To examine associations of individuals and their social networks with high frequency ED use among persons reporting substance use.	653	Poisson regression
Samuels-Kalow et al 201764 USA	All	≥4 visits per year	To derive and test a predictive model for high frequency (four or more visits per year), low-acuity (emergency severity index 4 or 5) utilisation of the paediatric ED.	60 799 (number of visits)	Likelihood ratio test Χ2 test Receiver operating characteristic curve Logistic regression
Samuels-Kalow et al 201888 USA	Patients with asthma exacerbation	≥4 visits per year	To create a predictive model to prospectively identify patients at risk of high-frequency ED utilisation for asthma and to examine how that model differed using state wide versus single-centre data.	254 132	Χ2 test Fisher exact tests Wilcoxon rank-sum test Hosmer-Lemeshow test Receiver operating characteristic curve Logistic regression
Samuels-Kalow et al 2018119 USA	Children	≥3 visits per year	To develop a population-based model for predicting Medicaid-insured children at risk for high frequency of low-resource-intensity ED visits.	743 016	Χ2 test Receiver operating characteristic curve Logistic regression
Schlichting et al 201799 USA	Children	≥2 visits per year	To examine the utilisation of the ED by children with different forms of insurance and describe factors associated with repeat ED use and high reliance on the ED in a nationally representative sample of children in the USA.	47 926	Logistic regression
Schmoll et al 201565 France	Psychiatric	≥9 visits during the six past years	To describe demographic and clinical characteristics of frequent visitors to a psychiatric emergency ward in a French Academic hospital over 6 years in comparison to non-frequent visitors.	8800	t-test Χ2 test Logistic regression
Soler et al 200466 Spain	People with chronic obstructive pulmonary disease	≥3 visits per year	To identify factors associated with frequent use of hospital services (emergency care and admissions) in patients with chronic obstructive pulmonary disease.	64	t-test Χ2 test Kolmogorov-Smirnov test Mann-Whitney U test Logistic regression
Street et al 2018123 Australia	Adults aged≥65 years	≥4 visits per year	To characterise older people who frequently use ED and compare patient outcomes with older non-frequent ED attenders.	21 073	Χ2 test Wilcoxon rank-sum test Ordinal regression
Sun et al 200367 USA	All	≥4 visits per year	To identify predictors and outcomes associated with frequent ED users.	2333	Likelihood ratio test Χ2 test Hosmer-Lemeshow test Logistic regression Bootstrap
Supat et al 2018100 USA	Children	≥6 visits per year	To assess paediatric ED utilisation in California and to describe those identified as frequent ED users.	690 130	Logistic regression
Tangherlini et al 201068 USA	All	≥4 visits per year	To identify the factors that lead to increased use of EMS by patients≥65 years of age in an urban EMS system.	10 918	Kruskal-Wallis test Χ2 test Logistic regression
Thakarar et al 201569 USA	Homeless	≥2 visits per year	To identify risk factors for frequent ER visits and to examine the effects of housing status and HIV serostatus on ER utilisation.	412	Χ2 test Logistic regression
Vandyk et al 201470 Canada	Mental health	≥5 visits per year	To explore the population profile and associated socio demographic, clinical and service use factors of individuals who make frequent visits (5+ annually) to hospital EDs for mental health complaints.	536	Hosmer-Lemeshow test Logistic regression
Vinton et al 201471 USA	Chronic diseases and mental health	≥4 visits per year	To compare the characteristics of US adults by frequency of ED utilisation, specifically the prevalence of chronic diseases and outpatient primary care and mental health utilisation.	157 818	Logistic regression
Vu et al 201572 Switzerland	Mental health and substance users	≥4 visits per year	To determine the proportions of psychiatric and substance use disorders suffered by EDs’ frequent users compared with the mainstream ED population, to evaluate how effectively these disorders were diagnosed in both groups of patients by ED physicians, and to determine if these disorders were predictive of a frequent use of ED services.	389	Fisher exact tests Χ2 test Logistic regression
Wajnberg et al 2012122 USA	All	≥4 visits over 6 months	To determine factors associated with frequent ED utilisation by older adults.	5718	Χ2 test t-test
Watase et al 201573 Japan	Adults with asthma	≥2 visits per year	To characterise the adult patients who frequently presented to the ED for asthma exacerbation in Japan.	1002	One-way analysis of variance Χ2 test Kruskal-Wallis test Logistic regression Negative binomial regression
Weidner et al 201889 USA	Patients with colorectal cancer	≥3 visits per year	To assess ED utilisation in patients with colorectal cancer to identify factors associated with ED visits and subsequent admission, as well as identify a high-risk subset of patients that could be targeted to reduce ED visits.	13 446	Χ2 test t-test Logistic regression Negative binomial regression
Wong et al 2018116 Singapore	Patients with cancer	≥4 visits per year	To identify factors associated with patients becoming ED frequent attenders after a cancer-related hospitalisation.	47 235	Cox regression Survival analysis
Woo et al 201674 Korea	All	≥4 visits per year	To understand whether the findings about frequent ED users in prior studies in the US healthcare system would be replicated in the Korean population, and whether these findings are independent of insurance status or ethnicity.	156 246	t-test Χ2 test Linear regression Logistic regression
Wu et al 201675 USA	All	≥16 visits during the two past years	To assess the feasibility of using routinely gathered registration data to predict patients who will visit EDs with high frequency.	1 272 367	Logistic regression Receiver operating characteristic curve
Zook et al 201890 USA	Native American children	≥4 visits per year	To determine differences in ED use by Native American children in rural and urban settings and identify factors associated with frequent ED visits.	39 220	Logistic regression Hierarchical model Multiple imputations

ED, emergency department; EMS, emergency medical services; ER, emergency room.

Figure 2.

Number of studies by country.

Regression

Regression tools consist of a set of processes aimed at quantifying the relationships between a dependent variable and other explanatory variables.¹⁴ They are useful for description and prediction. Some regression models may be regularised, which in this case means avoiding overfitting with too many explanatory variables, or zero-truncated, which means that the model is not allowed to take null values.

Out of the four categories (regression, hypothesis testing, machine learning and other tools), the most reported tool was the logistic regression (90 studies,3–5 15–101 two of which are regularised by LASSO or elastic net techniques), followed by the binomial regression (13 studies,18 46 55 73 76 77 82 89 102–106 2 of which are zero-truncated). To a lesser extent, the Poisson regression (seven studies,^{77 107–112} one of which is zero-truncated), the linear regression (six studies74 76 102 113–115), the analysis of variance (six studies44 59 73 96 103 104), the Cox regression (four studies87 93 105 116) and hierarchical models (one study⁹⁰) were also used. In those studies, the results are often associated with ORs. The mixed-effects models were mentioned three times.^{39 91 117} Regression parameters were estimated by generalised estimating equations in four studies18 103 106 118 while parameter confidence intervals were estimated by the bootstrap procedure (two studies^{25 67}) and the Clopper-Pearson method (one study²⁵). The receiver operating characteristic curve, or equivalently the sensitivity, specificity or area under the curve (‘c-statistic’), was computed in 10 studies.4 36 48 64 75 83 88 107 117 119 Finally, two studies performed imputation to account for missing data (Markov chain Monte Carlo and multiple imputations^{78 90}).

Hypothesis testing

Statistical tests aim at testing a specific hypothesis about data and rely on probability distributions.¹²⁰ In the selected studies, the tests aimed mainly at comparing two samples (frequent users and non-frequent users).

The most common statistical tests were the χ² test (53 studies17 28 31 34 36–38 40–42 47 49 52 54 56 58 60 62–69 72–74 76 77 79–82 85 88 89 94 96 97 101–104 109 110 112 115 119 121–124) and the t-test (24 studies40 45 47 49 62 63 65 66 74 77 79 81 85 89 94 95 97 98 109 114 115 122 124 125) which measured association between variables or goodness-of-fit. As an alternative to the χ² test for association, five studies used the Fisher exact test.63 72 94 98 119 Sample mean differences were assessed by 23 studies with the Mann-Whitney U test (also called the Wilcoxon rank-sum test20 23 31 47 58 66 77 98 110 119 121 123–125), its variant for dependent samples the Wilcoxon signed rank test,^{40 101} or the Kruskal-Wallis test.23 37 42 68 73 76 102 The difference in proportions test,¹²⁶ Mantel-Haenszel test (test for differences in contingency tables, two studies^{44 117}), the likelihood ratio test (significance test for nested models, two studies^{64 67}), the Hosmer-Lemeshow test (goodness-of-fit for logistic regression, two studies^{67 70}), the Wald test (significance test for regression coefficients, two studies^{30 96}) and the Breslow-Day test (test for homogeneity in contingency tables OR⁵³) were also used to a lesser degree. Finally, one study checked the assumption of normality with the Kolmogorov-Smirnov test.⁶⁶

Machine learning

Machine learning tools are a set of algorithms that can learn and adapt to data in order to classify or predict, for instance.¹²⁷ In the selected studies, the machine learning tools aimed mainly at classifying users (frequent vs non-frequent).

Two studies used random forests^{31 36} along with support vector machines. Decision trees, which include classification and regression trees, were implemented by five studies.5 31 36 61 113 Adaptive boosting, or AdaBoost, is a meta-algorithm that combines with other algorithms and helps for better performances. It was computed in two studies.^{36 61}

Other tools

Two studies used survival analysis,^{50 116} while another one fitted a non-parametric distribution to their data.²⁵ Finally, maximum likelihood monotone coarse classifier algorithm was used as a binning method⁹¹ and non-negative matrix factorisation as a clustering technique.¹¹⁵

Discussion

The most exploited statistical tools arguably came from regression analysis. This may be because regression is well established in medical statistics or also because it is the most natural tool when trying to find significant variables to explain a dependent variable (in this case, to be a frequent user). Moreover, it allows predicting easily the risk of a new user becoming a frequent user, depending on its covariates. Other tools from hypothesis testing or machine learning also proved to be popular, although to a much lesser extent. Combining these statistical techniques may help in discovering significant and complementary patterns, compared with using tools from one class only. In our scoping review, two studies mixed statistical tools from regression, hypothesis testing and machine learning.^{31 36} In those studies, the author evaluated various performance criteria. While logistic regression performed well, other techniques such as random forests or LASSO regression were also competitive. Besides the fact that logistic regression can display modest performances,¹²⁸ random forests and LASSO regression can complete logistic regression. The first technique can be used to assess the importance of each independent variable in the model, while the second technique can be useful for automatic selection of features. Likewise, using a variety of statistical tools can help complete or confirm results obtained with established methodologies. Different tools from one class can also be mixed in order to achieve different stages of the analysis (for instance, different types of regression⁸²).

The analysis of frequent ED users could benefit from using more machine learning techniques. Those were found to be not as common as regression or hypothesis testing, although they are especially appropriate when dealing with classification, prediction or big data. Tools such as support vector machines (which were used by two studies in this scoping review^{31 36}), artificial neural networks or Bayesian networks are common classifiers and predictors in the artificial intelligence community.¹²⁹ They are popular for instance in cancer diagnostic and prognosis, which strongly rely on classification and prediction.^130–132 In particular, support vector machines, decision trees or self-organising maps can deal with binary outcomes, which is usually the case for frequent use outcomes. They usually require large datasets in order to overcome overfitting, but this is becoming less and less of an issue in health sciences.¹³³ Nevertheless, machine learning tools often use a black box approach as there are many intermediary steps leading to the final solution. While each step usually consists of simple arithmetic operations, their multiple interactions can be more difficult to interpret. In spite of this opacity, they still display good performances in classifying and predicting. In some cases, they may be more accurate than the widely used logistic regression.¹³⁴ Those methods would thus turn out to be less useful in data exploration.¹³⁵ Machine learning tools are getting popular in other fields in health sciences, such as critical care,¹³⁶ cardiology¹³⁷ or emergency medicine.¹³⁸ The authors state that their fields would benefit from this growing popularity, though results need to be analysed and interpreted in collaboration with clinicians.

Other tools exist that may also be suitable for describing the associated variables or the prediction of frequent ED users but were not reported in the literature. Among those, principal component analysis (PCA) is a dimensional reduction and visualisation technique, sometimes used with cluster or discriminant analysis.¹³⁹ Based on all the original explanatory variables, PCA constructs new ones by summing and weighing them differently. More weight is given to relevant variables so that those latter become dominant in the new constructions while still including all variables. For instance, Burgel et al ¹⁴⁰ built chronic obstructive pulmonary disease clinical phenotypes by constructing new relevant variables with PCA and by grouping similar subjects in this new space with cluster analysis.¹⁴⁰ Moreover, PCA has already been used for the construction of questionnaires and diagnosis tools in a medical context,^{141 142} both of which can prove useful in the identification of frequent users.

As mentioned, regression techniques were common in the selected studies. Yet, quantile regression (QR)¹⁴³ was not mentioned. QR is a generalisation of mean regression in the sense that its focus is not only the mean of the dependent variable distribution (such as in classical linear regression) but any quantile of it. QR thus represents an alternative to define frequent users by the high quantiles of ED visit distribution (eg, the 90th quantile). Eight studies25 27 46 48 51 54 62 121 defined frequent users with quantiles, but they did not use QR. QR would allow for finer investigations in the different quantiles of ED users in relationship to the explanatory variables. For instance, the association between age and the number of ED visits may be significantly different across the 10th (low users) and 90th (frequent users) quantiles. Such a heterogeneous association would be uncovered by QR, while usually unseen with a classical mean regression. Ding et al ¹⁴⁴ used QR to characterise waiting room and treatment times in EDs.¹⁴⁴ They explored the lowest, median and highest of those times and highlighted predictors that were significant only in particular quantiles. Usually, QR requires a continuous dependent variable as opposed to a logistic regression, though it is possible to combine these two regressions.¹⁴⁵ Furthermore, defining frequent users by quantiles would allow for better comparison between studies as there is no common definition for frequent users.

Strengths and limitations

To the best of our knowledge, this scoping review is the first to list statistical tools that are used in the identification of variables associated with frequent ED use and the prediction of frequent users. Besides, it was conducted following a well-defined methodological framework. The search strategies were designed with an information specialist in three different databases. Two independent evaluators selected the articles and extracted the data while a third independent evaluator settled disagreements, ensuring that all included studies were relevant. One limitation of our study is that quality assessment is not performed in a scoping review. However, this should not alter the results, since the aim was to list which statistical tools have been applied in the literature. Moreover, the majority of articles were in English which may introduce a selection bias (for instance, one excluded article was in Spanish). More than half of the reviewed studies were indeed conducted in the USA, making the results difficult to compare to other countries.

Conclusions

Frequent ED users represent a complex issue, and their analysis require adequate statistical tools. In this context, this scoping review shows that some tools are well established, such as logistic regression and χ² test, while others such as support vector machines are less so, though they would deserve to get more attention. It also outlines some research opportunities with other tools not yet explored.