Performance of the interRAI ED Screener for Risk-Screening in Older Adults Accessing Paramedic Services

Abstract

Background

Paramedics respond to a significant number of non-emergency calls generated by older adults each year. Paramedics routinely assess and screen older adults to determine risk level and need for additional follow-up. This project implemented the interRAI ED Screener into routine care to determine whether the screener and resulting Assessment Urgency Algorithm (AUA) score is useful in predicting adverse outcomes.

Methods

We conducted a population-based retrospective study using administrative health data for patients aged 65+ assessed by paramedics from July 2016 to February 2017. Patients were assigned an AUA score and classified into three risk categories. Outcome data including hospitalizations, Emergency Department (ED) visits, home care status, and survival were collected and compared across AUA risk categories using descriptive and analytical statistics.

Results

Of the 2,801 patients screened, 31.9% were classified as high risk, 23.6% as moderate risk, and 44.6% as low risk. Patients who scored in the highest risk category were found to have longer hospital stays, and were more likely to require home care (p<.0001). The AUA risk category also predicted survival (p<.001).

Conclusions

The AUA predicted multiple adverse outcomes in this population. Use of the AUA by paramedics may aid in earlier identification of those in need of additional intervention and services.

INTRODUCTION

Older adults have the highest rates of Emergency Department (ED) use, and tend to be more complex and time-consuming to assess and treat compared to younger adults.^(1–8) A high rate of utilization is also seen with paramedic services; older adults are 4.5 times more likely to arrive to the ED by ambulance,⁽⁹⁾ and are more than twice as likely to call an ambulance for a non-emergency.⁽¹⁰⁾ In response to the increased burden on paramedic services, community paramedicine (CP) programs have been implemented which expand the paramedic scope of practice beyond acute care and transport to the ED. CP programs have emerged throughout Canada and internationally in an effort to maximize efficiencies in patient care and resources. The development and expansion of CP programs allows paramedics to apply their education and skills beyond the traditional role of emergency medical response.^(11–13) As CP programs expand, there is potential value in developing the capacity of paramedics, both in the proactive community and emergency setting, to identify, advocate, and refer older adults who are at high risk for adverse health outcomes and who are most likely to benefit from additional health-care services or assessments.^(14–16) Adopting a risk-screening protocol in the paramedicine context has been identified as a practical approach to ensure that high-risk older adults are effectively targeted, flagged, and provided with appropriate assessment, treatment, and follow-up.⁽⁹⁾

A variety of screening tools have been implemented and studied in the paramedicine context, including those for emergencies (e.g., stroke⁽¹⁷⁾ and sepsis⁽¹⁸⁾) and for non-emergencies (e.g., social needs screening⁽¹⁹⁾). Within Ontario, the Paramedic Assessing Elders at Risk of Independence Loss (PERIL)⁽²⁰⁾ screening tool has been implemented in 86% of Community Assessment and Referral Programs;⁽²¹⁾ however, the policies around how and when the tool is used vary by region. Additionally, there are several risk-stratification tools that have been studied in the ED.^(22–24)

In recent years, the interRAI suite of tools has been introduced into multiple care settings both within Canada and worldwide in an effort to bring a standard level of care to geriatric populations.⁽²⁵⁾ In Ontario, interRAI tools have been introduced into primary care, hospital, and home care settings in several regions. The interRAI ED Screener is a risk-stratification tool for use in older adults attending the ED that has been developed through multinational pilot studies reported by the interRAI research collaborative.^(24,26) The interRAI ED Screener is designed to “rapidly prioritize older patients who require more detailed assessment so that vulnerable patients are not overlooked, and to provide a reliable method to organize follow-up in the hospital or the community”.⁽²⁵⁾ Research into the predictive validity of the interRAI ED screener has been limited, with just one recently published study that examined the ability of the interRAI ED Screener to predict one outcome (re-presentation to the ED within 28 days).⁽²⁷⁾ In this study, we aim to determine the usefulness of the interRAI ED screener in the paramedicine context in predicting adverse outcomes for older adults in their care.

METHODS

Study Design

The data used in this study were collected as part of an evaluation of a pilot project undertaken by the Middlesex-London Paramedic Service (MLPS) as part of the Assess and Restore initiative released by the Ontario Ministry of Health and Long-Term Care, of which one essential element for improving health outcomes was the use of proactive risk-screening tools.⁽²⁸⁾ Beginning in July 2016, the MLPS introduced a six-month, service-wide pilot project of the Assessment Urgency Algorithm (AUA; generated by the interRAI ED Screener) in an effort to introduce a standard, evidence-based method for identifying and communicating the level of risk of functional decline and adverse outcomes to community partners. The ED Screener was added to usual care which included the PERIL screening tool, as well as paramedic-direct referral to the regional home and community care program based on clinical judgement by the paramedic. The AUA score complemented usual care by adding risk-stratification and clinical recommendations on how to proceed based on risk level.

The interRAI ED Screener includes four activity of daily living questions: bathing, personal hygiene, dressing the lower body, and locomotion; as well as questions related to cognitive skills, depressive symptoms, caregiver burden, self-rated health, stability of health conditions, and presence of dyspnea.⁽²⁹⁾ The answers to these questions are then subjected to the algorithm that produces the AUA score. The AUA score is used to stratify older adults into six categories (1–6) which can then be condensed to three risk categories: 1 & 2 = low risk (unlikely to require further assessment and follow-up), 3 & 4 = moderate risk (further assessment and referral to community support services), and 5 & 6 = high risk (further assessment and follow-up by specialist services recommended).⁽²⁵⁾

To examine the predictive value of the interRAI ED Screener in this context, we conducted a population-based retrospective analysis using de-identified health administrative data obtained for secondary use for all older adult patients screened by MLPS paramedics following a 911 call from July 2016 to January 2017. Administrative health data collected by MLPS were linked by health card number to hospitalization and home and community care data provided by the regional health authority by MLPS staff. This database included AUA score, age and gender, as well as dates of 911 calls, dates of ED visits, and hospitalizations for hospitals in the region, and dates of home and community care service use, and date of death. As the data provided for this analysis were de-identified, individual patient consent for this research was considered impractical. This study received ethics clearance from the University of Waterloo Office of Research Ethics (ORE #31933).

Study Setting and Population

This study was conducted in the mixed urban/rural region of Southwestern Ontario’s Middlesex County, serving a population of approximately 459,000. Patient records were included in the analysis if they met the following criteria: AUA score obtained by paramedics at the time of initial 911 call, patient was community-dwelling, aged ≥ 65 at the time of initial assessment, and the patient was triaged as Level 3 (Urgent), Level 4 (Less Urgent, or Level 5 (Non-Urgent) as per the Canadian Triage and Acuity Scale (CTAS). The CTAS a nationally used five-level scale that assigns a level of acuity for patients based on the type and severity of their presenting complaint, signs, and symptoms.⁽³⁰⁾ Level 1 (Resuscitation) represents the patients in the worst condition and Level 5 (Non-urgent) represents the group of patients in the best condition. Those patients the score a CTAS level 1 or 2 are considered inappropriate for risk-screening as patients are unlikely to be able to answer any screening questions, and their condition is already a life-threatening emergency that requires significant intervention.

Intervention

As part of the pilot project, the Superintendent of Community Paramedicine participated in a standardized interRAI train-the-trainer session provided by the St. Joseph’s Healthcare London interRAI working group. The Superintendent then trained all MLPS paramedics on staff (n=240) to appropriately administer the interRAI ED Screener over a one-hour session as part of their annual spring training. For the duration of this pilot project, electronic patient care report (ePCR) compliance rules mandated that paramedics complete the AUA screening for all patients that were CTAS 3, 4, or 5, were aged 65 years and older, and who had a community-dwelling pick-up code. The AUA screener was included in the MLPS paramedic’s ePCR system whereby paramedics used a Toughbook to record patient answers to auto-calculate the AUA score. As was part of their existing practice, paramedics could choose to refer patients to the home and community care access team. However, as part of the implementation of the interRAI ED Screener, paramedics were encouraged to consider the patient’s AUA score when making referral decisions, and submitted the AUA score along with their standard referral to the home and community care access team. If a patient scored in the high-risk category (AUA score 5 or 6) and the patient was transported to the ED, a secondary referral pathway was activated automatically to alert Geriatric Emergency Management (GEM) nurses of the patient’s arrival to the ED.

Outcome Measures

The primary outcome for this study is the total number of days hospitalized post-ED visit. We also examined the number of subsequent ED visits, number of discrete hospitalizations, use of home care services, and mortality. Administrative health data for these patients were available for one year after the end of the six-month pilot project, and as such, the follow-up time for participants with 911 calls towards the beginning of the pilot project was approximately 18 months, while the follow-up time for patients with calls towards the end of the pilot project was approximately 12 months.

Data Analysis

All data were analyzed using IBM SPSS software for Windows, version 25 (IBM SPSS Corp., Armonk, NY). Dichotomous variables were compared using chi-square analysis, continuous variables using independent sample t tests, and multiple comparisons were done using analysis of variance with Tukey’s post-hoc test and Mann-Whitney tests, as appropriate. Additional survival analysis was conducted using Cox proportional hazards regression models. All statistical tests were two-tailed and p < .05 was taken to indicate statistical significance.

RESULTS

Characteristics of the Study Population

A total of 2,801 patients, aged 65 or older and with a median age of 80.0 years (range: 65–103) were included in this study. Of these patients, 884 (31.6%) were 65–74 years, 1,042 (37.2%) were 75–84 years, 790 (28.2%) were 85–94 years, and 85 (3%) were 95+ years. Of the patients for which gender was recorded (2,728), 56.8% were female and 40.6% were male. The reason that the 73 patients did not have their gender recorded is unknown; however, this was likely due to missed data entry by MLPS and/or the hospital. Patient characteristics are listed in Table 1.

TABLE 1.

Demographic characteristics of the study population across AUA risk level

Demographic Characteristics	All Patients	AUA Low-Risk Group	AUA Moderate-Risk Group	AUA High-Risk Group	p
	n =2801	n = 1248 (44.6)	n = 660 (23.6)	n = 893 (31.9)
Age (yrs)
Mean	79.65	78.77	79.51	80.99	<.0001
Median	80	79	80	82
Range	65–103	65–101	65–101	65–103
Gender
Male (%)	1138 (41.7)	489 (43.0)	283 (24.9)	366 (32.2)	.185
Female (%)	1590 (58.3)	738 (46.4)	362 (22.8)	490 (30.8)

AUA Score

All patients were evaluated using the interRAI ED Screener tool and assigned an AUA score of 1–6 which was then collapsed into a three-level risk score for analysis: 44.6% of patients belonged in the low-risk category, 23.6% in the moderate-risk, and 31.9% in the high-risk. The AUA risk group and patient characteristics are shown in Table 1.

Analysis of variance showed a main effect of age on AUA risk level, F(2, 2798)=18.41, p<.0001. Post hoc analyses using Tukey’s HSD indicated that the average age was higher for patients in the high-risk group (p=.002), but did not differ significantly between the low- and moderate-risk groups (p=.162).

An independent-samples t-test indicated that female patients (M=80.1, SD=8.69), were on average older than male patients (M=78.8, SD=8.03), t(2555.7)=3.88, p<.0001, d=0.15). However, a Mann-Whitney test indicated that AUA risk scores did not significantly differ between male and female patients (U=876632.00, p=.132, r=−0.028).

Patient Outcomes

A summary of the patient outcomes by AUA score is found in Table 2. The number of patients who had an overnight admission to hospital within the 18 month study time frame was higher for high-risk patients when compared to the low- and medium-risk groups combined (χ²=52.89, df=1, p<.0001). Additionally, analysis of variance with a Tukey’s HSD showed that patients in the high-risk group spent on average more days in hospital (M=20.07, SD=31.91) when compared to the low- (M=10.78, SD=13.91) and moderate-risk groups (M=12.63, SD=15.58), F(2, 720)=11.66, p<.002). However, the number of days in hospital did not differ significantly between the low- and moderate-risk groups (p=.709).

TABLE 2.

Outcomes across AUA risk level

	All Patients (N =2801)	AUA Low-Risk Group (N=1248)	AUA Moderate-Risk Group (N=660)	AUA High-Risk Group (N=893)	P
Total # of patients hospitalized (%)	723 (25.8)	229 (18.3)	185 (28.0)	309 (34.6)	<.0001
Mean # of days hospitalized total (SD)	15.22 (23.10)	10.78 (13.9)	12.63 (15.6)	20.07(31.9)	<.0001
Mean # Emergency Room Visits (SD)	1.91 (1.33)	1.79 (1.19)a	2.07 (1.55)a	1.91 (1.30)	<.05
# of patients receiving home care services (%)	1217 (43.4)	346 (27.2)	300 (45.5)	571 (63.9)	<.0001
Mean survival time in days (SD)	489.33 (149.02)	526.81 (86.70)	486.05 (153.44)	439.37 (193.23)	<.0001

^aStatistical significance in this row achieved for these groups.

The average number of ED visits within 30 days of the patient’s initial 911 call was significantly different between the moderate-risk group (M=2.07, SD=1.55), and the low-risk group (M=1.79, SD=1.19) p=.022, with a small effect size (d=0.21), but did not differ between the high-risk (M=1.91, SD=1.30) and low-risk groups (p=.453), nor between the high-risk and moderate-risk groups (p=.237).

Home care services data indicated that 1,217 (43.4%) patients received new or increased home care in the 18 month study period. A Chi-square analysis indicated that patients in the high-risk group were more likely to be referred to, or remain, in-home care than those in the moderate- and low-risk groups (χ²(1)=196.245, p<.000).

Survival

The death of a patient was indicated in the dataset if the patient’s date of death was recorded by the MLPS, the regional hospital, or home and community care records at any time in the 18-month study period. Overall survival (OS) was calculated from the date of initial 911 call to the date of death from any cause. Patients who did not have their death recorded (n=2,352) were right-censored at the date of study end at 18 months (547 days) (Table 1). Survival curves were obtained using the Kaplan-Meier technique (Figure 1), and AUA risk level associations with OS were examined using a Cox proportional hazards regression. Throughout the study, death from any cause was significantly higher in the high-risk group (28.6%; n=255) than in the moderate- (16.8%; n=111) and low- risk groups (6.7%; n=83) combined (hazard ratio 2.120, 95% CI: 1.855 to 2.384).

FIGURE 1.

Overall survival by AUA risk level

Many variables which might have been potential confounders were not included in our dataset, which we recognize as a limitation of this study. We were, however, able to investigate age and gender in a multivariable Cox model. There was little effect on the beta coefficient for the AUA (from .766 for the AUA alone to .751 in the three variable models). This suggests that age and gender were not confounders and the AUA was an independent predictor of mortality.

DISCUSSION

This study examined the ability of the interRAI ED Screener and AUA score to predict adverse outcomes in older adults in the paramedicine context. Our results indicate that a higher AUA score is associated with future hospitalizations, increased length of stay in hospital, increased need for home care services, and mortality in the older adult population. The AUA score is less associated with repeat ED visits, an outcome which has been identified as an important component of the problems with overcrowding in EDs.⁽³¹⁾ A recent study of ED usage found that older adults were more likely to be frequent ED users, which may partly be explained by their increased number of comorbidities and chronic diseases.^(31–32) The inability of the AUA to predict ED use, however, is not unique among risk-screening tools for this population. A recent review of one of the most well-studied and widely used screening tools for older adults in the ED, the Identification of Seniors at Risk (ISAR), found only one out of five studies reported positive predictive validity of revisits to the ED.⁽³³⁾ Further research is needed to determine what specific factors might predict repeat ED visits in the paramedicine context.

In this study, a higher number of patients scored in the high-risk category than was found in the Australian population reported by Taylor et al.⁽²⁷⁾ This difference could be due to the context of the test administration (in ambulance vs. in ED upon self-presentation), such that patients that are calling 911 may be in worse condition (and in general at higher risk for adverse outcomes) than those that are able to self-present to the ED. It is also possible that the screener administrator had an impact on the score. In the Taylor study, the administration of the screener was done by specialist geriatric nurses with geriatric expertise who, the authors note, may have made inferences regarding the screener scores due to their clinical expertise. It may be that the assumptions made by paramedics when completing the screener were different. Determining if one administrator population is more accurate than another in completing the screener requires further research.

In recent years, the role of paramedics in many regions has evolved to include primary care and community referral services for older adults.⁽²¹⁾ CP programs have rapidly expanded, resulting in paramedics seeing patients in contexts other than emergency 911 calls. This research demonstrates that the interRAI ED screener may be a useful and accurate tool for risk stratification of older adults in the paramedicine context. An internal report provided to researchers by the MLPS indicated that, according to their own data, the average number of referrals to all available services (this includes home care services such as nursing, personal support worker [PSW] services, as well as referrals to assisted living facilities) was up 221% for the time period that the AUA was implemented (July–January) compared with the same time period the prior year, which may suggest some patients might have previously been overlooked when only the PERIL assessment or clinical judgement is used. It is worth noting that the higher percentage of referrals was not sustained and went back to the prior year’s level within three months after the AUA trial was completed. This suggests that it was not just additional training in risk-assessment that resulted in this increase, but rather something about the use of the tool itself that resulted in the increased number of referrals.

Limitations

This study was conducted with urgent to non-urgent older adults who arrive at the ED by ambulance; however, data that could be relevant, such as triage level and diagnosis, were not available and thus not included in the analysis. We were also not able to differentiate between those patients who were first-time referrals to home and community care and those who were being referred for additional services. Additionally, the number of deaths recorded is based on the data reported by the regional health authority at the time of study end. This may underestimate the number of events—for example, if reporting of deaths had been delayed or if location of death was outside of the region and/or not reported to the regional health authority. Because the interRAI ED Screener has not been widely tested for predictive validity in the ED itself, it is not clear whether the results of this study are generalizable to the ED, where the screener is more commonly used. This study did not extensively examine the ease of implementation of this screener, and further information regarding feasibility, time of administration, and ease of use in the paramedicine context is needed. Additionally, without direct observation of the administration of the tests, it is unknown whether the screeners were completed accurately and consistently for all patients and by all paramedics. A prospective study of the interRAI ED Screener is needed to confirm the results of this retrospective assessment.

CONCLUSION

The data from this study suggest that the AUA score is an accurate predictor of adverse outcomes, including hospitalizations, length of stay, home care needs, and death, when administered in the paramedicine context. It is not as accurate in predicting repeat ED visits.

Implementing the screener and the use of the AUA more broadly will require additional research to demonstrate the impact of risk identification on avoiding adverse outcomes for this population through appropriate interventions.