Abstract
Background
Social risk is rarely evaluated in older emergency department (ED) patients, although its identification can reduce re-attendance.
Objectives
This study assessed the diagnostic accuracy of the Triage Risk Screening Tool (TRST) in the ED of a developing country.
Methods
The diagnostic accuracy of the TRST to detect elderly adults in need of social service intervention was compared with routine clinical evaluation, using comprehensive evaluation by an experienced social worker as the “gold standard”. The inter-rater reliability of the TRST was assessed on a separate cohort of patients prior to the main study.
Results
The sensitivity of the TRST was 94.7% versus 55.6% for physician assessment. The TRST had good inter-rater reliability (Cohen’s kappa = .882), and physicians found it easy to use.
Conclusion
The TRST provides a rapid means of assessing risk in older ED patients. This study confirmed the validity of this screening tool in a third world setting.
Keywords: Older adults, Caribbean, risk assessment, triage, emergency department, social risk
Introduction
Psychosocial assessments are not routinely performed among older people attending emergency departments (EDs) in Trinidad, although this is routine in many developed countries.1–3
Adverse outcomes, including higher mortality and re-attendance rates in older ED patients are higher in those with inadequate social support. For example, Dramé et al.4 demonstrated that social isolation and increased caregiver burden were associated with higher 6-month mortality rates in older hospitalised patients, and Foo et al.5 demonstrated that identification and addressing the social support needs of older ED patients significantly reduced their re-attendance rates.
Few screening tools assess the need for social support among older ED patients.6 The Comprehensive geriatric assessment (CGA) has been in use for this purpose since the 1980s. However, the CGA is time-consuming and, while used in EDs, is not always suitable for use in that environment.7 The Triage Risk Screening Tool (TRST) is a screening questionnaire that has been validated in North America and parts of Europe and is moderately accurate, with good inter-rater reliability.8,9 Other questionnaires, such as the Identification of Seniors at Risk (ISAR), are also used to screen older ED patients. However, the ISAR is self-administered and may not be appropriate in third world countries, where literacy rates among older people are likely to be relatively high. Both the TRST and ISAR identify seniors at risk of ED re-attendance and in need of social service intervention.
As populations in developing countries age, screening tools such as the TRST are becoming more relevant. By 2050, it is estimated that older adults (age ≥ 60 years) will account for 25% of the population in lower and middle income countries, such as those of the Commonwealth Caribbean, and that 75% of the world’s older population will live in developing countries by this time.10
This study was conducted at the Eric Williams Medical Sciences Complex, a tertiary teaching hospital in Trinidad. This Caribbean island has a population of 1.3 million, with health services provided by publicly and privately financed health facilities. As with many Caribbean countries, the population is ageing and patients older than 60 years currently account for approximately 10% to 20% of all ED admissions. Previous studies done among this population have demonstrated relatively high levels of social support for older people; however, some authors suggest that this is declining with time owing to disintegration of the extended family and an increase in the number of households in which both adults are working.11,12
This prospective, single-centre diagnostic study tested the predictive validity of the TRST in determining the need for social intervention among older ED patients in Trinidad.
Methods
The aim of this study was to determine whether the TRST was more accurate than routine assessment at detecting patients in need of social service intervention, compared with the “gold standard” of assessment by an experienced social worker.
Patients ≥ 65 years old who were being discharged from the ED were included. Patients were excluded if they were too unwell to take part in the study, refused to participate, or were admitted to the hospital.
A pilot of 20 patients was conducted prior to commencement of the main study. This pilot study assessed the acceptability and inter-rater reliability of the TRST in a local population of older people, when used by local ED physicians. The physicians and patients who participated in the pilot were not included in the main trial. Each patient was assessed independently by two ED physicians using the TRST. The inter-rater reliability of the tool was then estimated and the physicians were asked about the ease of use of the TRST and the time needed to complete it.
Prior to commencement of the main study in May 2012, ED physicians were educated on the benefits of social assessment in older patients. Following this, they were asked to use their clinical judgement to assess whether their older patients needed social service intervention (Phase 1 of the study). The physicians’ notes were written in free form in this phase of the study, which ran for 2 months. In Phase 2, which was undertaken over a 2-month period, patients were assessed using the TRST. There was a 4-week “wash-out” period between phases 1 and 2, to minimise the influence of Phase 1 on Phase 2 assessments. All physicians taking part in the study were fully qualified and registered doctors, working in the ED as emergency medicine residents or consultants.
Two weeks after discharge, each patient was assessed independently by an experienced social worker. The timeframe between physician and social worker assessment was chosen to allow the social worker time to contact patients while minimising the risk of any significant changes to the patients’ circumstances between assessments. The social worker was blinded to the ED physicians’ assessments. Social worker assessments covered the main areas associated with social risk, including lack of social support, financial insecurity, inability to perform daily activities, mental incapacity, loneliness, and depression.
The main outcome was analysed as a categorical dichotomous variable (need for intervention; yes/no). Patients with two or more positive questions on the TRST were assessed as in need of intervention. Using assessment by an experienced social worker as the gold standard, the sensitivity and specificity of the TRST versus routine physician assessment were calculated. P values of <.05 were taken as statistically significant. Sensitivity and specificity were compared using Pearson’s chi-squared test. A minimum sample size of 80 (40 per intervention) was required to detect a difference of 15% in sensitivity between the TRST and clinical assessment (with a power of 0.8 and significance level of 0.05). Data were analysed using IBM SPSS version 21 (IBM Corp., Armonk, NY, USA).
Ethical approval was obtained from the ethics committees of the University of the West Indies. Informed written consent was obtained from all participants.
Results
A total of 99 patients were enrolled in the study, 36 in the intervention arm (TRST) and 63 in the control arm (physician assessment) (Appendix 1). Patients were broadly similar in their age, sex, and clinical presentation (Table 1).
Table 1.
Demographic characteristics of sample.
| TRST assessment | Clinical assessment | Total | P value | |
|---|---|---|---|---|
| Number of patients | 36 | 63 | 99 | |
| Age (95% CI) | 73.9 (71.4–76.5) | 75.5 (73.6–77.4) | 74.9 (73.4–76.4) | 0.319 |
| Sex, n (%) | ||||
| Male | 17 (47.2%) | 28 (44.4%) | 45 (45.5%) | 0.836 |
| Female | 19 (52.8%) | 35 (55.6%) | 54 (54.5%) | |
| Presenting complaint n (%) | ||||
| Medical | 24 (47.1%) | 59 (60.8%) | 83 (56.1%) | 0.064 |
| Surgical | 1 (2%) | 8 (8.2%) | 9 (6.1%) | |
| Trauma | 5 (9.8%) | 4 (4.1%) | 9 (6.1%) | |
| Not recorded | 21 (41.2%) | 26 (26.8%) | 47 (31.8%) |
Abbreviation: TRST, Triage Risk Screening Tool; CI, confidence interval.
In the pilot study, the TRST demonstrated a high inter-rater reliability (Cohen’s kappa 0.882) and was easy to use; the time needed for completion was less than 2 minutes.
Overall, 37 patients were found by doctors to be in need of social service support; a similar number was found by the medical social worker. This included 19 patients in the TRST arm of the study and 18 in the clinical assessment arm. In comparison, physicians assessed 22 patients in the TRST arm of the study as needing social service support and 15 in the clinical assessment arm. The sensitivity of the TRST was significantly higher than that of physician assessment (94.7% vs. 55.6%, P = 0.025) whereas its specificity was significantly lower (60.0% vs. 88.9%, P = 0.021). The area under the curve (AUC) of the receiver operator characteristic was not significantly different between the TRST and physician assessment (0.752 vs. 0.622, P = 0.311) (Table 2).
Table 2.
Diagnostic accuracy of the TRST compared with clinician assessment.
| TRST assessment | Clinical assessment | Total | P value | |
|---|---|---|---|---|
| Number of patients | 36 | 63 | 99 | |
| Sensitivity (95% CI) | 94.7% (74.0%–99.9%) | 55.6% (30.8%–78.5%) | 0.025 | |
| Specificity (95% CI) | 60.0% (26.7%–87.4%) | 88.9% (76.0%–96.3%) | 0.021 | |
| AUC (95% CI) | 0.622 (0.455–0.789) | 0.752 (0.580–0.924) | 0.311 |
Abbreviation: TRST, Triage Risk Screening Tool; CI, confidence interval, AUC, area under the receiver operator characteristic curve.
Appendix 2 shows a subgroup analysis of the diagnostic accuracy of individual items of the TRST against medical social worker assessment. The most frequent positive item on the scale was “Professional Opinion”, which was further broken down into seven subcategories: nutrition/weight loss; failure to cope; sensory deficits; incontinence; medication issues; depression, and other.
Discussion
The TRST was more sensitive than clinical assessment for detecting the need for social service intervention among older people in our study. While this has previously been demonstrated in developed countries, it is the first time the TRST has been assessed in the Caribbean.8 Whereas the AUC showed no significant difference between physician assessment and the TRST, it reflected a very low sensitivity but high specificity of physician assessment. High sensitivity is of primary importance for a screening tool because such tools attempt to rule out people who do not have the condition (in this case, those who do not need social service input), usually with the intention of providing further assessment for those who have been “ruled in” by the screening tool (in our case, those identified as needing social service input). In this context, the TRST clearly outperformed clinician judgement, with much higher sensitivity and a moderate level of specificity.
Our ED physicians’ ability to identify older patients in need of social intervention was very low, confirming the poor detection of psychosocial problems by ED physicians in general.1 This is significant, as there is a link between inadequate social support and adverse clinical outcomes, including patient re-admission and mortality.4 More structured screening for social risk is needed in our hospitals, as family support for older people in Trinidad may be declining with disintegration of the extended family. In addition, both adults in most households now work, leaving dependent older relatives more vulnerable to neglect.11,12
Interestingly, one item on the TRST accounted for most of the diagnostic accuracy of the questionnaire. The item entitled “Professional Opinion” had 90% sensitivity and 75% specificity for detecting patients in need of social service input. This aspect of the questionnaire provides an opportunity for physicians to use their clinical judgement to decide whether patients require further social service intervention. There is need for a larger study investigating the utility of this aspect of the TRST as a stand-alone tool to validate these findings.
Our pilot study demonstrated good inter-rater reliability of the TRST. The simplicity of administration is likely to have contributed to this.8 It took less than 2 minutes to administer the TRST, unlike the CGA, and did not rely on self-administration, as does the ISAR.13 However, there was a perception among doctors that this screening questionnaire created an additional burden of work, and if it is to be implemented more widely, these perceptions and attitudes will need to be taken into account.
Limitations
The sample size, and single-centre design of this study were major limitations, particularly in light of the high attrition rate of patients in the intervention arm of the study (those in whom the TRST was used for assessment). However, our a priori sample size calculation suggested the need for 40 patients in each arm of the study and our recruitment was very close to this target. While a relatively high proportion of study subjects were lost to follow-up, there was no significant difference in age, sex, or clinical presentation between patients who completed the study and those lost to follow-up (Appendix 3). In light of this, we believe that our findings remain valid, although there is a need to conduct a larger, multi-centre study to confirm the results.
Patients presenting to our hospital are broadly representative of the general population of the island, as it serves just under half of the population (approximately 450,000 people). Therefore, the single-centre study design is unlikely to have produced a significant selection bias. The before-and-after nature of the study may also have potentially biased results; the 4-week “wash-out” period between phases attempted to minimise this.
Other studies assessing the utility of the TRST used hospital re-attendance as the main outcome.8 We chose the assessment of an experienced social worker as our gold standard because the less formal organisation of the health service in Trinidad made it difficult to verify patient re-attendance.
Conclusion
This study supported the validity of the TRST in a third world setting. We plan to conduct a larger multi-centre assessment of the TRST and to measure re-attendance rates among older ED patients, once the limitations mentioned above can be overcome.
Acknowledgements
The authors would like to acknowledge the assistance an encouragement of Sr Clara Tsang RN(EC), Outreach Team, Geriatric Assessment and Intervention Network (GAIN), Ontario, Canada, who introduced us to the TRST and advised us on its use in Ontario, and its implementation in Trinidad and Tobago.
Appendix 1 Recruitment flow diagram for the study.
Appendix 2 Diagnostic accuracy of individual items of the TRST compared with medical social worker assessment. Item 5 (“Professional Opinion”) was divided into seven sub-categories. Appendix 3 Comparison of participants who completed the study with those lost to follow-up. There was no significant difference between the groups.
| Item | Positive responses n (%) | Sensitivity | Specificity |
|---|---|---|---|
| 1. History of Cognitive Impairment | 7 (19.44%) | 38.6% | 100.0% |
| 2. Difficulty Walking/Transferring or Recent Falls | 4 (11.11%) | 21.1% | 100.0% |
| 3. Recent ED use or Hospitalisation | 6 (16.67%) | 31.6% | 100.0% |
| 4. Polypharmacy | 1 (2.78%) | 5.3% | 100.0% |
| 5. Professional Opinion | 21 (58.33%) | 90.0% | 75.0% |
| Nutrition/Weight Loss | 10 (27.78%) | ||
| Failure to Cope | 4 (11.11%) | ||
| Sensory Deficits | 4 (11.11%) | ||
| Incontinence | 0 (0%) | ||
| Medication Issues | 1 (2.78%) | ||
| Depression | 7 (19.44%) | ||
| Other | 6 (16.67%) | ||
| 6. Lives Alone/No Available Care Giver | 9 (25%) | 36.8% | 83.3% |
Abbreviation: ED, emergency department.
| Completed study | Lost to follow-up | Total | P value | |
|---|---|---|---|---|
| Number of patients | 99 | 49 | 148 | |
| Age (95% CI) | 74.5 (72.4–76.6) | 74.9 (73.4–76.4) | 74.8 (73.6–76.0) | 0.675 |
| Sex, n (%) | ||||
| Male | 45 (45.5%) | 25 (51%) | 70 (47.3%) | 0.601 |
| Female | 54 (54.5%) | 24 (49%) | 78 (52.7%) | |
| Presenting complaint n (%) | ||||
| Medical | 55 (55.6%) | 28 (57.1%) | 83 (56.1%) | 0.127 |
| Surgical | 3 (3%) | 6 (12.2%) | 9 (6.1%) | |
| Trauma | 7 (7.1%) | 2 (4.1%) | 9 (6.1%) | |
| Not Recorded | 34 (34.3%) | 13 (26.5%) | 47 (31.8%) | |
| Triage-positive patients, n (%) | 40 (40.4%) | 22 (45.8%) | 62 (42.2%) | 0.595 |
Abbreviation: CI, confidence interval.
Declaration of conflicting interest
The authors declare that there is no conflict of interest.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
References
- 1.Hustey FM, Meldon SW. The prevalence and documentation of impaired mental status in elderly emergency department patients. Ann Emerg Med 2002; 39: 248–253. [DOI] [PubMed] [Google Scholar]
- 2.Lowthian J, Straney LD, Brand CA, et al. Unplanned early return to the emergency department by older patients: the Safe Elderly Emergency Department Discharge (SEED) project. Age Ageing 2016; 45: 255–261. [DOI] [PubMed] [Google Scholar]
- 3.Huded JM, Dresden SM, Gravenor SJ, et al. Screening for fall risks in the emergency department: a novel nursing-driven program. West J Emerg Med 2015; 16: 1043–1043. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Dramé M, Mahmoudi R, Jolly D, et al. Social support and six-month outcome among elderly patients hospitalised via emergency department: The SAFES Cohort Study. Eur Geriatr Med 2013; 4: 161–166. [Google Scholar]
- 5.Foo CL, Siu VWY, Tan TL, et al. Geriatric assessment and intervention in an emergency department observation unit reduced re-attendance and hospitalisation rates. Australas J Ageing 2012; 31: 40–46. [DOI] [PubMed] [Google Scholar]
- 6.Taylor BJ. Developing an integrated assessment tool for the health and social care of older people. Br J Soc Work 2012; 42: 1293–1314. [Google Scholar]
- 7.Ellis G, Whitehead MA, Robinson D, et al. Comprehensive geriatric assessment for older adults admitted to hospital: meta-analysis of randomised controlled trials. BMJ 2011; 343: d6553–d6553. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Meldon SW, Mion LC, Palmer RM, et al. A brief risk-stratification tool to predict repeat emergency department visits and hospitalizations in older patients discharged from the emergency department. Acad Emerg Med 2003; 10: 224–232. [DOI] [PubMed] [Google Scholar]
- 9.Goldstein J, Andrew MK, Travers A. Frailty in older adults using pre-hospital care and the emergency department: a narrative review. Can Geriatr J 2012; 15: 16–22. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.World Population Aging 2013. New York: United Nations, Department of Economic and Social Affairs, Population Division; 2013 [Available from: http://www.un.org/en/development/desa/population/publications/pdf/ageing/WorldPopulationAgeing2013.pdf].
- 11.Moonesar R, Sammy I, Nunes P, et al. Social support in older people: lessons from a developing country. Qual Life Res 2016; 25: 233–236. [DOI] [PubMed] [Google Scholar]
- 12.Rawlins JM, Simeon DT, Ramdath DD, et al. The elderly in Trinidad: Health, social and economic status and issues of loneliness. West Indian Med J 2008; 57: 589–595. [PubMed] [Google Scholar]
- 13.Yao JL, Fang J, Lou QQ, et al. A systematic review of the identification of seniors at risk (ISAR) tool for the prediction of adverse outcome in elderly patients seen in the emergency department. Int J Clin Exp Med 2015; 8: 4778–4778. [PMC free article] [PubMed] [Google Scholar]