Using trigger tools to identify triage errors by ambulance dispatch nurses in Sweden: an observational study

Abstract

Objectives

This study aimed to assess whether trigger tools were useful identifying triage errors among patients referred to non-emergency care by emergency medical dispatch nurses, and to describe the characteristics of these patients.

Design

An observational study of patients referred by dispatch nurses to non-emergency care.

Setting

Dispatch centres in two Swedish regions.

Participants

A total of 1089 adult patients directed to non-emergency care by dispatch nurses between October 2016 and February 2017. 53% were female and the median age was 61 years.

Primary and secondary outcome measures

The primary outcome was a visit to an emergency department within 7 days of contact with the dispatch centre. Secondary outcomes were (1) visits related to the primary contact with the dispatch centre, (2) provision of care above the primary level (ie, interventions not available at a typical local primary care centre) and (3) admission to hospital in-patient care.

Results

Of 1089 included patients, 260 (24%) visited an emergency department within 7 days. Of these, 209 (80%) were related to the dispatch centre contact, 143 (55%) received interventions above the primary care level and 99 (38%) were admitted to in-patient care. Elderly (65+) patients (OR 1.45, 95% CI 1.05 to 1.98) and patients referred onwards to other healthcare providers (OR 1.58, 95% CI 1.15 to 2.19) had higher likelihoods of visiting an emergency department. Six avoidable patient harms were identified, none of which were captured by existing incident reporting systems, and all of which would have received an ambulance if the decision support system had been strictly adhered to.

Conclusion

The use of these patient outcomes in the framework of a Global Trigger Tool-based review can identify patient harms missed by incident reporting systems in the context of emergency medical dispatching. Increased compliance with the decision support system has the potential to improve patient safety.

Strengths and limitations of this study.

• A strength of the study was the use and presentation of detailed journal review criteria allowing for a high degree of reproducibility.

• A strength of the study is application of a widely used quality assurance framework (the Global Trigger Tool) to investigate a novel context (emergency medical dispatch) for which triggers have not been previously defined.

• A limitation of the study was that journal abstraction and review were performed by employees of the studied regions rather than independent observers.

• A limitation of the study is that some review criteria (care above the primary level and contact relationship) had only acceptable inter-rater reliability.

Introduction

In the practice of emergency medical dispatch (EMD), patients not in need of an ambulance or other forms of emergency care are in some systems directed to less resource-intensive healthcare services including nursing advice lines, alternative forms of transport to healthcare facilities or self-care.^{1 2} Ruling out conditions requiring emergency care over the telephone is difficult, leading to significant rates of ‘over-triage’ and potentially unnecessary ambulance responses.^3–5 The 2016 Effective Care investigation established the principle that patients in Sweden should be treated at the lowest effective level of care.⁶ EMD systems incorporating triage to non-emergency care often employ a combination of highly trained staff, typically nurses and decision support systems to ensure safety in the triage process.1 2 7–9 Any such triage practice comes with the risk of missing true emergencies requiring an ambulance, or ‘under-triage’. Given the serious consequences of missing such patients, the safety of this practice should be continuously and systematically assessed.

A common source of data for quality assessment in Swedish prehospital care are incident reports as generated by patients or healthcare providers. Such reporting systems have, however, been found to miss a substantial portion of adverse events in the context of hospital care.^{10 11} A number of studies of telephone triage have utilised simulated patients or review by an expert panel to determine accuracy,^{1 12 13} but there are few studies investigating Emergency Department (ED) resource utilisation by patients referred by EMD centres to non-emergency care.¹⁴ The rate of return visits following ED care is an established measure of care quality in emergency department care,^15–19 and has also been used in evaluating on-scene triage by ambulance personnel.²⁰ Various follow-up durations have been used in prior studies, commonly employing 1–7-day follow-up intervals, and investigating either all-cause or related ED contacts only.^{14 20–22}

The Global Trigger Tool (GTT)²³ has seen widespread use in assessing the quality of hospital-based care,10 21 24 25 and has also been used in evaluating ambulance care.^{26 27} This method involves the structured review of patient records to identify specific indicators thought to be associated with adverse events and patient harm. These indicators are then used to identify cases requiring more in-depth review. Using these tools, the efficiency of the review process in terms of the number of work-hours required to identify each adverse event can thus be improved.²⁴ The use of this method however necessitates the definition of indicators appropriate to the context in which the tool will be applied. In reviewing the literature, we found no published criteria for identifying potential patient harms in the context of EMD triage. In this study we thus aimed to propose and evaluate a set of criteria for use within the GTT framework to identify triage errors by EMD nurses. A secondary aim was to describe the characteristics of this patient cohort.

Methods

Setting

Sweden is divided into 21 regions responsible for providing healthcare services. The two EMD centres included in this study serve regions with areas of 5146 km² (region 1) and 8209 km² (region 2), and had populations of 267 629 and 361 373, respectively, in 2016.²⁸ The EMD centres handled 88 293 calls in 2016, resulting in 65 358 ambulance responses (a response rate of 74%). Both EMD centres exclusively employ nurses in the primary call-taking role, including both registered nurses with 3 years of university education and clinical nursing specialists with an additional year of education. On employment, nurses receive 5 weeks of training, consisting of 2 weeks of theory and 3 weeks of practical training with a senior nurse mentor. Both EMD centres implemented the same computerised clinical decision support system (CDSS) in May 2015.²⁹ Both studied regions use incident reporting systems in which care providers, patients and relatives can report adverse events, as mandated by Swedish law.

It is the current clinical practice at both EMD centres to refer patients who are determined not to require an ambulance to alternate forms of transport to an ED (not investigated in this study), or to non-emergency care. Possible non-emergency patient dispositions include transport via ambulette, taxi or other means to a primary care facility, the dispatch of mobile geriatric or psychiatric care teams, referral to a nursing advice line, referral to poison control and referral to other on-scene healthcare personnel. Dispatchers may also choose to end the call without such an onward referral (ie, self-care). A full list of dispositions and associated patient volumes is provided in the second table of online supplementary appendix 1.

Study design

An observational study of patients triaged to non-emergency care by nurses at EMD centres in two Swedish counties from 15 October 2016 to 14 February 2017 was performed. The Marker-based Journal Review method,³⁰ the Swedish adaptation of the GTT,²⁴ was used to guide the process. Personal Identification Numbers (PINs) captured in the EMD call were used to identify corresponding medical records at the hospital. The first ED visit within a 7-day time frame following the EMD contact was identified, and each visit was classified with regards to the presence of the secondary outcomes. Cases where a patient harm could not be ruled out, as well as cases which were difficult to classify with regards to the outcomes, were marked for further review. Preliminary reviews were completed by senior nurses at each dispatch centre, one with 18 years of clinical experience and a specialist nursing degree in primary care and one with 20 years of clinical experience and a specialist degree in ambulance care (authors JCB and HBo). Secondary reviews were performed in group sessions including both nurses and the medical directors of both dispatch centres, both of whom have backgrounds in anaesthesia/intensive care and ambulance care, with 35 and 25 years of clinical experience, respectively (authors LE and HBl). In group sessions including all reviewers, patient hospital records were reviewed to establish the presence of a patient harm. The group then listened to the audio recording of the call, and the patient interview was evaluated with regards to adherence with clinical guidelines. Identified patient harms were classified using the National Coordinating Council for Medication Error Reporting and Prevention index (NCC MERP).²⁴

We sought to minimise bias by adhering to established guidelines in performing the review,³¹ and in reporting our results.³² The outcomes and independent variables were selected a priori based on the existing literature and clinical judgement. A 1-month, single-site pilot study was performed to establish the feasibility of the methodology, adjust the definitions of the criteria used in the study, and to establish the duration of the main study. During the pilot study, the two nurses reviewed data together to ensure that classifications were made uniformly, and reviewers met regularly during the main study to discuss criteria interpretation. Reviewers were blinded to findings in the statistical analysis until the end of the data collection phase.

Patient and public involvement

This research was done without patient or public involvement.

Participants

All patients contacting the EMD centres and who were directed to non-emergency care during the study period were considered for inclusion in the study. Patients with a missing or invalid PIN, patients under 18 years of age, patients calling from municipalities close to EDs situated outside of the studied regions and calls miscategorised by dispatchers were excluded.

Outcomes

The primary outcome was whether a patient visited an ED within 7 days of triage by the EMD centre to non-emergency care. Secondary outcomes consisted of whether the primary complaint recorded in the hospital record was related to the reason for the primary contact with EMD centre, whether interventions above the primary care level were provided (also referred to as ‘specialist care’) and whether the patient was admitted to an in-patient hospital ward. Detailed definitions of the criteria used to establish the presence or absence of these outcomes in the patient medical records are provided in online supplementary appendix 2.

The Swedish patient safety law (SFS 2010:659) was used to define healthcare related harms. This law defines the term as: Suffering, physical or mental injury or disease, or mortality which could have been avoided if adequate measures had been taken upon the patients contact with the healthcare provider.³³ We considered a patient harm to have met this definition if (1) provision of an ambulance response without delay on contact with the EMD centre could have alleviated suffering (eg, by providing pain relieving medication) or potentially improved the patient’s outcome and (2) the dispatcher failed to perform an adequate patient interview which could have revealed the need for an ambulance (ie, a triage error occurred). The presence of a patient harm was determined through the review of hospital medical records, while the presence of a triage error was established by listening to the audio recording of the call and comparing the determinations made by nurses with applicable clinical guidelines.

Independent variables

We investigated eight independent variables which we believed a priori could impact ED utilisation rates following triage to non-emergency care. Owing to the presence of non-linear effects in the continuous variables, each was dichotomised at an appropriate threshold value. Variables included: (1) the gender of the patient, (2) the age of the patient dichotomised as older or younger than 65 years of age (ie, the age at which eldercare services become available), (3) whether the call occurred on a weekday or weekend, (4) whether the contact occurring during the daytime (ie, during the typical primary care centre open hours of 07:00 to 17:00), (5) whether the patient frequently contacted the EMD centre, defined as six contacts or more within the previous 6 months (note that definition of ‘frequent utilisation’ varies widely in the literature),^{34 35} (6) the region in which the call occurred, (7) whether the CDSS had been used while triaging the patient, defined as the dispatcher having at a minimum selected a call type and viewed the CDSS and (8) whether the patient was referred onwards to another healthcare provider or was provided with advice only.

Statistical analysis

Data from the journal review process were recorded in Excel (2013), and R v.3.5.3³⁶ was used to perform the statistical analysis. Characteristics of the patient population were described using means, medians and proportions as appropriate, with 95% CIs generated based on the percentiles of 1000 non-parametric bootstraps as implemented in the ‘boot’ R package.³⁷ Claims regarding the population of patients experiencing avoidable harms were tested using a two-sided Wilcoxon signed rank test.³⁸ There were no missing data to handle following application of the exclusion criteria.

The association between the independent variables and the likelihood of visiting an ED following referral to non-emergency care by the EMD centre was investigated using logistic regression, reporting ORs and their 95% CIs. To avoid violating the assumption of independent observations, only the first contact of each patient with the EMD centre during the study period was included in the analysis. The sensitivity of our results to alternate choices of follow-up duration (1-day, 3-day or 7-day), the exclusion of non-related visits, the use of secondary outcomes (specialist care and hospital admission) and the use of covariate adjustment was assessed as reported in online supplementary appendix 3. In the regression analysis, we dichotomised continuous variables to conserve model degrees of freedom in the presence of non-linear effects, and a bivariate analysis of the dichotomised variables (age, call hour and prior contacts) using locally estimated scatterplot smoothing is provided in online supplementary appendix 4. We assessed inter-rater reliability by having both nurses review 1 month of records at one site, and agreement was described both in absolute terms and using Cronbach’s alpha.³⁹

Results

Participants

A total of 32 380 records were identified in the EMD system over the 4-month study period, with 15 254 captured in region 1 and 17 126 in region 2. As presented in figure 1, 21 779 (67%) were patient contacts, with the remainder of records consisting of stand-by requests, duplicate calls, misdirected calls and so on. Of the actual patient calls, 1 876 (9%) had a non-emergency disposition which met the inclusion criteria for this study, with the remainder either receiving an ambulance, or being directed to other forms of emergency care (most commonly the use of alternate means of transport to an ED). The exclusion of 787 records (42%) resulted in a final study population of 1089, with the largest cause of exclusion being the non-documentation of a valid PIN (commonly due to third-party callers, calls from patients with reduced mental status or dispatchers simply forgetting to ask). Categorisation errors consisted mainly of patients directed to the ED via alternate transport which were documented as referrals to other healthcare providers or as a decision to close the call without onward referral, as reported in the third table in online supplementary appendix 1.

Figure 1.

Inclusion flow chart. This figure describes the steps taken to extract the cohort of included patients from the full set of dispatch records collected over the course of the study. Patient volumes are reported both overall and for each region (1/2), respectively. *PIN = personal identification number.

Descriptive statistics for all patients, the population of patients with non-emergency dispositions prior to applying exclusion criteria, and for the included patients are presented in table 1. Patients with non-emergency dispositions were younger in comparison with the full population (a median age of 57 years vs 69 years among all patients) and contacted the EMD centre more often (14% vs 4% frequent callers) than the population of all patients calling the EMD centre. A lower percentage of calls also occurred during the daytime among non-emergency dispositions (48% vs 59%). The CDSS was used less frequently for callers referred to non-emergency care (60% vs 84%). The proportion of patients with a missing or invalid PIN was also higher among patients referred to non-emergency care (30% vs 9%). The application of the exclusion criteria resulted in a higher median age (due to the exclusion of patients under 18 years of age) and a higher rate of CDSS utilisation. Differences were identified between regions with regards to patient age (with region 1 having older patients), the proportion of frequent callers (with a larger proportion of frequent callers on region 1), the proportion of onward referrals (more onward referrals in region 1), and the rate of missing PINs (more missing PINs in region 2).

Table 1.

Comparison of patient characteristics

	All patients	Non-emergency	Included in study
n	21 779	1876	1089
By region	(10917/10862)	(1005/871)	(602/487)
Female (%)	51.9 (51.3–52.6)	52.1 (49.5–54.8)	53.3 (50.3–56.2)
By region	(52.1/51.8)	(50.7/53.9)	(51.8/55)
Age (median)	69 (68–69)	57 (56–60)	61 (59–64)
By region	(70/67)	(61/52)	(66/57)
Weekday (%)	86.1 (85.5–86.5)	85.5 (83.9–87)	85.6 (83.3–87.6)
By region	(85.9/86.3)	(84.6/86.6)	(85.5/85.6)
Daytime* (%)	58.8 (58.1–59.4)	47.9 (45.6–50.3)	47.9 (45.1–51)
By region	(58.9/58.6)	(46.5/49.5)	(48.2/47.6)
Frequent caller (%)	4 (3.7–4.2)	13.7 (11.7–15.6)	15.7 (13.5–17.8)
By region	(5.3/2.5)	(18.7/7.1)	(21.8/8.2)
Region 2 (%)	49.9 (49.3–50.5)	46.4 (44.1–48.9)	44.7 (41.8–47.7)
By region	(0/100)	(0/100)	(0/100)
CDSS used (%)	83.9 (83.4–84.4)	60.3 (57.9–62.6)	69.8 (66.9–72.4)
By region	(84.6/83.2)	(61.8/58.7)	(68.4/71.5)
Onward referral (%)	4.6 (4.3–4.9)	53.6 (51.5–55.9)	52.3 (49.3–55.3)
By region	(5.6/3.6)	(61.3/44.8)	(61.8/40.7)
Missing/invalid PIN (%)	9 (8.6–9.3)	29.9 (27.8–31.9)	0 (0–0)
By region	7.9/10.1	26.2/34.1	0/0

This table presents descriptive statistics for all patients contacting the EMD centre during the study period, patients with a non-emergency disposition meeting the inclusion criteria for the study and the patient cohort following the application of the exclusion criteria. All statistics are presented with their bootstrapped 95% CIs. Statistics are presented for each region separately.

*Defined as between 07:00 and 17:00.

†Patients referred to the nursing advice line, alternate transport to a non-ED destination, an on-site healthcare provider, a mobile care team or the poison control centre, as compared with patients receiving advice only.

CDSS, clinical decision support system; EMD, emergency medical dispatch.

Outcome data

Of 1089 included patients, 258 (24%) visited the ED within 7 days of contacting the EMD centre. Of these, 208 (81%) had a complaint related to the EMD centre contact, 143 (55%) received interventions above the primary care level and 99 (38%) were admitted to in-patient care.

The distribution of ED contacts by the number of elapsed hours between contact with the EMD centre and contact with the ED is presented in figure 2. Of all ED contacts, 68% occurred within the first 24 hours. Additional contacts continued to occur until the end of the follow-up period at 7 days. The percentage of related calls was highest during the first 24 hours, with 88% of ED visits being related to the reason for contacting the EMD centre. This rate decreased to 73% during the 2–3-day time frame, and to 61% over the final 4 days. We identified no clear trends with regards to the other secondary outcomes over time. ED contacts involving a patient harm occurred up to 73 hours following the EMD contact.

Figure 2.

Time from emergency medical dispatch (EMD) call to ED visit. This figure presents a histogram depicting the number of elapsed hours between contact with the EMD centre and the ED visit. Each column represents a span of 2 hours, with vertical lines placed at threshold intervals commonly found in the literature. Black dots represent the time at which patients with an avoidable patient harm arrived at the ED.

Logistic regression analysis

On excluding repeat contacts, 903 unique patients remained, 213 (24%) of whom had a subsequent ED visit within 7 days. As reported in table 2, patients over the age of 65 years were more likely to visit the ED within 7 days with an adjusted OR of 1.48 (1.08–2.04). Being referred onwards to another healthcare provider was also associated with a higher likelihood of visiting the ED, with an adjusted OR of 1.58 (1.15–2.19). The differences in ED visitation likelihood between regions were substantially reduced on adjusting for differences in the other covariates.

Table 2.

Association of independent variables with ED visitation within 7 days

	Unadjusted OR (95% CI)	Adjusted OR (95% CI)
Female	0.97 (0.71 to 1.32)	0.93 (0.68 to 1.27)
Elderly (65+)	1.46 (1.07 to 1.99)	1.48 (1.08 to 2.04)
Weekday	1.02 (0.67 to 1.59)	0.95 (0.61 to 1.49)
Daytime*	1.04 (0.77 to 1.42)	0.97 (0.71 to 1.33)
Frequent caller	1.24 (0.64 to 2.29)	1.30 (0.94 to 1.81)
Region 2	0.67 (0.49 to 0.91)	0.77 (0.56 to 1.07)
Decision support used	0.87 (0.62 to 1.22)	0.95 (0.67 to 1.35)
Onward referral†	1.62 (1.19 to 2.21)	1.58 (1.15 to 2.19)

Logistic regression coefficients are presented as ORs and their 95% CIs in models adjusted for the effect of all other covariates presented in the table.

*Defined as occurring between 07:00 and 17:00.

†Patients referred to the nursing advice line, alternate transport to a non-ED destination, an on-site healthcare provider, a mobile care team or the poison control centre as compared with patients receiving advice only.

The sensitivity analysis reported in online supplementary appendix 3 suggested that, with the exception of patient age and onward referral, the effects found with regards to the independent variables were sensitive to alternate model specifications. The results of this analysis suggested that the effect of age was stronger when investigated with regards to outcomes reflecting a higher level of care intensity (ie, specialist care and hospital admission). The effect of onward referral on the higher intensity outcome measures was, however, weaker than those found with regards to ED visitation.

Inter-rater reliability analysis

In order to assess the inter-rater reliability of the proposed outcome criteria, 1 month of data in one region were reviewed by both nurses separately. A total of 139 cases (13% of the total number of included records) were reviewed by both nurses, who could choose to either definitively classify the record with regards to each outcome or abstain from doing so. Inter-rater agreement for classification of ED visits was 98.5% (standardised Cronbach’s alpha 0.94) and all calls were definitively classified by the nurses. Hospital admission had an agreement of 93.7% (alpha=0.94) with a single abstention from definitive classification. The rating regarding the relationship between the complaint at the ED and the complaint at the EMD had a lower reliability (78.1%, alpha=0.71), with a large portion of records being referred onwards for further review (22% and 28% for each abstractor). Agreement was also lower as to whether specialist care had been provided or not (62.5%, alpha=0.7), with an abstention rate of 0% and 12% for each abstractor. Five out of seven cases of disagreement regarding the level of care were due to whether standard panel blood tests were to be considered specialist care.

Identification of avoidable patient harms

In order to identify cases of avoidable patient harm, all records in which a patient harm could not be ruled out by the nurses were reviewed and discussed in group sessions. A total of 32 cases of potential patient harm were identified on primary review of hospital records by the two nurses. These cases, along with 48 cases of uncertainty with regards to one or more outcome criteria, were assessed by the group. Of these, six cases fulfilling the definition of avoidable patient harm were identified, while the remainder of cases were determined not to constitute an avoidable patient harm. Cases were marked as not an avoidable harm for two primary reasons: That the patient’s condition would not have been improved by sending an ambulance, or that the patient’s later condition at the ED was the result of deterioration which could not reasonably have been anticipated by the dispatch nurse. The appropriateness of one additional triage decision could not be established due to a missing audio file.

All patient harms were found among patients both receiving specialist care and admitted to in-patient care. We found that in all cases, strict adherence to clinical guidelines as encoded in CDSS would have resulted in ambulance dispatch. In three of the six cases of patient harm, the CDSS had not been used at all, while in the others the recommended priority had been over-ridden. While the number of identified patients harms was too low to enable detailed statistical analysis, we performed Wilcoxon signed rank tests to verify the significance of two post hoc findings: That a 72-hour interval would capture the majority of avoidable patient harms (p=0.06), and that patients experiencing an avoidable harm were older than the population of all patients triaged to non-emergency care (at a median age of 79.5 years vs 61 years, p=0.03).

On application of the NCC MERP criteria, one patient was classified as suffering an error which may have contributed to permanent harm (category G), which involved the delayed provision of antibiotics to a patient later diagnosed with diverticulitis. Two patients suffered from errors which may have contributed to prolonged hospitalisation (category F), and two suffered from errors causing temporary harm (category E). These errors involved the non-provision of an ambulance to a psychiatric patient lacking decisional capacity, and several elderly patients with pain who suffered a delay in the administration of pain-relieving medication. One patient initially deemed to have suffered from an avoidable patient harm was categorised as suffering only a potential harm (category C) based on NCC MERP criteria. A single patient death within 7 days was identified, but it was found to have been appropriately triaged, and would not have been prevented by earlier care at the ED. On reviewing the incident reporting systems of each region during study time frame, no incident reports relating to the non-provision of ambulances were identified.

Discussion

In this study, we found that 24% of patients referred by EMD nurses to non-emergency care subsequently visited an ED within 7 days. Of these, 80% had a complaint related to the initial EMD contact, 55% received care which could not have been provided by a primary care centre and 38% were admitted to in-patient care. Using the trigger tool methodology, we identified six triage errors resulting in a patient harm, all of which were missed by existing incident reporting systems. In all identified cases involving an avoidable patient harm, strict adherence to the CDSS would have resulted in the dispatch of an ambulance. Effects found in the logistic regression analysis were generally weak, but there were indications that elderly patients (65+) and patients referred onwards to other healthcare providers were more likely to visit an ED within 7 days.

Only a few studies have investigated patient outcomes directly comparable to this study. One Danish study of low acuity callers to EMD centres found a rate of ED visitation within 7 days of 26%.¹⁴ In the Danish study however, callers directed to the ED were not excluded, and the true comparable rate to the 24% found in our study lies between 10% and 26%. A systematic review identified several studies investigating non-conveyance by ambulances using subsequent ED visit as a measure, identifying rates ranging from 4.6% to 19% with varying follow-up periods.²⁰ In the Swedish context, a pilot study of triage by a single responder unit found a 3-day related ED visitation rate of 19% among patients left at the scene or referred to primary care (n=200), with a rate of admission to in-patient care of 53%.²² The corresponding 3-day figures identified in our study were 16% and 37%, respectively. However, the patient populations investigated in these studies vary widely, and conclusions based on these comparisons should be cautious.

We found that all the patient harms identified in this study occurred among patients admitted to the hospital, and that only a single ED contact associated with a patient harm occurred outside a 72-hour window (at 73 hours). Our results suggest that in performing quality improvement work, reviewing the records of patients admitted to the hospital within 3 days of referral to non-emergency care is likely to capture the bulk of avoidable patient harms, while reducing the number of records necessary to review by a factor of over 10 as compared with random sampling. The rate of CDSS utilisation among patients triaged to non-emergency care was comparatively low, and we found that an ambulance would have been dispatched in all cases of avoidable patient harm if the CDSS had been strictly adhered to. As such, quality improvement interventions intended to improve CDSS utilisation rates have the potential to improve the safety of the triage process and reduce under-triage. Such interventions have been implemented at both EMD centres, though compliance with the CDSS has not been made mandatory. While increased compliance with the CDSS is likely to reduce under-triage, it may also have the consequence of increasing rates of over-triage thus reducing ambulance availability for true emergencies. The overall impact of deviations from CDSS in terms of both over-triage and under-triage (as opposed to this study which was concerned with only under-triage) should be investigated further.

Our analysis suggests that the elderly constitute a cohort of patients for whom improved care pathways may be needed. Despite being referred to non-emergency care less often, elderly patients appeared more likely to visit the ED following such a decision, and even more likely to receive more intensive forms of care on arriving at the ED. Our findings also suggested that patients experiencing an avoidable harm were older than average. Our results thus indicate that a high degree of caution is warranted in triaging this cohort of patients. We found that patients referred to non-emergency care often had non-specific call types (eg, ‘general elderly’), multiple call types or no call type at all documented in the CDSS as noted in online supplementary appendix 3. This emphasises the diffuse nature of the complaints found in this patient cohort. Our results also suggested that while patients referred onwards to other healthcare providers were more likely to visit an ED, they were not more likely to receive specialist care or be admitted as may be seen in online supplementary appendix 3. One explanation for this could be that the care providers who later assessed these patients were more likely to direct patients not requiring specialist-level or in-patient care to the ED.

There are several limitations to this observational study. The data collection duration of 4 months leaves open the possibility that there is uncaptured seasonal variation. It is also likely that additional ED visits continued to occur beyond the 7-day interval during which we gathered data. Based on the distribution of the identified patient harms, it is however unlikely that additional patient harms occurred beyond this interval. The exclusion of patients for whom no valid PIN was obtained was also a potential source of bias, and additional harms may have occurred among these patients. Our criteria for avoidable patient harms were designed to capture triage errors suitable for quality improvement interventions and were based on Swedish legal definitions and local clinical guidelines which may have limited applicability to other contexts. While previous studies have investigated the rate of potential patient harms,^{26 27} we found that a clear definition of a ‘potential’ patient harm in this context was difficult to arrive at. As such, we chose to limit this study to the investigation of concrete, avoidable patient harms only. Due to a lack of sufficiently detailed definitions of ‘specialist care’ and ‘related symptoms’ in the literature, refinement was needed during the review process, and as previously noted, some deviation from these definitions was identified on a review of disagreements. Reviews were furthermore performed by employees of the studied regions, rather than independent observers. While Cronbach’s alpha values of 0.7 may considered acceptable,³⁹ further refinement of the ‘specialist care’ and ‘contact relationship’ criteria could improve reliability. This study took place in the uncommon setting of fully nurse-staffed EMD centres, and the results should be generalised only with caution to EMD centres staffed by personnel with other levels of training, other CDSS and protocols, or settings where other alternatives to emergency care are available.

The trigger tool-based review process was successful in identifying several avoidable patient harms which were not detected by existing incident reporting systems, and levels of subsequent ED utilisation were comparable to those found in previous studies. Elderly patients were more likely to be cared for at an ED following triage to non-emergency care, and extra caution is warranted in triaging this patient cohort. Our results suggest that review of ED visits resulting in hospital admission within 3 days of contact with the EMD centre would capture the majority of patient harms occurring as the result of a triage error, while minimising the costs associated with dedicating staff to manually reviewing records.