Safety of telephone triage in general practitioner cooperatives: do triage nurses correctly estimate urgency?

Abstract

Background

In recent years, there has been a growth in the use of triage nurses to decrease general practitioner (GP) workloads and increase the efficiency of telephone triage. The actual safety of decisions made by triage nurses has not yet been assessed.

Objectives

To investigate whether triage nurses accurately estimate the urgency level of health complaints when using the national telephone guidelines, and to examine the relationship between the performance of triage nurses and their education and training.

Method

A cross‐sectional, multicentre, observational study employing five mystery (simulated) patients who telephoned triage nurses in four GP cooperatives. The mystery patients played standardised roles. Each role had one of four urgency levels as determined by experts. The triage nurses called were asked to estimate the level of urgency after the contact. This level of urgency was compared with a gold standard.

Results

Triage nurses estimated the level of urgency of 69% of the 352 contacts correctly and underestimated the level of urgency of 19% of the contacts. The sensitivity and specificity of the urgency estimates provided by the triage nurses were found to be 0.76 and 0.95, respectively. The positive and negative predictive values of the urgency estimates were 0.83 and 0.93, respectively. A significant correlation was found between correct estimation of urgency and specific training on the use of the guidelines. The educational background (primary or secondary care) of the nurses had no significant relationship with the rate of underestimation.

Conclusion

Telephone triage by triage nurses is efficient but possibly not safe, with potentially severe consequences for the patient. An educational programme for triage nurses is recommended. Also, a direct second safety check of all cases by a specially trained GP telephone doctor is advisable.

Out‐of‐hours general practitioner (GP) care in Europe is increasingly being handled by large‐scale organisations (ie, GP cooperatives), with nurses generally handling the telephone triage. This trend is in response to the increased demand for GP care and attempts to reduce their workload.¹,²,³,⁴,⁵,⁶ Research shows that the handling of telephone triage by nurses does indeed cause a substantial decrease in the immediate workload and increased efficiency of GPs.³,⁴,⁵,⁶,⁷ On the other hand, some studies have shown that subsequent GP consultations increase after nurse‐led telephone triage.⁷,⁸,⁹

Telephone triage is nevertheless considered the most complex and vulnerable part of the out‐of‐hours GP care process.¹⁰,¹¹,¹² It has not yet been proven safe, perhaps due to underestimation of complaints.⁴,⁷,¹⁰,¹¹,¹² There is no research on the effects of specific education and training on the efficiency and safety of telephone triage. Perhaps that is why it differs in different countries: Denmark opts for GPs² and England and The Netherlands opt for nurses who have a primary‐ or secondary‐care background.¹,³,¹¹

To improve the balance between safety and efficacy the care provided by triage nurses, explicit national telephone guidelines were implemented The Netherlands. The telephone guidelines include a classification system with four levels of urgency.¹⁰

In this study, we examined the ability of triage nurses to adequately estimate the urgency of health problems presented via the telephone, using the telephone guidelines. We also examined the correlations between underestimation of the urgency estimates, educational backgrounds of the triage nurses and their training on the telephone guidelines.

Methods

Design

A cross‐sectional, multicentre, observational study employed five mystery (simulated) patients who telephoned triage nurses in four GP cooperatives. Each of the five mystery patients played four different standardised roles. We chose mystery patients for investigation, because they mimic the reality in care quite reliably and naturally.¹³,¹⁶

Population

The study was conducted in the last quarter of 2003 and included 118 triage nurses from four GP cooperatives in The Netherlands (box 1). We attempted to obtain a representative picture of the quality of triage in The Netherlands by selecting cooperatives with different periods of existence from different regions with different levels of urbanisation.

Box 1: Features of the 120 GP cooperatives in The Netherlands³,¹¹

• Mostly situated near or within a hospital.

• Access via a single regional telephone number.

• Large‐scale handling of 100 000–500 000 patients within distances up to 20–30 km.

• Chauffeurs in recognisable GP cars, which are fully equipped (eg, O₂, infusion drip, automatic defibrillation).

• Information and communications technology support including electronic patient files, electronic feedback to the GPs and online connection to the GP car.

• Triage nurses on the telephone (primary‐ or secondary‐care background).

• GP shifts of 6–8 h.

Instruments and procedures

A total of 20 Dutch vignettes were written on the basis of practice cases matched to the protocols from the national telephone guidelines. The vignettes represented different complaints, different levels of urgency and different age and sex groups in the most natural manner possible. Life‐threatening cases were included slightly more often than in actual daily life. A panel of seven GP experts evaluated the medical accuracy, completeness, representativeness and level of urgency for the vignettes. After modifying the vignettes, the panel of experts again judged the level of urgency for the different vignettes. It was decided a priori that at least six of the seven experts had to assign the same level of urgency to a vignette and that such agreement had to be achieved for >95% of the vignettes. Using this gold standard, only 6 (or 4%) of the 140 judgements (7 experts × 20 vignettes) were found to disagree with regard to the level of urgency for a vignette. Table 1 gives an overview on the vignettes.

Table 1 Vignettes, including fake name and birth date.

Name	Complaint	Birth date	Urgency class
1. Mrs Aalbers	Painful leg and fever	22–10–1951	U3
2. Mr Burgers	Dyspnoea	15–12–1938	U2
3. Mr Cornelissen	Diarrhoea	06–03–2002	U4
4. Mr Van Doorn	Chest pain	04–06–1943	U1
5. Mrs Everts	Back pain	02–04–1974	U4
6. Mr Fontijn	Constipation	02–09–1950	U4
7. Mrs Gerritsen	Paralyses	02–10–1939	U2
8. Mrs Hoogakker	Intoxication/suicide treat	03–09–1980	U1
9. Mrs Van Ingen	Diabetes/hypoglycaemia	03–04–1953	U2
10. Mrs Jansen	Ankle trauma	03–07–1978	U3
11. Mr Klaassen	Abdominal pain	05–02–1948	U3
12. Mrs De Lange	Abdominal pain and pregnant	06–10–1975	U3
13. Mr Maassen	Collapse	10–02–1989	U4
14. Mrs De Nooy	Throat pain	12–01–1960	U4
15. Mrs Otten	Pregnant and bleeding	12–12–1970	U4
16. Mrs Peters	Painful eye	06–06–2003	U4
17. Mrs Roelofsen	Dizziness	04–04–1957	U4
18. Mr Schipper	Anal bleeding	10–12–1945	U3
19. Mrs Timmer	Headache	12–11–1978	U4
20. Mrs De Vries	Contraception forgotten	02–03–1982	U4

The mystery patients were very experienced in their job as simulation patient at the Radboud University Nijmegen Medical Centre and were able to mimic reality quite naturally.¹³,¹⁴,¹⁵,¹⁶ The five mystery patients were trained to play four vignettes matched with respect to content and emotion. We checked whether the mystery patients played the vignettes naturally and reliably in a pilot study and also by listening to all the audio‐taped contacts on a weekly basis.

We made an agreement with the four GP cooperatives with respect to logistics and safety.

The triage nurses were informed of the purpose of the present research and all consented to participate. The triage nurses completed a questionnaire about their education in primary care (2–3 years) or secondary care (4 years). They also provided information on specific training received on the use of national telephone guidelines.

The mystery patients phoned the GP cooperative at random times in out‐of‐hours. The triage nurses did not know whether the individual on the telephone was a mystery patient and they presumably dealt with the contact in accordance with their own normal routines. At the end of the call or right before the triage nurse went to take action, the standardised patients revealed themselves. They asked the triage nurse her decision regarding the level of urgency on the triage criteria outlined in box 2.

Box 2: Urgency criteria from national telephone guidelines¹⁷

Life threatening (U1): Vital functions are in danger. Triage nurse informs GP at once. GP interrupts work and immediately goes to patient. When necessary, ambulance is simultaneously called.

Acute (U2): Real danger of patient's condition quickly deteriorating with risk of vital functions breaking down. The triage nurse informs GP at once. GP goes to patient as soon as possible—within 1 h at most.

Urgent (U3): Complaint(s) should be evaluated within 2 h for medical or emotional reasons.

Routine (U4): Complaint(s) not urgent. Triage nurse arranges an appointment with the GP or gives advice him/herself.

Each of the 20 vignettes was presented 18–20 times equally spread out across the four GP cooperatives, so that each cooperative received the same set.

Consultations that were prematurely terminated and those lacking an urgency estimation were excluded from the analyses.

Analysis

We compared the urgency score assigned by the triage nurse with the gold standard.

We calculated the sensitivity, specificity and predictive values of the estimated urgency. For this purpose, a dichotomy was created between the U1–U2 life‐threatening or potentially life‐threatening categories of urgency and the remaining U3–U4 categories.

Next, we analysed the underestimation of urgency in the subset U1–U2 life‐threatening cases. We carried out a multilevel logistic regression analysis, with underestimation of urgency as the dependent variable and educational background and hours of training on the national telephone guidelines as independent variables, with the GP cooperatives as a random factor. A significance level of p<0.05 was adopted.

Results

The mystery patients made a total of 370 telephone contacts, with 352 of the contacts proving usable in the end. A total of 18 contacts were dropped mostly due to no urgency estimate (n = 5) or premature termination due to recognition by the mystery patient (n = 13).

For 242 (69%) of the 352 telephone contacts, the urgency estimation was in perfect concordance with the gold standard. For the 110 non‐concordance contacts, the urgency estimate of 106 (30%) contacts differed not more than 1 point, and for 4 (1%) of the contacts, the urgency estimates differed ⩾2 points from the gold standard.

For 44 (12.5%) of the telephone contacts the triage nurses scored higher urgency, and for 66 (19%) of the contacts the triage nurses scored lower urgency than the gold standard (table 2).

Table 2 Judgements of urgency by triage nurses relative to gold standard.

Criteria assessed by triage nurse	Gold standard U1–U4
	U1	U2	U3	U4	Total
U1	22	2	1	0	25
U2	11	28	11	1	51
U3	1	18	38	29	86
U4	1	0	35	154	190
Total	35	48	85	184	352

Underlined numbers denote correct estimation.

The capacity of the triage nurses to discriminate (potentially) life‐threatening cases from less urgent cases was examined through comparison of the U1 and U2 figures with the U3 and U4 figures. The sensitivity was 0.76 (63/83) and the specificity was 0.95 (256/269). The positive predictive value of the estimates of triage nurses was 0.83 (63/76). This is therefore much higher than the 0.24 ((35+48)/352) a priori probability of a life‐threatening problem. The negative predictive value of the estimates of triage nurses was 0.93 (256/267), while the a priori probability of a non‐life‐threatening problem was 0.76 (269/352).

The triage nurses who were trained on the use of national telephone guidelines had a lower rate of underestimation of the urgency (OR = 0.10, 95% CI 0.01 to 0.81). The educational background (primary or secondary care) of the nurses had no significant relationship with the rate of underestimation.

Discussion

The urgency judgements made by the triage nurses after telephone contact with a mystery patient were found to be in two‐thirds agreement with the gold standard. The sensitivity and positive predictive values were not high, which meant that triage nurses tended to underestimate urgent complaints. Conversely, the specificity and negative predictive values were high, which meant that the triage nurses delivered efficient care.

We found a significant correlation between the accurate estimation of the urgency and specific training on telephone guidelines. This finding suggests that training can help improve the safety of care, but we do not know what is the required intensity or frequency of this training.

The use of mystery patients in practice mimics reality in an outstanding way. However, despite our efforts, one cannot prevent subtle differences in the presentations of the different complaints by the standardised mystery patients.

Additional research is needed to unravel the different determinants of the quality and safety of telephone consultations with triage nurses. A follow‐up study in general practice reviewing safety patient contacts with the GP cooperative is also recommended.¹⁸

In The Netherlands, there is a trend to employ a special telephone doctor to supervise about 4–6 triage nurses.¹⁰,¹¹ Investigation of this role is needed: does it improve the quality and safety of telephone triage?

In the delivery of high‐quality care, safety should always take priority over efficiency because of the potential severe consequences for the patient.¹⁰,¹¹,¹² This research proves that telephone triage by nurses is efficient, but is possibly not safe.

Should we be worried about the outcome of this study? Should we remove the nurses from their triage tasks and should doctors perform telephone triage as is done in Denmark?² Other studies have shown that telephone triage by nurses is safe.⁴,⁵,⁶,⁷,¹⁹ Moreover, there is no research comparing doctors and nurses performing triage. In addition, we cannot conclude that nurse triage is less safe than GP triage, because that requires another study design. Perhaps it does not matter who performs the triage because the telephone as a medium is unsafe.

Indeed we should worry about the safety of telephone triage and take major steps to improve this. GP cooperatives should adopt safety rules such as: “When patients ring for the second time you should arrange a face‐to‐face meeting with the doctor.” They should also encourage the attitude of not being too restrictive in arranging face‐to‐face contact, because the telephone is perhaps an unsafe medium.¹¹,¹²,¹⁷

We recommend an educational certified programme for triage nurses and a direct second safety check of all cases by a specially trained GP, who supervises telephone triage nurses.¹⁰,¹¹ Further, the use of computerised decision support may also be helpful to enhance the safety of telephone triage.⁴,⁷,²⁰,²¹,²² Finally, we recommend analysis of medical (near)calamities in peer group meetings.

Abbreviations

GP - general practitioner