Factors associated with variation in urgency of primary out-of-hours contacts in the Netherlands: a cross-sectional study

Abstract

Objectives

Dutch primary out-of-hours care is provided by general practice cooperatives (GPCs). Although most GPCs use the same standardised triage system, differences between GPCs exist in the urgency assigned to patients’ health problems. This cross-sectional study aims to provide insight into factors associated with the variation in assigned urgency between GPCs.

Design and methods

Data were derived from routine electronic health records of 895 253 patients who attended 17 GPCs in 2012. Patients’ gender, age, travel distance to the GPC, and the use of a computer-based decision support system for triage were investigated as possibly affecting assigned urgency. Multilevel linear regression analyses were executed for the 3 most frequently presented health problems (cystitis/other urinary infection, laceration/cut and fever).

Results

Variation in urgency levels between GPCs was significant for the selected health problems (p=0.00). Assigned urgency was mainly related to patient gender and age. It was not associated with the use of a computer-based decision support system, or with travel distance to the GPC. Most variation in urgency (93.4–96.7%) could be ascribed to variation in patient characteristics.

Conclusions

There is significant variation in urgency levels between GPCs, even for the same health problem. This variation is mainly associated with differences in characteristics of individuals contacting the GPCs, rather than with variables such as patients’ travel distance or the use of a computer-based decision support system. Since patient characteristics are likely to affect patients’ clinical need, our results are an indication of the adequate functioning of the triage system.

Strengths and limitations of this study.

• To our knowledge, we were the first to use a large data set derived from routine electronic health records of nearly 900 000 patients to analyse the variation in urgency between general practice cooperatives (GPCs) and multiple factors associated with assigned urgency.

• Comparison with Dutch population data and data reported by the national association for out-of-hours care underlines the representativeness of our data.

• Our finding that the variation in urgency can mainly be ascribed to variation in patient characteristics provides support for the adequate functioning of the triage system used in almost all Dutch GPCs.

• We studied factors associated with assigned urgency for three selected health problems. Further research is needed to investigate whether our results can be generalised to other health problems and to other countries with comparable healthcare systems.

Introduction

As in several other Western countries, primary out-of-hours care in the Netherlands is provided by large scale general practice cooperatives (GPCs).^{1 2} A GPC consists of one or more locations at which primary out-of-hours care is being provided. GPCs can be contacted by patients living in a specified postal code area surrounding the GPC (catchment area) at hours when the patient's own general practice is closed (ie, at weekdays from 17:00 until 8:00, in weekends and during public holidays).

Patients must seek contact with the GPC by telephone before attending. Trained nurses execute telephone triage and decide what type of consultation the patient requires. These nurses are supervised by a general practitioner (GP), who may be consulted in case of doubt. All calls handled by the triage nurses have to be checked and authorised by a GP,¹ who subsequently records the patient's health problem using codes from a standardised classification system (see Methods). This coding is not necessarily done after a consultation with the GP. The GP can also record a diagnostic code based on the information provided by the triage nurse.

The triage is executed by means of a standardised six-level triage system, the Netherlands Triage System (NTS).³ When vital functions are threatened, urgency level 0 is applicable. If this is not the case, the triage nurse selects the patient's main health problem out of a list of 48 problems and indicates its main discriminators (triage criteria). Based on these data, one of the remaining five urgency levels is recommended by the system, ranging from urgency levels 1 (life-threatening) to 5 (self-care advice) (table 1). As can be seen in table 1, a higher NTS urgency level represents a clinically less urgent health problem. To clarify this difference, the term ‘urgency’ will be used throughout this paper, to indicate the clinical urgency instead of the NTS urgency level.

Table 1.

Urgency levels of the Netherlands Triage System

Urgency level	Classification of health problem and recommended care
U0	Failure of vital functions (airway, breathing, circulation, disability), resuscitation
U1	Life-threatening, immediate care
U2	Acute, evaluation within 1 h
U3	Urgent, evaluation within a few hours
U4	Non-urgent, no time pressure, evaluation at the same day and/or within the same working shift
U5	Self-care advice, evaluation can be postponed to regular primary care

In 2012, there were 54 GPCs in the Netherlands.⁴ Almost all cooperatives (96%) used the NTS or a triage system that is comparable in content.⁴ The NTS is also available as paper guideline, but most GPCs (62%) use a computer-based decision support system to assist triage nurses in using the NTS. The use of a standardised triage system is expected to lead to more uniformity in the assignment of urgency to patients’ health problems. Still, differences between GPCs exist in the distribution of assigned urgency of primary out-of-hours contacts.⁵ Assigned urgency may be affected by factors other than the triage system.⁵

First, differences in characteristics of the population contacting the GPCs may lead to differences in assigned urgency between GPCs. Previous research has shown that the distribution of urgency is associated with patient gender and age. Women have been shown to contact the GPC more frequently than men, except for life-threatening health problems.⁶ This may imply that assigned urgency is generally lower for women than for men.⁵ Contacts for life-threatening health problems have been shown to increase with patient age, whereas non-urgent contacts most frequently occur for young children.^{5 6}

Second, the distance between patients’ homes and the GPC may affect the variation in urgency between GPCs. Patients living further away may experience barriers to consult the GPC, which may cause them to consult the GPC only for more urgent health problems. A previous study showed that an increasing travel distance between the patient's home and the out-of-hours service was associated with lower utilisation of out-of-hours care,⁷ a phenomenon known as distance decay.⁸ This was particularly the case for non-urgent health problems.⁷ Thus, GPCs in densely populated areas, with relatively short distances between patients’ homes and the GPC, may have relatively more contacts with low urgency. The effect of distance is likely to be most pronounced for face-to-face consultations, and is less likely to occur for telephone consultations. ^{8 9}

Third, the use of a computer-based decision support system may affect the assignment of urgency. A common problem with traditional paper-based triage is the reliance on memory, which is affected by experience and may be negatively affected by lack of time or recall.¹⁰ Computer-based decision support tools, which guide the triage nurse through each step of the triage process, may improve the reliability of the triage and thereby increase its uniformity. Indeed, a study which compared computer-supported triage with standard triage at the emergency department(ED) showed that variation in assigned urgency between triage nurses was higher when using standard triage.¹⁰

Differences between GPCs with regard to the factors aforementioned may lead to variation between cooperatives in the urgency assigned to patients’ health problems. This study aims to provide insight into factors associated with the variation in assigned urgency between GPCs. Is assigned urgency associated with relevant patient characteristics such as patient age and gender? Or is it associated with variables which are clinically less relevant, such as patients’ travel distance to the GPC or the use of a computer-based decision support system for triage? In the latter case, the variation in assigned urgency may be regarded as undesirable: it may hamper adequate communication and collaboration between healthcare providers, and may thereby negatively affect the quality and safety of care. This may require actions to improve the uniformity of the urgency assignment.

Methods

Study population

Data were derived from routine electronic health records of patients attending GPCs participating in the NIVEL Primary Care Database in 2012.¹¹ This database includes longitudinal data on morbidity and treatment of 28 Dutch GPCs. For this study, only data of GPCs with sufficient data quality regarding health problems were used (see below). All patient contacts of these GPCs (telephone consultations, consultations at the out-of-hours service and home visits) in 2012 were used in this study. The data were anonymised by a trusted third party to ensure patients’ privacy.¹²

Electronic health records

Health problems of patients who consulted their GPC were recorded using codes from the International Classification of Primary Care, V.1 (ICPC).¹³ This version of the ICPC is the standard for coding and classification of complaints, symptoms and disorders in Dutch general practice. ICPC-1 forms an integral part of the training of Dutch GPs and is included in all electronic health records in general practice. A GPC was selected for this study if a meaningful ICPC code was recorded in at least 70% of its contacts. ICPC codes considered to be meaningful range from 01 to 29 (symptoms) and from 70 to 99 (diagnoses). Since ICPC codes A97 (no disease) and A99 (other generalised disease/multiple syndromes) are sometimes used when healthcare providers do not directly know how to classify the patient's health problem, we do not regard these ICPC codes as meaningful. The same holds for codes in the range 30–69 (procedures).

When more than one ICPC code had been recorded during a contact (in 0.02% of all contacts, N=289 contacts), we included in our analyses only the one that was recorded first, assuming that this was the patient's most important health problem.

Apart from ICPC codes, the extracts of electronic health records used for this study included patients’ year of birth, gender and the first four digits of their six-digit postal code, as well as the postal code of the consulted GPC location. Distance in kilometres between the patient's postal code and the postal code of the GPC location was calculated using the Drive Time Matrix of the Netherlands 2012 (Geodan IT).

Questionnaire

Managers of the participating GPCs were asked to indicate whether or not a computer-based decision support system was being used for triage in their GPC. This question was part of a more extensive questionnaire. All managers of the included GPCs completed the questionnaire.

Analyses

Since we expected the distribution of urgency levels to be comparable for the same health problem, we chose to perform our analyses for three specific health problems, namely the three health problems most frequently presented at the out-of-hours service: cystitis/other urinary infection (ICPC code U71), laceration/cut (S18) and fever (A03).

First, we investigated the distribution of urgency levels (ie, the percentage of contacts associated with each of the six urgency levels) for the total number of contacts, and for contacts for the three selected health problems. We also performed these analyses for each GPC separately to obtain insight into the range of percentages for each urgency level.

Subsequently, multilevel linear regression analyses with two-level hierarchically structured data (patients within GPCs) were used to investigate whether the urgency of contacts was associated with patients’ gender, age, the distance between patient's home and the GPC, and the use of a computer-based decision support system by the GPC. Cases with missing data on any of the study variables were deleted from the analyses. However, as shown in table 2, there were few missing data. The null model was used to test whether the distribution of urgency was significantly different between the various GPCs.

Table 2.

Characteristics of participating GPCs and their patients

GPCs (N=17)
Number of inhabitants in catchment area, per GPC (M, range)	364 548	106 270–1 452 738
Use of computer-based decision support system for triage (%, N)
Yes	82.4	14
No	17.6	3
Patients (N=895 253)
Gender (%, N)
Male	45.1	403 381
Female	54.9	491 793
Unknown	0.0	79
Age (%, N) (years)
0–4	14.8	132 425
5–17	14.2	126 919
18–44	33.0	295 726
45–64	19.5	174 240
65–74	7.5	67 291
75–84	6.8	61 064
≥85	4.2	37 583
Unknown	0.0	5
Distance between patient's home and GPC in kilometres (M, SD)	11.7	21.9*

*Data available for 854 119 patients (95.4%).

GPC, general practice cooperation; M, mean.

Because of the ordinal nature of our dependent variable (urgency of primary out-of-hours contacts), multilevel multinomial analyses would ideally have to be executed. However, we chose to perform multilevel linear regression analyses, because this method generates more easily interpretable data. Apart from unstandardised regression coefficients and 99.99% CIs, we calculated intraclass correlation coefficients (ICCs), which indicate the proportion of the variation in urgency that can be ascribed to the patient level versus the level of the GPC. We have added diagnostic plots (q-q plots), which show the distribution of residuals versus the normal distribution, in online supplementary material file. All analyses were executed in Stata, V.13. We applied the Bonferroni method to correct for multiple comparisons, resulting in a p value <0.002 being regarded as significant.

Results

Seventeen GPCs met our criteria for sufficient data quality regarding health problems. The population in the catchment area of these GPCs (N=6 144 649) is representative of the Dutch population with regard to gender and age. A total number of 895 253 patients contacted one of the included GPCs, resulting in 1 350 964 contacts. These contacts included telephone consultations (N=570 915), consultations at the out-of-hours service (N=648 150) and home visits (N=131 899). Characteristics of the participating GPCs and their patients are presented in table 2.

Table 3 displays the distribution of urgency levels for the total number of contacts, and for the three health problems most frequently presented at out-of-hours services. The ranges of urgency levels (table 3) show that variation in urgency between GPCs particularly occurs at urgency levels 4 and 5. For each of the three selected health problems, the distribution of urgency levels was significantly different between the various GPCs (p=0.00).

Table 3.

Urgency of patients’ health problems in contacts with primary out-of-hours services, in percentages

	Contacts (N=1 350 964)			Contacts for ICPC code U71 (N=52 207)			Contacts for ICPC code S18 (N=44 791)			Contacts for ICPC code A03 (N=43 201)
U	Per cent	Minimum*	Maximum*	Per cent	Minimum*	Maximum*	Per cent	Minimum*	Maximum*	Per cent	Minimum*	Maximum*
0	0.01	0.00	0.15	0.00	0.00	0.03	0.00	0.00	0.03	0.00	0.00	0.10
1	1.06	0.42	1.90	0.03	0.02	0.12	0.04	0.01	0.17	0.30	0.10	1.52
2	7.99	4.63	12.24	3.63	1.76	8.72	4.43	0.74	14.76	7.32	3.90	11.83
3	37.13	21.58	45.69	27.53	7.34	47.24	60.09	30.72	77.05	34.22	26.37	57.65
4	26.11	16.78	51.46	42.27	24.01	65.25	24.02	10.77	58.45	16.86	6.31	42.09
5	27.69	7.03	46.83	26.53	2.84	51.97	11.43	1.75	32.79	41.29	13.04	57.28

*Range of percentages between GPCs.

A03, fever; GPC, general practice cooperative; ICPC, International Classification of Primary Care; S18, laceration/cut; U, the Netherlands Triage System urgency level; U71, cystitis/other urinary infection.

The ICCs resulting from our multilevel linear regression analyses showed that the main part of the total variation in urgency can be ascribed to variation in characteristics of patients. For cystitis/other urinary infection, 93.4% of the variation in urgency could be ascribed to variation in patient characteristics, and 6.6% to the level of the GPC. Comparable results were found for laceration/cut (95.1% patient level, 4.9% GPC level) and fever (96.7% patient level, 3.3% GPC level).

Results of the multilevel linear regression analyses are presented in table 4. When interpreting these results, one should keep in mind that a higher NTS urgency level represents a less urgent health problem (see table 1). Thus, a positive association between an independent variable and urgency level implies that this variable is associated with a lower clinical urgency.

Table 4.

Factors associated with urgency: results of multilevel linear regression analyses

	ICPC code U71			ICPC code S18			ICPC code A03
	B	99.99% CI	p Value	B	99.99% CI	p Value	B	99.99% CI	p Value
Female	0.48*	0.44 to 0.52	0.000	0.06*	0.03 to 0.08	0.000	0.03	−0.00 to 0.07	0.001
Age (years)
5–17	0.03	−0.07 to 0.13	0.238	−0.05*	−0.09 to −0.00	0.000	−0.04	−0.10 to 0.03	0.024
18–44	0.25*	0.17 to 0.34	0.000	−0.04*	−0.08 to 0.00	0.000	−0.12*	−0.19 to −0.05	0.000
45–64	0.26*	0.18 to 0.35	0.000	−0.10*	−0.14 to −0.05	0.000	−0.40*	−0.49 to −0.32	0.000
65–74	0.20*	0.11 to 0.29	0.000	−0.14*	−0.21 to −0.07	0.000	−0.62*	−0.73 to −0.51	0.000
75–84	0.15*	0.06 to 0.25	0.000	−0.20*	−0.27 to −0.12	0.000	−0.75*	−0.85 to −0.64	0.000
≥85	0.21*	0.12 to 0.31	0.000	−0.25*	−0.33 to −0.17	0.000	−0.69*	−0.82 to −0.56	0.000
Distance in kilometres†	0.00	−0.00 to 0.00	0.002	−0.00	−0.00 to 0.00	0.307	0.00	−0.00 to 0.00	0.016
Use of decision support system	0.04	−0.47 to 0.55	0.749	−0.13	−0.54 to 0.28	0.227	0.09	−0.37 to 0.54	0.452

Reference categories: male gender, 0–4 years, no decision support system. A higher NTS urgency level represents a clinically less urgent health problem (see table 1).

*Significant at p<0.002.

†Distance between patient's home and the GPC.

A03, fever; B, unstandardised regression coefficient; GPC, general practice cooperative; ICPC, International Classification of Primary Care; NTS, the Netherlands Triage System; S18, laceration/cut; U71, cystitis/other urinary infection.

The urgency of all three selected health problems was mainly related to patient gender and age. For cystitis and laceration/cut, urgency was significantly lower for female patients than for males. Urgency for cystitis was significantly lower for adult patients (>18 years old) than for young children, whereas urgency for fever was significantly higher for adult patients than for young children. Urgency for lacerations and cuts was significantly higher for patients >5 years old than for young children. The urgency of contacts was not associated with the use of a computer-based decision support system for triage, or with the distance between the patient's home and the out-of-hours service.

Discussion

Our results show significant variation in assigned urgency between GPCs, even for the same health problem. At first glance, this may be regarded as an undesirable finding, since we would expect equal urgency to be assigned to similar cases. However, most of the total variation in assigned urgency (93.4–96.7%) could be ascribed to variation in individual patient characteristics. Apparently, the variation in urgency can mainly be ascribed to variations in the population contacting the GPCs. A relatively small part of the variation in assigned urgency (3.3–6.6%) could be ascribed to variables at the level of the GPC. However, the variable that we included at this level (ie, the use of a computer-based decision support tool for triage) had no significant association with assigned urgency.

Patient age was found to be an important factor associated with assigned urgency. Previous studies also showed urgency to be generally higher for older patients, whereas non-urgent contacts most frequently occur for young children.^{5 6} GPCs with many elderly patients may therefore have more highly urgent contacts than GPCs which operate in a younger population.

Previous research has shown women to contact the GPC more frequently than men, except for life-threatening health problems.⁶ Assigned urgency may therefore generally be lower for women than for men.⁵ Our results confirmed this hypothesis for cystitis and lacerations/cuts. Since cystitis predominantly occurs in women,¹⁴ one can expect contacts for cystitis in men to be regarded as more urgent than contacts for cystitis in women. This association is less clear for lacerations and cuts.

In contrast with previous research by Raknes et al,⁷ we did not find a significant association between patients’ travel distance to the out-of-hours service and the assigned urgency. However, the previous study was conducted in Norway, where the distance between patients’ homes and the out-of-hours service is much larger than in the Netherlands. Giesen et al¹⁵ found that Dutch patients have to travel a maximum of 30 km to attend the out-of-hours service, whereas the travel distance for Norwegian patients can be more than 130 km. Apparently, the variation in assigned urgency between GPCs is not associated with variations in travel distances between GPCs. However, from the patient's perspective, the time needed to travel to the GPC may be more important than the travel distance per se. It would be interesting to investigate whether there is an association between patients’ travel time and the assigned urgency.

To our knowledge, we were the first to use a large data set derived from routine electronic health records of nearly 900 000 patients to analyse the variation in urgency between GPCs and factors associated with assigned urgency. The population in the catchment area of the included GPCs was representative of the Dutch population with regard to gender and age. GPCs that used a computer-based decision support system for triage were over-represented in our sample (82% in our sample vs 62% in the Netherlands). The overall distribution of urgency levels found in this study (table 3) is similar to the distribution reported by the national association for out-of-hours care,⁴ which underlines the representativeness of our data.

Although we tried to discover some of the key variables associated with the variation in urgency between GPCs, other variables deserve attention in future research. Patient characteristics like socioeconomic status, comorbidity, the use of medication, living conditions, strategy to cope with physical symptoms and perceptions about the ease to visit a GPC may also be associated with assigned urgency.

A factor of interest on the GPC level may be the collaboration between the GPC and the ED of the hospital. Some Dutch GPCs are part of so-called integrated EDs, which combine their entrance and triage with the ED.¹⁶ The joint triage is expected to cause patients with less urgent health problems to be diverted to the GPC, and patients with highly urgent health problems to be diverted mainly to the ED. GPCs participating in an integrated ED may therefore have to deal with more low urgent health problems than GPCs not participating in such a department.¹⁷

Another variable that may affect the variation in assigned urgency between GPCs is the accessibility of general practices during office hours. A recent study showed that contact rates at the GPC were higher when the associated general practices closed early or were otherwise less accessible.¹⁸ Limited accessibility of patients’ general practice may particularly lead to an increase of low urgent health problems presented at the GPC.

Still, since these variables are characteristics of GPCs and our study indicated that only a small part of the variation in urgency for the three selected health problems could be ascribed to variation at the level of the GPC, we expect such variables to only marginally affect the variation in assigned urgency.

For our analyses, we could only use the ICPC codes that were recorded in the GPCs’ electronic health records. We could not test the validity of these ICPC codes with respect to patients’ health problems. Therefore, we cannot completely rule out the possibility that the variation in assigned urgency in fact reflects different ICPC coding practices between GPCs. However, our multilevel analyses showed that only a small part of the variation in assigned urgency can be ascribed to the level of the GPCs, which argues against this possibility.

We studied factors associated with assigned urgency for the three health problems most frequently presented at the out-of-hours service. This strategy was chosen because it would guarantee a sufficient number of contacts for our analyses. However, our choice to focus on these specific health problems also has some limitations. First, two of the selected ICPC codes (fever, A03; and laceration/cut, S18) are symptoms, which are less specific than diagnoses. Fever, for instance, can be a sign of a variety of diagnoses (eg, pneumonia, tuberculosis, acute myocardial infarction, a malignancy, arthritis), most likely to be associated with different urgency levels. Information about such diagnoses is not included in our study. Second, selecting three specific health problems obviously does not provide the full picture of the variety in urgency for health problems presented in primary out-of-hours care. Further research is needed to investigate whether our results can be generalised to other health problems and to other countries with comparable healthcare systems. It would also be interesting to investigate the variation in assigned urgency in association with the reason for encounter recorded by the triage nurse.

In sum, we showed that the variation in assigned urgency is not associated with one aspect of accessibility (ie, travel distance), or with one of the core facilities of GPCs (ie, the use of a computer-based decision support system for triage), but rather with differences in characteristics of individuals contacting the GPCs. Since patient characteristics are likely to affect patients’ clinical need, our results are an indication of the adequate functioning of the triage system. Although this is a promising result, additional research is needed to shed more light on the sensitivity and specificity of the triage system.