Skip to main content
Family Practice logoLink to Family Practice
. 2022 Jul 18;40(1):23–29. doi: 10.1093/fampra/cmac077

Evaluation of telephone triage among chest pain patients in out-of-hours primary care in the Netherlands (TRACE)

Amy Manten 1,, Remco P Rietveld 2, Lukas de Clercq 3, Inge van Hulst 4, Wim A M Lucassen 5, Eric P Moll van Charante 6,7, Ralf E Harskamp 8
PMCID: PMC9909662  PMID: 35849343

Abstract

Background

Telephone triage is fully integrated in Dutch out-of-hours primary care (OOH-PC). Patients presenting with chest pain are initially assessed according to a standardized protocol (“Netherlands Triage Standard” [NTS]). Nevertheless, little is known about its (diagnostic) performance, nor on the impact of subsequent clinical judgements made by triage assistants and general practitioners (GPs).

Objective

To evaluate the performance of the current NTS chest pain protocol.

Methods

Observational, retrospective cohort study of adult patients with chest pain who contacted a regional OOH-PC facility in the Netherlands, in 2017. The clinical outcome measure involved the occurrence of a “major event,” which is a composite of all-cause mortality and urgent cardiovascular and noncardiovascular conditions, occurring ≤6 weeks of initial contact. We assessed the performance using diagnostic and discriminatory properties.

Results

In total, 1,803 patients were included, median age was 54.0 and 57.5% were female. Major events occurred in 16.2% of patients with complete follow-up, including 99 (6.7%) cases of acute coronary syndrome and 22 (1.5%) fatal events. NTS urgency assessment showed moderate discriminatory abilities for predicting major events (c-statistic 0.66). Overall, NTS performance showed a sensitivity and specificity of 83.0% and 42.4% with a 17.0% underestimated major event rate. Triage assistants’ revisions hardly improved urgency allocation. Further consideration of the clinical course following OOH-PC contact did generate a more pronounced improvement with a sensitivity of 89.4% and specificity of 61.9%.

Conclusion

Performance of telephone triage of chest pain appears moderate at best, with acceptable safety yet limited efficiency, even after including further work-up by GPs.

Keywords: acute coronary syndrome, chest pain, major event, primary care, triage


Key messages.

  • One in every 6 patients with chest pain suffered from a major event.

  • Performance of telephone triage of chest pain appears moderate at best.

  • Triage safety is acceptable, yet efficiency is limited, even after including GP work-up.

  • Triage protocols should be revised using algorithms trained on clinical outcomes data.

Background

Chest pain is a common symptom among patients reaching out to primary care. It accounts for approximately 0.7%–2.7% of daytime general practice consultations and is consistently present in the top 10 of main complaints in out-of-hours primary care (OOH-PC) settings.1–6 When it comes to chest pain, acute coronary syndrome (ACS) is usually the first diagnosis that comes to mind. Understandably so, since missing an ACS may have serious consequences for the patient.7 However, the spectrum of possible underlying conditions reaches far beyond ACS and includes several serious diagnoses that should not be overlooked (e.g. pulmonary embolism, aortic dissection, and sepsis).1,5,6,8 Timely detection of these cases is vital in order to start adequate treatment to prevent complications, and even death.7

In several countries with general practitioner (GP) involvement in urgent care, telephone triage protocols are in place to aid adequate assessment of patients.9 In the Netherlands, triage is performed using standardized, symptom-based questions, which is integrated in software developed by the “Netherlands Triage Standard” (NTS).10 In essence, the NTS protocols were developed by an expert panel using a modified version of the Manchester Triage System,11 in combination with existing telephone triage protocols used in daytime primary care.12 However, the specific protocol for assessing chest pain was not validated before its instalment and a survey among Dutch GPs showed that 83.9% believed that the current system performs too defensively, contributing to overcrowding of out-of-hours care services.13 Furthermore, a nation-wide increase in the number of ambulance deployments was seen, partly induced by increased usage of ambulance services through OOH-PC.14

Despite its common use and clinical relevance, research efforts on evaluating the performance of triage systems for chest pain are limited, and the influence of OOH-PC personnel (i.e. triage assistants, attending GP) on triage decision-making is often neglected. Thus far, 1 prior study evaluated the NTS for chest pain and found that urgencies were underestimated in a quarter of cases.15 We aim to build upon this knowledge through an elaborate assessment of the several layers of decision-making in a large stand-alone OOH-PC facility.

Methods

Study design

Our study “TRiage of Acute Chest pain Evaluation in primary care (TRACE)” is reported according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement.16 A detailed description of study design and methods was published previously.17 In short, we performed an observational, retrospective cohort study of consecutive, adult patients with chest pain who contacted the OOH-PC facility in Alkmaar, the Netherlands, between 1 January and 31 December 2017. At the outset of the TRACE study, eligible patients received information by mail and were provided the opportunity to opt-out of sharing data.17,18 Patients who bypassed OOH-PC, and immediately contacted the national emergency number, were not included.

Background and setting

In the Netherlands, the main organizational structure for OOH-PC consists of regional GP cooperatives that provide care to patients of affiliated GPs. The designated out-of-hours facility in Alkmaar provides care to 240,000 individuals, living in both rural and urban areas.

The flow of care in Dutch OOH-PC consists of multiple interacting layers, illustrated in Fig. 1. In short, when patients contact the facility, triage assistants use the NTS decision support tool to triage a patient’s urgency. Each complaint-specific NTS protocol starts by ensuring patient’s vital signs using the ABCDE method.19 In patients assessed as “unsafe” (i.e. hemodynamically threatened/unstable), ambulance services are immediately activated. In all others, the NTS protocol for chest pain continues with 7 standardized questions regarding the symptom characteristics (type, duration, severity, course, location, radiation, and associated symptoms). The supporting NTS algorithm uses the available answers (minimum of 1) to continuously calculate a recommended urgency level (NTS urgency). Urgencies vary from U1 (life threatening) to U5 (no harm) and are linked, but not restricted to a subsequent recommendation of time-until-care. Triage assistants who operate the NTS may have considerable influence on the system in multiple ways: (i) triage assistants conduct the conversation and may gather additional information not covered by the protocol (e.g. paralanguage, “gut-feeling”), (ii) in case of disagreement with the NTS tool, they may overrule the recommended urgency level, and (iii) they ultimately decide on the most fitting course of action (e.g. ambulance, GP consultation, and self-care advice).

Fig. 1.

Fig. 1.

Chest pain evaluation in Dutch OOH-PC. The figure illustrates the flow of care for chest pain patients in OOH-PC. Our study population corresponds to the second tier in the figure “patients who contact OOH-PC.” Patients from the top tier bypassed the triage system and were not included in our study. The first step of triage is to identify, and filter out, patients that are hemodynamically unstable using the ABCDE method.19 Further triage of the remaining, hemodynamically stable patients (third tier), follows the NTS protocol for chest pain evaluation, composed of 7 standardized questions. The supporting algorithm uses the answers (minimum of 1) to continuously calculate an urgency level (NTS urgency). These urgency codes are linked to a recommended time-until-care (U1—as soon as possible, U2—care <60 min, U3—care within several hours, U4—care within 24 hours, and U5—next workday). Triage assistants can accept the recommended urgency or revise it and ultimately decide on the most appropriate course of action (i.e. the course of action is often associated with, but not restricted to a certain urgency code). Patients who are further assessed face-to-face by a GP are represented in the bottom tier. Notes: *Contact occurs through telephone contact, and occasionally, in person (“self-referrals”).

The attending GP forms another layer of the system and holds final responsibility. In this role, GPs may be consulted by the triage assistant, but also conduct face-to-face patient consultations at the OOH-PC facility or at the patient’s home.

Data collection

Data collection consisted of 2 phases. In the first phase, digital triage registries were accessed from the OOH-PC facility and entered into a secure electronic data-capturing platform (Castor EDC, Ciwit BV, Amsterdam, The Netherlands)20 after deidentification. These registries included information on triage assessment: symptom characteristics, urgency levels, course of action, and GP assessment (in case of GP involvement). In the second phase, researchers visited daytime GP’s offices to collect available information regarding patient characteristics (e.g. medical history and medication use) and clinical outcomes. Interobserver variability between researchers was restricted by the use of Standard Operating Procedures (SOPs) on data registering and by an internal audit.

Definition of clinical outcomes

The main clinical outcome of interest was the occurrence of a major event within 6 weeks of initial contact. Major events were defined as a composite of all-cause mortality, and both cardiovascular and noncardiovascular urgent conditions that were linked to the initial complaint of chest pain, and required hospital admittance or urgent in-hospital treatment (Supplement 1). ACS was included as a major event and was also assessed separately as a secondary outcome. We deliberately chose to consider ACS as a secondary, not primary outcome, to maintain the broad perspective of unselected chest pain.

The occurrence of clinical outcomes was based on the final diagnosis, registered as International Classification of Primary Care (ICPC) codes in the daytime GP’s patient record file. The validity of these diagnoses was examined using information from the GP’s file (including all relevant consultation notes and correspondence from hospital-based specialists or emergency departments), and cross-validated by an independent panel of expert GPs.

Analysing triage performance

We evaluated the performance of triage according to the separate layers of OOH-PC decision-making. First, triage was evaluated using NTS software derived urgency levels (NTS urgency) regarding the occurrence of a major event (and separately for ACS). Second, these were compared with final urgencies, which are the eventual urgency codes after possible up- and downscaling by the triage assistant. We assessed the discriminatory ability using c-statistics, and assessed the agreement between predicted and observed major events (and ACS) using calibration plots.

Diagnostic test properties (sensitivity, specificity, positive and negative predictive values) were assessed after dichotomizing the urgencies into a high (U1–U2; care <60 min) and low level (U3–U5; care >60 min).

Third, we looked beyond urgency assessment and evaluated the performance of overall OOH-PC decision-making. Here, we used “urgent referral” as a reference standard, indicating whether patients were urgently referred to the hospital/emergency department (i.e. directly following initial triage or following subsequent GP consultation). Thus, forming a composite of the eventual course of action following OOH-PC contact.

Results

A total of 2,043 consecutive patients with chest pain reached out to the OOH-PC facility. Of those, 1,803 patients met our inclusion criteria (Fig. 2) and the vast majority of patients (1,738 [96.4%]) contacted the facility by telephone.

Fig. 2.

Fig. 2.

Flowchart of inclusion and exclusion of patients. A total of 2,043 patients contacted the OOH-PC facility regarding chest pain in 2017. After exclusions, baseline information was completed for 1,803 patients. Upon contact, 23 patients were deemed hemodynamically unstable, resulting in discontinuation of triage and immediate action (i.e. activation of ambulance services). In total, follow-up data could be obtained for 1,471 patients. The allocated urgency levels were available for 1,434 patients. Notes: *Two patients bypassed the triage system entirely and were excluded from the dataset.

Supplement 2 displays the characteristics of the study population. Overall, the median age was 54.0 years [IQR: 37.0–70.0] and 1,038 (57.6%) were female. Most patients had at least 1 preexisting medical condition (69.9%) and/or used chronic medication (59.8%). A history of cardiovascular disease (CVD) was present in 363 (20.1%).

Clinical outcomes

Follow-up data on outcomes could be obtained from 1,471 patients (81.6%). A mentionable number of lost to follow-up was caused by GPs who expressed liability concerns (4 practices representing 108 patients).

In most patients, chest pain was caused by musculoskeletal conditions (674 [45.8%]). A total of 238 (16.2%) patients suffered from a major event within 6 weeks after initial contact, and majority of the major events were caused by CVD (184 [77.3%]), including ACS (n = 99), atrial fibrillation (n = 38), and pulmonary embolisms (n = 10) (Supplement 3). Of the remaining group (n = 54, 22.7%), most could be ascribed to respiratory (n = 23) and abdominal conditions (n = 17). In total, 22 (1.5%) patients died in the first 6 weeks following contact. Patients with a major event were older, more often male, and more often known with a preexisting medical condition, including prior CVD (Table 1).

Table 1.

Patient characteristics of 1,471 patients with chest pain that contacted the out-of-hours primary care facility in 2017.

Total (n = 1,471) Major event (n = 238) Nonmajor (n = 1,233) P
Age 55.0 (38.0–71.0) 71.0 (61.0–80.3) 51.0 (35.0–68.0) <0.001
Sex (female)* 849 (57.7%) 109 (45.8%) 740 (60.0%) <0.001
Preexisting medical condition (any) 1,093 (74.3%) 212 (89.1%) 881 (71.5%) <0.001
History of cardiovascular disease 329 (22.4%) 100 (42.0%) 229 (18.6%) <0.001
 Prior CAD 270 (18.4%) 84 (35.3%) 186 (15.1%) <0.001
 Prior stroke/TIA 102 (6.9%) 28 (11.8%) 74 (6.0%) 0.027
 Prior peripheral arterial disease 38 (2.6%) 14 (5.9%) 24 (1.9%) 0.005
Cardiovascular risk factors (any) 841 (57.2%) 186 (78.2%) 655 (53.1%) <0.001
 Smoking (current) 265 (18.0%) 37 (15.5%) 228 (18.5%) 0.08
 Hypertension 461 (31.3%) 124 (52.1%) 337 (27.3%) <0.001
 Hypercholesterolaemia 243 (16.5%) 70 (29.4%) 173 (14.0%) <0.001
 Diabetes mellitus 181 (12.3%) 55 (23.1%) 126 (10.2%) <0.001
Chronic use of medication (any) 945 (64.2%) 195 (81.9%) 750 (60.8%) <0.001

The table lists the patient characteristics for the group of patients in whom follow-up data could be completed (n = 1,471) and a division based on the occurrence of a major event. Continuous variables are presented as medians (IQR) due to a non-normal distribution. All categorical variables are presented as frequencies (%). Abbreviation: TIA, transient ischemic attack.

The patient sample included 1 transgender patient who transitioned from male to female gender, the sex of this patient was referred to as ‘other’.

Prior coronary artery disease (CAD) was defined as a history of either acute or stable coronary syndromes.

Of the 99 (6.7%) patients that suffered an ACS, 24 (24.2%) were diagnosed with ST-elevation myocardial infarction, 44 (44.4%) with non-ST-elevation myocardial infarction, 19 (19.2%) with unstable angina pectoris, and 12 (12.1%) cases were not further specified. Differences in patient characteristics between patients with and without an ACS were similar to those with or without major events.

Performance

1. NTS urgency assessment. Urgency codes were registered and available for 1,434 of the 1,471 patients with completed follow-up. c-Statistics were 0.66 [95% CI: 0.62–0.69] for predicting major events and 0.69 [95% CI: 0.65–0.72] for predicting an ACS. Calibration was adequate when comparing observed versus predicted major events or ACS. Next, we assessed the diagnostic properties of the NTS tool derived urgencies by dichotomizing them into a high (i.e. either U1 or U2; corresponding to a recommended time-until-care <1 h) and low level (i.e. U3–U5; care >1 h), resulting in 886 (61.8%) patients for whom the NTS tool yielded a high urgency. The corresponding diagnostic properties are listed in Table 2. The NTS adequately classified the urgency in 83.0% of patients with a major event and 85.7% of patients with an ACS (sensitivity), underestimating 40 (17.0%) major events and 14 (14.3%) ACS. The specificity was 42.4% for predicting a major event and 40.0% for predicting ACS, with 691 and 802 unnecessary referrals (false-positive rate), respectively.

Table 2.

Performance of the Dutch triage system on major events and acute coronary syndrome among 1,434 patients (2017).

TP FP FN TN Sensitivity (%) Specificity (%) PPV (%) NPV (%) Accuracy (%)
Major event
 NTS urgency 195 691 40 508 83.0
[77.6–87.6]
42.4
[39.6–45.2]
22.0
[20.7–23.3]
92.7
[90.5–94.4]
49.0
[46.4–51.7]
 Final urgency 199 693 36 506 84.7
[79.4–89.0]
42.2
[39.4–45.1]
22.3
[21.1–23.6]
93.4
[91.2–95.0]
49.2
[46.5–51.8]
 Urgent referrals 210 457 25 742 89.4
[84.7–93.0]
61.9
[59.1–64.6]
31.5
[29.7–33.3]
96.7
[95.3–97.7]
66.4
[63.9–68.8]
ACS
 NTS urgency 84 802 14 534 85.7
[77.2–92.0]
40.0
[37.3–42.7]
9.5
[8.7–10.3]
97.5
[95.9–98.4]
43.1
[40.5–45.7]
 Final urgency 85 807 13 529 86.7
[78.4–92.7]
39.6
[37.0–42.3]
9.5
[8.8–10.3]
97.6
[96.1–98.6]
42.8
[40.2–45.4]
 Urgent referrals 93 574 5 762 94.9
[88.5–98.3]
57.0
[54.3–59.7]
13.9
[13.0–14.9]
99.4
[98.5–99.7]
59.6
[57.0–62.2]

The table illustrates the performance of the triage system in out-of-hours primary care in the Netherlands, regarding the occurrence of a major event and acute coronary syndrome. Diagnostic properties are presented as percentages [95%-CI]. “NTS urgency” reflects the initial urgency recommendation provided by the NTS algorithm, while “Final urgency” represents the concluding urgency levels after triage assistants’ revisions. “Urgent referrals” represent the course of action after contact with the OOH-PC facility. Abbreviations: ACS, acute coronary syndrome; FN, false negative; FP, false positive; NPV, negative predictive value; PPV, positive predictive value; TN, true negative; TP, true positive.

2. Final urgency assessment. Triage assistants revised the recommended urgency levels in 271 (18.9%) cases, downscaling 160 (11.2%) and upscaling 86 (6.0%) urgencies. c-Statistics of the resulting final urgencies were 0.69 [95% CI: 0.65–0.72] and 0.71 [95% CI: 0.66–0.76] for predicting major events and ACS, respectively, with adequate calibration. Dichotomizing the final urgencies into a high (U1/U2) and low level (U3–U5) showed that triage assistants revisions of the urgency codes changed the composition of these groups only slightly (44 (3.1%) patients shifted from high to low and 50 (3.5%) patients shifted from low to high). This resulted in a high final urgency level for 892 (62.2%) patients. The diagnostic properties listed in Table 2 further illustrate that the urgency revisions by the triage assistants marginally improved triage safety with only 4 (major events) and 1 (ACS) fewer underestimated cases.

3. Course of action. In 588 (32.6%) of the 1,803 patients that sought contact with the OOH-PC facility, triage assistants decided on activation of ambulance services directly following triage. This group included the 23 (1.3%) patients that were deemed hemodynamically unstable. When we combine urgency assessment with the course of action, we saw that 52.5% of the 892 patients with a high final urgency level received immediate ambulance activation (93.1% of the patients with an U1 [431/436] and 8.6% of U2 [37/429]). We further extended the evaluation of the course of action by assessing whether patients were urgently referred. Thus, either directly following triage or after subsequent GP consultation. Eventually, 89.4% of the major events and 94.9% of the patients with an ACS were urgently referred, while 25 and 5 patients with a major event or ACS were not (Table 2).

Discussion

In this study, we assessed the performance of chest pain evaluation in a Dutch OOH-PC facility by assessing the current triage protocol (NTS), and the influence of OOH-PC personnel on its performance. The NTS tool itself showed moderate discriminatory and diagnostic properties for predicting both major events and ACS. Using it as an independent tool would lead to inappropriate urgency levels in a substantial proportion of patients, causing both large numbers of unnecessary referrals and underestimated urgencies. Triage assistants’ revisions hardly improved urgency allocation. Final decision-making on the course of action (urgent referral or not) was more substantially improved by the combination of triage assistants and GPs, although the resulting performance remained moderate at best.

Strengths and limitations

Our study provides a representative reflection of OOH-PC in the Netherlands and may also be applicable in many other developed countries with a similar health care system. For the TRACE project, we enrolled consecutive patients who presented to the facility during the course of a full year, decreasing the chance of selection and seasonal bias. Furthermore, by using GPs’ patient record files for the collection of follow-up data, we were able to examine patients’ medical status and their clinical course following contact more thoroughly. Finally, we defined major events as our main clinical outcome to better match the variety of urgent underlying conditions that may cause chest pain, including but not restricted to ACS as one of the possible causes.

The most important limitation of our study is due to its retrospective design; observing the clinical course of patients with chest pain in which only those referred to hospital care underwent further diagnostic work-up (differential verification bias). However, we applied a follow-up period of 6 weeks to detect initially not-referred major events, which appeared to be sufficient. A mentionable amount of lost to follow-up was caused by GPs who expressed liability concerns due to the recent implementation of the European privacy regulations (GDPR).

How our findings fit with existing literature

The organizational structure of OOH-PC differs between and within countries. Nevertheless, in most European countries implementation of telephone triage has received considerable attention over the past decade.9 A systematic review concluded that the use of telephone triage is safe in 97% when assessing a broad spectrum of patients and 89% among a high-risk population.21,22

The NTS’s complaint-specific protocols were not separately validated prior to their introduction in OOH-PC and research focussed on triaging chest pain is scarce.

Currently, TRACE is one of 2 ongoing studies evaluating triage of chest pain in Dutch OOH-PC. This provides a unique opportunity to compare results and explore possible regional differences. Wouters et al.15 assessed the accuracy of NTS urgency allocation among patients with chest pain and found that 59.7% of their patient group was assigned with a high urgency level (U1–U2), compared with 61.8% in our sample. Despite slight differences in the definition of clinical endpoints (ACS and other life-threatening events), performance of the NTS algorithm was similar, with a sensitivity and specificity of 73% and 43%, respectively (TRACE: 83.0% and 42.4%). Differences stand out as well, with findings by Wouters et al. showing a higher number of underestimated cases (27% vs. 17.0% in our study) and greater improvement of triage safety after including triage assistants’ own revisions. As described in our results, a more pronounced improvement of safety would occur when we look beyond urgency assessment and consider the course of action following triage or GP consultation. Here, triage assistants and/or GPs may compensate inappropriate urgencies through the management action that follows (e.g. ambulance deployment or urgent referral despite a low urgency).

Giesen et al.23 performed a more general evaluation of triage assistants’ accuracy in 2006. Results of this study showed that triage assistants underestimated urgencies in 19% of 352 contacts and a significant correlation was found between specific training on the use of the NTS guidelines and correct urgency estimation.

This implies that the way triage assistants operate the NTS triage tool may contribute to its performance as well. A second study by Wouters et al.24 qualitatively explored the behaviour of triage assistants in their urgency assessment. They found that triage assistants use symptom characteristics and paralanguage (vocal but not worded elements) to create a mental image of the clinical status of the patient. When their assessment was inconsistent with the NTS recommendation, triage assistants used several strategies to make their final decision. One of those strategies was to overrule the given urgency. Another coping strategy was to tinker the NTS responses to make the final recommendation align with the assistant’s own assessment (e.g. up- or downgrading pain severity or exploring additional symptoms that might alter the calculated urgency). This may affect the way we ought to interpret our data, since we relied on a written depiction of the triage process, impeding our insight of the dynamics.

While chest pain can be caused by a variety of conditions, including several urgent diagnoses, the structure of the current NTS protocol appears to be quite ACS focussed. This is also reflected by available research, in which data focussed on the ACS prevalence in prehospital settings is more abundant. Here, the prevalence varies from 1.5% to 3.6% in primary care settings,1,8,25 rising to 9.4% among patients visiting the emergency department.26

An online survey among Dutch GPs conducted by our research group in 2018 investigated the missed ACS diagnosis rate GPs are willing to accept.27 Results showed that GPs wish to keep a maximum referral threshold of 50 unnecessary referrals for each ACS diagnosis, and that GPs would accept a 0.1%–1.0% missed diagnosis rate in clinical practice. Extrapolating this to our own results, we see that for each diagnosed ACS, 8 patients received a false-positive high urgency code and 6 patients were unnecessarily referred. The missed diagnosis rate amounted to 5.1% of ACS patients. Thus, despite GPs criticism on triage efficiency, triage results in an (more than) acceptable amount of unnecessary referrals, although the missed ACS rate exceeded the desired threshold.

Implications for further research

This study was primarily focussed on the performance of telephone triage of patients with chest pain in OOH-PC. Our results illustrate that while urgency assessment is commonly used as a risk-stratification tool, this is often insufficient for a safe and efficient distinction between patients with and without a major event. Further research should focus on ways to improve chest pain evaluation in OOH-PC without overlooking the influence of health care professionals operating the system. Improvement might be sought in the content or procedure of the triage process, or by examining ways to improve risk stratification, for instance by using clinical decision rules during triage or GP consultation.28

Conclusion

The urgency assessment during telephone triage of chest pain in OOH-PC is often inaccurate, particularly when relying on the NTS tool alone. Further exploration of ways to improve telephone triage and OOH-PC decision-making is warranted.

Supplementary Material

cmac077_suppl_Supplementary_Material_1
cmac077_suppl_Supplementary_Material_2
cmac077_suppl_Supplementary_Material_3
cmac077_suppl_Supplementary_Material_4

Acknowledgements

We thank the HONK organization and affiliated GPs for their general cooperation and for providing access to patient information.

Contributor Information

Amy Manten, Amsterdam UMC, University of Amsterdam, Academic Medical Center, Department of General Practice, Amsterdam Cardiovascular Sciences Research Institute, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands.

Remco P Rietveld, Huisartsenorganisatie Noord-Kennemerland, Hertog Aalbrechtweg 5A, 1823 DL Alkmaar, The Netherlands.

Lukas de Clercq, Amsterdam UMC, University of Amsterdam, Academic Medical Center, Department of General Practice, Amsterdam Cardiovascular Sciences Research Institute, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands.

Inge van Hulst, Huisartsenorganisatie Noord-Kennemerland, Hertog Aalbrechtweg 5A, 1823 DL Alkmaar, The Netherlands.

Wim A M Lucassen, Amsterdam UMC, University of Amsterdam, Academic Medical Center, Department of General Practice, Amsterdam Cardiovascular Sciences Research Institute, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands.

Eric P Moll van Charante, Amsterdam UMC, University of Amsterdam, Academic Medical Center, Department of General Practice, Amsterdam Cardiovascular Sciences Research Institute, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands; Amsterdam UMC, University of Amsterdam, Academic Medical Center, Department of Public & Occupational Health, Amsterdam Cardiovascular Sciences Research Institute, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands.

Ralf E Harskamp, Amsterdam UMC, University of Amsterdam, Academic Medical Center, Department of General Practice, Amsterdam Cardiovascular Sciences Research Institute, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands.

Funding

Funding for the TRACE study was provided by a grant from the Amsterdam Cardiovascular Sciences Research Institute (ACS-2018-TRACE-22254), a grant from ZonMw-HGOG (project number: 839150004), and internal funding from the Department of General Practice of the Amsterdam University Medical Centers, location AMC, the Netherlands.

Ethical approval

The Medical Ethical Committee (METC) of the Amsterdam University Medical Centers, location AMC, evaluated and approved the study protocol (Netherlands Trial Register: Trial NL7581). All patients were informed by mail on the conduct of this study and were provided with the opportunity to opt-out of sharing data for this study.

Conflict of interest

None declared.

Data availability

Data are available on reasonable request by contacting the corresponding author.

References

  • 1. Bösner S, Becker A, Haasenritter J, Abu Hani M, Keller H, Sonnichsen AC, Karatolios K, Schaefer JR, Seitz G, Baum E, et al. . Chest pain in primary care: epidemiology and pre-work-up probabilities. Eur J Gen Pract. 2009;15(3):141–146. [DOI] [PubMed] [Google Scholar]
  • 2. Jansen T, Ramerman L, Verheij R.. Data of out-of-hours primary care—Triage: index complaints and urgency allocation. Nivel Zorgregistraties Eerste Lijn. 2020. https://www.nivel.nl/sites/default/files/bestanden/1003915.pdf. [Google Scholar]
  • 3. Nilsson S, Scheike M, Engblom D, Karlsson LG, Mölstad S, Akerlind I, Örtoft K, Nylander E.. Chest pain and ischaemic heart disease in primary care. Br J Gen Pract. 2003;53(490):378–382. [PMC free article] [PubMed] [Google Scholar]
  • 4. Smits M, Verheij R.. Changes in urgencies of out-of-hours primary care contacts 2013–2016. Utrecht, the Netherlands: Nivel rapport; 2017. [Google Scholar]
  • 5. Svavarsdóttir AE, Jónasson MR, Gudmudsson GH, Fjeldsted K.. Chest pain in family practice. Can Fam Physician. 1996;42:1122–1128. [PMC free article] [PubMed] [Google Scholar]
  • 6. Verdon F, Herzig L, Burnand B, Bischoff T, Pécoud A, Junod M, Mühlemann N, Favrat, B.. Chest pain in daily practice: occurrence, causes and management. Swiss Med Wkly. 2008;138(23–24):340–347. [DOI] [PubMed] [Google Scholar]
  • 7. Harskamp RE, Fanaroff AC, Zhen SW, Sawe HR, Weber EJ.. Recognising acute coronary syndrome. BMJ. 2022;377:e069591. doi: 10.1136/bmj-2021-069591.. [DOI] [PubMed] [Google Scholar]
  • 8. Haasenritter J, Biroga T, Keunecke C, Becker A, Donner-Banzhoff N, Dornieden K, Stadje R, Viniol A, Bösner S.. Causes of chest pain in primary care—a systematic review and meta-analysis. Croat Med J. 2015;56(5):422–430. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Steeman L, Uijen M, Plat E, Huibers L, Smits M, Giesen P.. Out-of-hours primary care in 26 European countries: an overview of organizational models. Fam Pract. 2020;37(6):744–750. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Domus Medica. Netherlands Triage Standard. 2014. [Google Scholar]
  • 11. Mackway-Jones K, Robertson C.. Emergency triage. BMJ. 1997;314(7086):1056. [Google Scholar]
  • 12. National guidelines for telephone triage and advice in Family Practice [accessed 2022 Feb 14]. https://onderwerpen.nhg.org/spoedzorg/triagewijzer/
  • 13. Keizer E, Maassen I, Smits M, Wensing M, Giesen P.. Reducing the use of out-of-hours primary care services: a survey among Dutch general practitioners. Eur J Gen Pract. 2016;22(3):189–195. [DOI] [PubMed] [Google Scholar]
  • 14. Jansen T, de Hoon S, Zwaanswijk M, Verheij R.. Between ambulance and general practitioner. Utrecht, the Netherlands: Nivel rapport; 2016. [Google Scholar]
  • 15. Wouters LT, Rutten FH, Erkelens DC, De Groot E, Damoiseaux RA, Zwart DL.. Accuracy of telephone triage in primary care patients with chest discomfort: a cross-sectional study. Open Heart. 2020;7(2):e001376. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP.. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):574–577. [DOI] [PubMed] [Google Scholar]
  • 17. Manten A, Cuijpers CJJ, Rietveld RP, Groot E, Graaf F, Voerman S, Himmelreich JCL, Lucassen WAM, van Weert HCPM, Harskamp RE.. Rationale and design of an observational cohort study of triage of acute chest pain in out-of-hours primary care in The Netherlands (TRACE). Prim Health Care Res Dev. 2020;21:e10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18. Ploem C, Harskamp RE, van Dijk N, Scholte R, Hooghiemstra T, Willems D, van Weert HCPM.. Privacy legislation and scientific research. Utrecht, the Netherlands: Huisarts & Wetenschap; 2020. [Google Scholar]
  • 19. Thim T, Krarup NHV, Grove EL, Rohde CV, Løfgren B.. Initial assessment and treatment with the Airway, Breathing, Circulation, Disability, Exposure (ABCDE) approach. Int J Gen Med. 2012;5:117–121. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Ciwit BV. Castor electronic data capture. Amsterdam, the Netherlands: Ciwit B.V.; 2016. [Google Scholar]
  • 21. Huibers L, Smits M, Renaud V, Giesen P, Wensing M.. Safety of telephone triage in out-of-hours care: a systematic review. Scand J Prim Health Care. 2011;29(4):198–209. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Lake R, Georgiou A, Li J, Li L, Byrne M, Robinson M, Westbrook JL.. The quality, safety and governance of telephone triage and advice services—an overview of evidence from systematic reviews. BMC Health Serv Res. 2017;17(1):614. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Giesen P, Ferwerda R, Tijssen R, Mokkink H, Drijver R, van den Bosch W, Grol R.. Safety of telephone triage in general practitioner cooperatives: do triage nurses correctly estimate urgency? Qual Saf Health Care. 2007;16(3):181–184. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24. Wouters LT, Zwart DL, Erkelens DC, Huijsmans M, Hoes AW, Damoiseaux RA, Rutten FH, de Groot E.. Tinkering and overruling the computer decision support system: working strategies of telephone triage nurses who assess the urgency of callers suspected of having an acute cardiac event. J Clin Nurs. 2020;29(7–8):1175–1186. [DOI] [PubMed] [Google Scholar]
  • 25. McConaghy JR, Oza RS.. Outpatient diagnosis of acute chest pain in adults. Am Fam Physician. 2013;87(3):177–182. [PubMed] [Google Scholar]
  • 26. Leite L, Baptista R, Leitao J, Cochicho J, Breda F, Elvas L, Fonseca I, Carvalho A, Costa JN.. Chest pain in the emergency department: risk stratification with Manchester triage system and HEART score. BMC Cardiovasc Disord. 2015;15(1):48. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27. Harskamp R, van Peet P, Bont J, Ligthart S, Lucassen W, van Weert H. The conundrum of acute chest pain in general practice: a nationwide survey in The Netherlands. BJGP Open. 2018;2(4):bjgpopen18X101619. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Harskamp RE, Laeven SC, Himmelreich JC, Lucassen WAM, van Weert H.. Chest pain in general practice: a systematic review of prediction rules. BMJ Open. 2019;9(2):e027081. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

cmac077_suppl_Supplementary_Material_1
cmac077_suppl_Supplementary_Material_2
cmac077_suppl_Supplementary_Material_3
cmac077_suppl_Supplementary_Material_4

Data Availability Statement

Data are available on reasonable request by contacting the corresponding author.


Articles from Family Practice are provided here courtesy of Oxford University Press

RESOURCES