Systematic Review of Clinical Decision Support Systems for Prehospital Acute Coronary Syndrome Identification

Supplemental Digital Content is available in the text.

Abstract

Objectives:

Timely prehospital diagnosis and treatment of acute coronary syndrome (ACS) are required to achieve optimal outcomes. Clinical decision support systems (CDSS) are platforms designed to integrate multiple data and can aid with management decisions in the prehospital environment. The review aim was to describe the accuracy of CDSS and individual components in the prehospital ACS management.

Methods:

This systematic review examined the current literature regarding the accuracy of CDSS for ACS in the prehospital setting, the influence of computer-aided decision-making and of 4 components: electrocardiogram, biomarkers, patient history, and examination findings. The impact of these components on sensitivity, specificity, and positive and negative predictive values was assessed.

Results:

A total of 11,439 articles were identified from a search of databases, of which 199 were screened against the eligibility criteria. Eight studies were found to meet the eligibility and quality criteria. There was marked heterogeneity between studies which precluded formal meta-analysis. However, individual components analysis found that patient history led to significant improvement in the sensitivity and negative predictive values. CDSS which incorporated all 4 components tended to show higher sensitivities and negative predictive values. CDSS incorporating computer-aided electrocardiogram diagnosis showed higher specificities and positive predictive values.

Conclusions:

Although heterogeneity precluded meta-analysis, this review emphasizes the potential of ACS CDSS in prehospital environments that incorporate patient history in addition to integration of multiple components. The higher sensitivity of certain components, along with higher specificity of computer-aided decision-making, highlights the opportunity for developing an integrated algorithm with computer-aided decision support.

Despite a decline in coronary heart disease deaths by more than 50% between 1961 and 2016, coronary heart disease is still a leading cause of mortality in the United Kingdom.¹ In Scotland, acute coronary syndrome (ACS) is a major cause of mortality with 6697 deaths in 2016.² ST elevation myocardial infarction (STEMI) is the most acutely critical subtype of ACS with highest 30-day mortality.^3–5 Time is critical for STEMI management as mortality increases with treatment delays.^4,6 Prehospital STEMI identification has been shown to reduce treatment delays and improve mortality.^7,8

The importance of timely prehospital recognition of STEMI is well established,⁷ yet there are still recognized difficulties. Prehospital difficulties include the absence of complete medical records and lack of diagnostic support tools, such as imaging, which increases the risk of ACS misdiagnosis and creates a low positive predictive value for prehospital ACS diagnosis.⁹ A low positive predictive value increases inappropriate treatment of ACS including cardiac catheterization laboratory (“cath-lab”) activation.¹⁰ Over activation of the cath-lab is a potentially avoidable strain on a valuable clinical resource. False mobilization increases workload of the cath-lab team and often requires unnecessary redirection of emergency medical services to deliver patients to cath-lab centers outside their normal operating zones.

Conversely, under diagnosis of STEMI has obvious negative consequences. Delayed presentation of STEMI has significantly decreased long-term survival rates (73% survival with late presenters vs. 93% survival with early presenters)¹¹ and even late treatment of STEMI via reperfusion of the culprit occluded artery has no benefit in mortality compared to conservative medical therapy.¹² In addition, subtypes of ACS such as non-ST–elevation myocardial infarction (NSTEMI) and unstable angina can be just as critical as a STEMI as ST elevation on the electrocardiogram (ECG) is not exclusive for acute coronary artery occlusion,¹³ that is, a proportion of NSTEMI are actually caused by an occluded coronary artery.

Clinical decision support systems (CDSS) are platforms that combine multiple clinical data inputs (termed “components” in this review) to produce a single output, which can be a diagnosis, clinical advice or risk stratification, that can help clinicians with difficult decision-making.⁹ For instance, CDSS have already been developed for use in the emergency department for ACS,^14,15 where these tend to focus on a high negative predictive value to prioritize safe discharge. In the community, there is increased difficulty for out-of-hospital practitioners, like general practitioners (especially those in remote and rural communities) and ambulance crews, to make triage decisions in patients with ACS without the clinical diagnostic tools that are available in the hospital. These difficulties are compounded with suspected ACS that presents without obvious ST elevation as a nondiagnostic ECG creates further ambiguity. This challenge has been the target of CDSS-related research to assist prehospital clinicians to manage patients who have suspected ACS.¹⁶

With great interrogation of technology in healthcare, there is a large potential for computer-aided diagnosis of ACS in the prehospital setting. Computer-aided decision support has already been shown to be beneficial determining allocation for level of life support in the emergency department.¹⁷ In addition, computer-aided ECG interpretation algorithms have been developed to improve prehospital and emergency department ACS identification to reduce the delay or misdiagnosis of ACS associated with prolonged door-to-balloon time.¹⁸

However, computer-aided ECG interpretation is still limited by ECG artifact and other nonischemic causes of ST elevation such as early-repolarization and thus interpretation of the ECG should be done in combination of other components such as symptoms and medical history.^19,20

In the prehospital environment, there are concerns that CDSS can cause delays compared to standard care⁹ and that these systems might reduce the autonomy of clinicians.²¹ However, previous studies have shown the benefit of prehospital CDSS for patients with stroke²² and spinal injury.²³ One review looked at prehospital CDSS for ACS but excluded tests using computer-aided decision systems and biomarker tests.²⁴ With advances in computer technology and point-of-care testing, the use of these components is now increasingly realistic in a prehospital setting.

The aim of this systematic review was to describe the accuracy of CDSS and their individual components in the prehospital management of ACS.

METHODS

The search strategy followed the guidelines set by the preferred reporting items for systematic review and meta-analysis.²⁵ The review protocol was designed with guidance from the preferred reporting items for systematic review and meta-analysis protocol statement and was registered with Prospero (registration number: 116600).²⁶

Search Strategy

The search strategy was designed and executed by the first author. Five databases were searched: EMBASE, Medline, Cochrane Library, Web of Science, and CINAHL. The searches were performed between December 2018 and January 2019. Grey literature was also reviewed for any additional sources. The search terms used are in Appendix 1 (supplemental material, available at http://links.lww.com/HPC/A216).

Study Selection and Eligibility

Abstracts and titles were screened and selected if they were adjudged to be relevant to the review aim. Duplicates were excluded. The review focused on the use of CDSS in a prehospital setting where patients presented with symptoms suggestive of ACS. Definitions for ACS included STEMI, NSTEMI, or unstable angina as per European Society of Cardiology guidelines.²⁷ Prehospital was defined as contact with first emergency responders (including paramedics, medical dispatch callers, general practitioners). Studies carried out in the hospital environment or emergency department were excluded. Patient history was defined as subjective symptoms reported by the patient (eg, chest pain, shortness of breath, and clamminess), while vital signs/examination were defined as objective noninvasive clinical measurements obtained by clinical staff (eg, heart rate, blood pressure, and oxygen saturations).

Inclusion criteria were as follows:

1. Published source

2. Data on patient diagnosis or outcome such as major adverse cardiovascular events

3. Set in a prehospital setting

4. Use of CDSS as an intervention

5. Patients with suspected ACS

6. English language

Exclusion criteria were as follows:

1. No data on outcomes

2. Inclusion of emergency department/inhospital decision aids

3. Inclusion of nonsuspected ACS patients

4. No definition of myocardial infarction (MI)

5. Not in English language

Full-text versions of the papers selected were obtained and analyzed. Papers were then included or excluded based on the criteria. A second reviewer judged the selection process and analyzed the eligible papers separately by the criteria for consensus. Any disagreements were resolved by discussion between the 2 reviewers to reach a consensus. Cohen’s kappa coefficient was performed between the reviewers to analyze the rate of agreement.

Assessment of Quality and Risk of Bias

Quality assessment was conducted using a quality assessment tool for diagnostic accuracy studies (QUADAS).²⁸ Articles were analyzed to ensure there was no obvious missing data and that patients progressed through the study as described. Studies were excluded from analysis where there was a high or unclear risk of bias. They were then ranked according to level of evidence as determined by published hierarchy of evidence, which takes into account any validation and impact analysis of CDSS.²⁹

Data Extraction

Data were extracted using a data extraction tool that was piloted with 2 initial studies and subsequently refined. The data types that were extracted are outlined in Appendix 2 (supplemental material, available at http://links.lww.com/HPC/A216). The primary outcome recorded from studies was a final diagnosis of ACS accuracy.

Data Analysis

Data analysis was performed using statistical analysis software SPSS 24.0 (SPSS Inc., Chicago, IL). The sensitivity, specificity, and positive and negative predictive value of CDSS were examined. The results were reported as percentages and analyzed as continuous data. Whether a history, examination/vital signs, ECG, and biomarker components were included in the study, then this was described as binary (yes or no) and treated as categorical data. Independent-samples t test was used to analyze the difference of mean accuracy (percentage) between CDSS with and without components. A P value ≤0.05 was considered to be statistically significant. Because of the considerable heterogeneity between the papers selected, formal meta-analysis was deemed not possible.

RESULTS

Study Selection and Quality Assessment

Figure 1 outlines the search and selection process for this review. The titles and abstracts for 11,439 articles were screened. A total of 199 articles were initially identified through this process and reviewed. Of these, 182 articles did not fulfill eligibility criteria, leaving 17 articles.

Figure 1.

Flow chart of literature search and selection process.

The studies were assessed for their quality using the QUADAS 2 tool.²⁸ Four studies were rejected from the study due to high risk of bias. A further 5 studies were assessed to have some minimal or moderate bias, all with patient selection, as would be expected with nonrandomized prospective and observational studies.^20,30–33 Only 2 studies had validation phases for their CDSS and there was no impact analysis with any of the 17 articles thus undermining of the potential quality of evidence as judged by the predefined hierarchy.^31,33 Ideally, validation and impact analysis would be required before any CDSS could be judged suitable for implementation in other health localities. The second reviewer screened the 17 selected studies and Cohen’s kappa coefficient for interobserver agreement between the 2 reviewers was calculated at k = 0.46, which equates to moderate agreement. Following collaboration with the second reviewer and QUANDAS 2 tool quality control, 9 studies from the 17 were excluded, leaving 8 studies that were included in the analysis.^{16,20,30–35}

Study Characteristics

Seven of the 8 studies were prospective in nature, with use of CDSS performed “on-site” by a general practitioner, emergency medical services staff, or medical dispatcher.^16,30–35 Two studies were retrospective analyses of patients and included either computer-aided ECG interpretation or decision-making.^20,34 Table 1 displays the demographic characteristics of the patients in the 8 studies. A total of 354,259 patients were in the studies combined; however, 1 study contributed 347,989 patients making up 98% of the total population. The average number of patients excluded was 69, the majority being from 1 study which only had data on 15% of patients.³⁰ Mean age was 65 years and 54% of participants were male. Half of the studies were conducted in the Netherlands, with a further 2 in the United States and the remainder in Sweden and Japan. Five studies involved emergency medical services, 2 involved general practitioners, and 1 involved a medical call dispatch team.

Table 1.

Study Characteristics

Heterogeneity

There was a large degree of heterogeneity in the 8 studies. The first study was published in 1996, and the last 22 years later in 2018.^16,33 As noted above, 7 of the studies were prospective and 1 was retrospective in design. The composition of the CDSS components differed, with 7 studies involving patient history^16,30–35; 6 involving prehospital ECG interpretation^{16,20,31,33–35}; 5 involving examination and vital signs^{16,31–33,35}; and 2 involving a prehospital biomarker test.^16,35 The last 2 studies were the only ones to develop CDSS that incorporated all 4 components.^16,35 With regards to the outcomes measured, 3 reported ACS (MI including unstable angina)^30,32,33; 3 others reported STEMI^20,31,34; 1 reported NSTEMI¹⁶; and 1 reported a major adverse cardiovascular event³⁵(defined as any MI, primary PCI, coronary artery bypass graft or any cause of mortality). The definition of MI also differed between studies, with 3^30,32,34 using the universal guidance on the diagnosis of MI,³⁶ 2 other studies^16,35 used the third universal definition of MI,³⁷ while the final 3 used a combination of ECG findings, biomarkers, and history to diagnose ACS.^20,31,33 The incidence of the ACS also was widely different between the studies, ranging from 0.02% to 50%.

Statistical Analysis

The results of the analysis of the outcomes, sensitivities, specificity, positive predictive value, and negative predictive value of the studies are described in Table 2. The sensitivity between the studies varied from 100% to 58%, with specificity varying from 100% to 10%, positive predictive value between 100% and 7%, and negative predictive value between 100% and 30%. Table 3 shows that only the inclusion of patient history was found to have a significant impact on improving accuracy of sensitivity and negative predictive value of CDSS.

Table 2.

Results of the Analysis of the Outcomes, Sensitivities, Specificity, PPV, and NPV of the Individual Studies

Table 3.

Mean of Combined Clinical Decision Support Systems Accuracy With Incorporated Components

DISCUSSION

The utility of CDSS for ACS in prehospital settings is yet to be established. This systematic review of the literature, the first to be conducted on the topic, found considerable variations in the components of CDSS that were examined in existing studies. The extent of the heterogeneity precluded a formal meta-analysis; however, a comparison of which components were key in successful CDSS was performed.

This review found that the use of the patient history component in CDSS remains highly important in diagnosis with significant improvement on the sensitivity (P = 0.002) and negative predictive value (P < 0.001). These findings highlight the potential of CDSS that incorporate patient history in a “rule-out” capacity for an ACS diagnosis. The significant impact of patient history in this review may have been due to patient history being the most prevalent tool in CDSS with it being included in 7 of the 8 studies reviewed.^16,30–35 In comparison, prehospital ECG was used in 6 of the 8 studies,^{16,20,31,33–35} vital signs and examinations were used in 5 studies,^{16,31–33,35} and biomarkers were used in just 2 studies.^16,35

Interestingly, in the 2 studies^16,35 that used all 4 components (the ECG, a point-of-care biomarker, patient history, and vital signs/examinations), both achieved high sensitivity (100% and 96%) and negative predictive value (100% and 97%) but poor specificity (43% and 29%) and positive predictive value (29% and 21%). However, both studies excluded patients with clear ST elevation, thus focusing on the risk stratification of patients between ACS and non-ACS rather than triage of patients with NSTEMI or STEMI. Three of the studies^20,31,34 looked exclusively at the identification of STEMI and had a larger range of specificity (88%–100%) and positive predictive value (7%–100%). The heterogeneity of the findings appears to be dependent on the aim of a prehospital CDSS to differentiate between a “rule-in/out” for ACS or between NSTEMI and STEMI.

Recent advances have allowed the use of high-sensitivity troponins to achieve a high degree of sensitivity in the diagnosis of ACS, leading to the reduction of unstable angina diagnosis.³⁸ One of the studies reviewed demonstrated that there is the capability of the traditional point-of-care troponin to be used in CDSS.¹⁶ However, there were issues reported with the test used in this study, including device errors, inability to obtain blood, and the risk of false negatives when samples were taken shortly after symptom onset. The study was also limited by the use of a single troponin value in isolation, where clinicians are unable to observe any trends and a raised troponin does not always indicate myocardial ischemia but may be a result of myocardial injury.²⁷ A computer-based machine learning algorithm for the diagnosis of MI has been developed with a paired troponin, analyzing the rate of change of troponin along with age and sex showing strong sensitivity at 97.8% and specificity of 92.2%.³⁹ However, the study required a second troponin at 1–3 hours following the initial troponin measurement and therefore would not be feasible in the prehospital environment. The value of an isolated troponin in the prehospital situation maybe more apparent in combination with other components of CDSS such as patient history and suggestive ECG features. In addition, the use of prehospital high-sensitivity troponin tests in comparison to the inhospital test may aid in the sensitivity when identifying ACS where shorter time from symptom onset to test can reduce sensitivity.⁴⁰

The use of contemporary risk stratification algorithms for MI has been shown to be effective following hospital admission, with examples like the History, ECG, Age, Risk factors, Troponin (HEART) score, Thrombolysis in Myocardial Infarction (TIMI) score and Global Registry of Acute Coronary Events (GRACE) score.^41–43 Two studies^16,35 used the HEART score as the clinical decision algorithm to aid in ACS risk stratification, with 1 study³⁵ modifying the score with the use of a high-sensitivity troponin rather than the conventional fourth-generation troponin measurement. Although there was excellent sensitivity (100%) and negative predictive value (100%) for the modified HEART score algorithm, specificity (43%) and positive predictive value (29%) were less accurate. This could be due to the designation of intermediate and high values in the modified HEART scores as a “positive” score in this review. When adjusting for only the high scores on the modified HEART algorithm, then specificity increases to 87% and positive predictive value to 51%. As the authors acknowledge, the main objective of the HEART score is to rule-out rather than rule-in ACS; however, the risk stratification element could aid the rapid transfer of high-risk patients to specialist cardiac facilities.³⁵

The greatest area for future development in CDSS is with computer-aided interpretation. Three of the CDSS in this review incorporated computer integration for either ECG interpretation or for the final clinical decision.^20,30,34 The accuracy of MI diagnosis is seen in the 2 studies that utilized computer-aided interpretation of ECG, with high specificity (100% and 99%) and positive predictive value (100% and 83%).^20,34,44 However, 1 study which looked only at the digital ECG for the decision support had lower sensitivity (58%) and negative predictive value (30%).²⁰ The use of computer-aided decision-making is a rapidly developing field with advances in radiology and pathology especially.⁴⁵ However, the role of computer-aided decision-making in ECG interpretation has been previously reported with varying sensitivity and specificity.^46,47 Deep learning techniques for ECG interpretation have enormous potential to improve ECG ACS detection with the ability to detect subtle signs of ischemia and continually learn from their findings.⁴⁸

Computer-aided decision-making was not only limited to ECG interpretation. One study looked at the use of a computer-aided decision system for medical dispatch to patients presenting with chest pain, with the only component being patient history, and it found good sensitivity (92%) and negative predictive value (97%) but poor specificity (41%) and positive predictive value (17%).³⁰ The use of a computer-aided decision system can help assimilate a large amount of data when assessing a patient and help prioritize patients dependent on certain features in the history and risk factors. Innovations in computerized ACS diagnosis highlight the potential of machine learning where constant refinement of the algorithm accuracy can produce increasingly accurate decisions.³⁹

This use of computer-aided decision systems in the prehospital setting can be advantageous, where often there is no experienced cardiologist present and paramedic crews, with limited training, may have to interpret the clinical situation and ECG alone.⁴⁷ The one study which had the computer ECG interpretation with combination of a clinical screening tool led to high sensitivity (86.9%) and specificity (98.5%) suggesting that an integrated approach with other components could be beneficial.³⁴

Limitations

There are several important limitations with this study. Due to the high volume of ACS research, and in combination with the broad-search strategy, there is a possibility that some literature has been missed. This search strategy was employed to aid the identification of studies that examined principle components, such as patient history within CDSS, before the adoption of new technologies, such as prehospital ECG and biomarkers.

The considerable heterogeneity in CDSS which limited the statistical analyses that could be done, particularly with 1 study contributing 98% of the population for statistical analysis, hence the results must be taken with caution. In addition, MI was variously defined using the published universal definitions of MI,^{16,30,32,34,35} a combination of an ECG and biomarker criteria^31,33 or by ECG alone.²⁰ There was a notable variation in the incidence of ACS, ranging from 0.02%³¹ to 50%.²⁰ This was due to patient selection for analysis, with the first study having included all patients presenting to emergency services (n = 347,989), whereas the second study focused exclusively on prehospital transmitted ECGs suspected of STEMI (n = 200) and therefore targeted a select patient group with a higher incidence of ACS.

Despite this, the review was able to document the nature and extent of the heterogeneity of the studies, including the components of CDSS and the methods used to examine them. It also provided the opportunity to examine what components were important in the prehospital diagnosis of ACS, and to compare the value of individual components and combinations thereof.

Further Research

Further research would be useful to assess the accuracy of the high sensitivity of CDSS involving multiple components combined with the high specificity of computer-aided decision systems. CDSS research requires further validation in different clinical environments before CDSS are deployed for widespread use. In addition, impact analysis also helps judge whether the beneficial effects of CDSS would remain once fully incorporated into clinical use. Other effects that CDSS have on users need to be explored, including automation bias where the clinician can overtrust the decision aid.⁴⁹ The user interface design of CDSS is another area that needs further research. Human factors and interaction design guidelines are often ignored in designing CDSS.⁵⁰ However, 1 study used human–computer interaction design principles to design CDSS to aid ECG interpretation.⁵¹ They used eye-tracking analysis of ECG interpretation⁵² and their understanding of human cognition and working memory to breakdown the ECG interpretation process into manageable tasks on CDSS to eventually present multiple automated diagnoses to prevent automation bias and to encourage differential decision-making. While CDSS are mostly concerned with the provision of algorithmic text-based suggestions, future study may also involve better use of intelligent dynamic graphics as part of the algorithmic output for depicting more spatiotemporal data to augment the decision support.⁵³ Finally, new studies that evaluate diagnostic CDSS would ideally focus on sensitivity, specificity, positive predictive value and negative predictive value of CDSS algorithms and use consistent definitions of MI. Studies that also use consistent definitions and outcomes between them would help with the development of a successful CDSS algorithm that integrate multiple components to provide an effective clinical aid.

Summary

CDSS are increasingly prevalent in healthcare and in combination with computer-aided decision and point-of-care biomarkers, they could provide a way of improving the accuracy of prehospital diagnosis and outcomes of treatment. With risks associated with delayed treatment of ACS and, alternatively, pressures on hospital resources such as cardiac cath-lab activation, there is an opportunity to create an efficient and safe diagnostic pathway before hospital admission. This review has highlighted the importance of patient history in diagnosis but also the potential for combining components such as biomarkers and computer-aided decision ECG interpretation in the integration of CDSS for suspected ACS.

DISCLOSURES

The authors have no conflicts of interest to declare.