Systematic review of current natural language processing methods and applications in cardiology

Abstract

Natural language processing (NLP) is a set of automated methods to organize and evaluate the information contained in unstructured clinical notes, which are a rich source of real-world data from clinical care that may be used to improve outcomes and understanding of disease in cardiology. The purpose of this systematic review is to provide an understanding of NLP, review how it has been used to date within cardiology, and illustrate the opportunities that this approach provides for both research and clinical care. We systematically searched six scholarly databases (ACM Digital Library, Arxiv, Embase, IEEE Explore, PubMed, and Scopus) for studies published in 2015–2020 describing the development or application of NLP methods for clinical text focused on cardiac disease. Studies not published in English, lacking a description of NLP methods, non-cardiac focused, and duplicates were excluded. Two independent reviewers extracted general study information, clinical details, and NLP details, and appraised quality using a checklist of quality indicators for NLP studies. We identified 37 studies developing and applying NLP in heart failure, imaging, coronary artery disease, electrophysiology general cardiology and valvular heart disease. Most studies used NLP to identify patients with a specific diagnosis and extract disease severity using rule-based NLP methods. Some used NLP algorithms to predict clinical outcomes. A major limitation is the inability to aggregate findings across studies due to vastly different NLP methods, evaluation, and reporting. This review reveals numerous opportunities for future NLP work in cardiology with more diverse patient samples, cardiac diseases, datasets, methods, and applications.

Introduction

A vast amount of data is collected during routine clinical care. Clinicians cognitively process this data, organizing it into contextual information which is then documented in clinical notes. Data has inherent structure, while the information contained in clinical notes is unstructured text. Structured data are managed as computable data elements (e.g., diagnosis codes, blood pressure reading, laboratory values), while unstructured text (clinical notes) lacks organization and standardized formatting, making it challenging to analyze at scale in its raw form. The ability to organize and evaluate the information contained in clinical notes at scale provides a rich source of real-world data from clinical care.¹ Unfortunately, by some estimates, more than 80% of the information in electronic health records (EHRs) is in unstructured formats.²

Natural language processing (NLP) is a set of automated methods for interpreting different aspects of natural language, including syntax (the arrangement) and semantics (the meaning) of words and phrases (Figure 1). A spectrum of NLP approaches exists, ranging from identification of text strings to deep learning. Many NLP models can interpret the complex natural language contained in clinical text, including medical jargon, misspellings, and abbreviations, into accurate representations of clinical information.

Figure 1.

Basic steps of natural language processing (NLP) and NLP applications in cardiology

There is potential for researchers and clinicians to use NLP to extract information from unstructured clinical notes which may then be used in studies to improve outcomes and understanding of disease.³ The involvement of researchers and healthcare professionals in cardiology in the development and application of novel NLP methods is needed to ensure these methods are accurate and representative, and that envisioned use cases are relevant and feasible in cardiac contexts. Researchers are increasingly applying machine learning methods in cardiology,⁴ but NLP methods and its applications in clinical care have not been thoroughly described in systematic reviews.

The purpose of this systematic review is to provide investigators and clinicians in the field of cardiology with an understanding of NLP, to review how it has been used to date within cardiology, and to illustrate the opportunities that this approach provides for both research and clinical care. We synthesize and discuss current trends in clinical applications, applicability of test datasets, NLP methods, and primary findings of recent NLP research in cardiology, with the goal of increasing awareness of how these methods can be used to extract information from clinical text and encouraging future innovations and applications among researchers and healthcare professionals in cardiology.

Methods

This review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for reporting in systematic reviews.

Article retrieval

We searched the metadata of articles in six scholarly databases in medicine, science, and computer science: ACM Digital Library, Arxiv, Embase, IEEE Explore, PubMed, and Scopus. A combination of search terms relating to NLP and cardiology were selected based on the Medical Subject Headings vocabulary (U.S. National Library of Medicine) with additional terms identified from prior NLP-focused systematic reviews^{5, 6} through collaboration with a medical librarian (DW). We applied filters to only search publications in the English language and from 1/1/2015 through 12/31/2020 to ensure relevance given the rapid advances in NLP in recent years. The reference lists of included studies were reviewed to identify additional relevant studies for potential inclusion. Details of the search strategy are provided in Supplemental File 1.

Study selection

We used Covidence software (www.covidence.org) to organize and structure our review of retrieved studies. Studies considered eligible for inclusion were published in calendar years 2015–2020, described the development or application of NLP methods for clinical text (EHRs), and were clinically focused on a patient population with existing cardiac disease. Studies were excluded for the following reasons: (1) not published or available in English, (2) duplicate studies, (3) aspects of the same study published by the same research group in multiple publications, (4) lacking a description of NLP methods or applications, and (5) focus on patients without existing cardiac disease, such as patients with cardiac risk factors but no diagnosed disease; while these do represent additional areas of interest in NLP, they were felt to be beyond the scope of this work. Following this strategy, three reviewers (MRT, AV, DS) performed two rounds of study selection: title and abstract screening followed by full-text review. Each article was screened by two independent reviewers and disagreements were discussed among the three reviewers until consensus was achieved.

Data extraction and synthesis

Data from each included article was independently extracted by two of three reviewers (MRT, AV, DS). Extracted data included general study information (design, objectives), clinical details (cardiac focus, patient characteristics), and NLP details (NLP methods, evaluation metrics). To reduce complexity, evaluation metrics were reported as ranges when performance metrics for multiple cohorts or methods were reported separately. All reviewers worked from the same understanding of common NLP terms and methods, described in Table 1. Data for each article was extracted by two independent reviewers and discrepancies were resolved through discussion.

Table 1.

Glossary of NLP methods and metrics

NLP methods
Key term search	Identify and extract terms from pre-specified list of terms of interest.
Named entity recognition	Locate and translate terms, or named entities, into predefined categories of concepts, often using controlled medical vocabularies.
Rule-based methods	Detect concepts of interest based on an established set of rules or logic, often using regular expressions, which are sequences of characters that define a search pattern.
Convolutional neural network	A deep learning neural network approach identifying, weighting, and connecting “nodes” across multiple convolutional “layers” of nodes (including a convolutional layer) and applying filters between layers.
Conditional random fields	A classification approach that accounts for context in order to recognize patterns and make predictions.
Decision tree	Hierarchical trees of knowledge used to classify concepts of interest.
Logistic regression	A basic building block for neural networks; a classification approach used to discover links between concepts of interest.
Random forest	An “ensemble” of decision trees built using a combination of learning models and used to produce more accurate and stable predictions.
Recurrent neural network	A deep learning neural network approach designed to interpret temporal or sequential information and used to make predictions.
Evaluation Methods
Manual annotation	The task of reading pre-selected texts and marking (i.e., annotating) linguistic components (paragraphs, sentences, phrases, or words) that represents concepts of interest.
Cross-validation; also called held out testing set	A technique to evaluate predictive models by partitioning the original sample into a training set to train the model, and a test set to evaluate it.
Performance metrics
Positive predictive value (PPV); also called precision	The percentage of results that were actually relevant among all results that the system obtained.
Negative predictive value (NPV)	The percentage of results that were actually irrelevant among all results that the system did not obtain.
Sensitivity; also called recall	The percentage of results that were actually obtained by the system among all results that should have been obtained.
Specificity	The percentage of results that were actually not obtained by the system among all results that should not have been obtained.
F-score	A combination of PPV/precision and sensitivity/recall; can be weighted to give more significance to one measure.
Accuracy	The percentage of results that were actually relevant among all results that were and were not obtained.
Area under the curve (AUC)	Reflects the degree to which a model is capable of classifying or distinguishing between classes or events of interest.

Quality appraisal

While relevant reporting standards for NLP research have not been established,⁷ we conducted a modified quality appraisal based on the approach described by Koleck and colleagues,⁶ who documented the presence of specific quality indicators in NLP articles. We included additional machine learning quality indicators described by Nascimento and colleagues.⁸ Each article was appraised by two of three reviewers (MRT, AV, DS) and disagreements were resolved through discussion.

Results

Article screening and included studies

After applying eligibility criteria, 37 articles were included in the review (Figure 2). We retrieved 653 studies from scholarly databases. Covidence automatically identified and excluded 261 studies as duplicates. During the title and abstract screening, the majority of studies were excluded for not having a cardiology focus (n=327) and not using NLP or providing a description of the NLP methods (n=200). During the full text screening, studies were mainly excluded for not having a cardiology focus (n=64) or not providing details about the NLP methods (n=51). The detailed exclusion cascade is provided in Supplemental File 1 and a complete list of screened articles and exclusion reason are provided in Supplemental File 2.

Figure 2.

PRISMA flow diagram

Description of the included studies

Of the 37 included studies, 15 were published in biomedical informatics or engineering journals, 12 in cardiology journals, six in general biomedical research journals, and the remaining four in other disciplines including nursing, public health, and radiology. Table 2 reports on the patient populations, datasets, and NLP methods of included studies. The samples of cardiac patients included a mix of hospitalized and non-hospitalized patients with sample sizes ranging from 60 to over 621,000 patients. Among studies reporting demographic characteristics of patient samples (n=15), the mean age ranged from 56 to 90 years old, 45–99% were male, and 48–94% were Caucasian. Data sources included a single hospital (n=16), regional or national healthcare systems (n=10), and an existing corpus of notes or patient registry (n=11). The majority of studies (n=28) were conducted in the US. The number of documents analyzed ranged widely from 310 to over 2.1 million notes, and consisted primarily of inpatient progress notes, outpatient notes, and echocardiogram reports. Fifteen studies used rule-based methods (n=15), named entity recognition (n=13), key term search (n=11), and other methods (n=9) including convolutional neural networks, conditional random fields, decision trees, logistic regression, random forests, and recurrent neural networks. Several studies used previously developed tools, primarily Leo and MedTagger.

Table 2.

Quality appraisal results by cardiac disease focus

	Clearly defined purpose	Number of patients specified	Patient demographic information reported	Number of documents specified	Detailed description of NLP approach	Parameterization conducted	Inclusion of comparative evaluation	Detailed description of comparative evaluation design	Justification for evaluation design and metrics	Evaluation metrics reported	Statistical treatment of results (e.g., confidence tests)	Discussion of model costs (time, resources) and explainability	Availability of code and datasets for reproducibility
Coronary Artery Disease
Esteban et al, 20179	✓	✓		✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Hu et al, 201613	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Owlia et al, 201910	✓	✓	✓	✓	✓	✓	✓			✓	✓
Safarova et al, 201611	✓	✓		✓	✓	✓	✓	✓	✓	✓	✓
Shah et al, 201912	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Toerper et al, 201614	✓	✓	✓	✓	✓		✓			✓
Electrophysiology
Bean et al, 201916	✓	✓	✓	✓	✓		✓			✓	✓	✓	✓
Hu et al, 201921	✓	✓	✓		✓	✓	✓			✓	✓
Moon et al, 201920	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓		✓
Moon et al, 202019	✓	✓		✓	✓	✓	✓	✓	✓	✓	✓		✓
Rosier et al, 201618	✓	✓			✓	✓	✓	✓	✓	✓	✓	✓
Shah et al, 2020a17			✓	✓	✓	✓	✓	✓	✓	✓	✓		✓
Shah et al, 2020b15	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
General cardiology
Viani et al, 201922	✓			✓	✓	✓	✓	✓	✓	✓	✓	✓
Heart Failure
Alnazzawi et al, 201623	✓			✓	✓	✓	✓	✓	✓	✓	✓
Bielinski et al, 201524	✓	✓	✓	✓	✓		✓	✓	✓	✓	✓	✓	✓
Eggerth et al, 202036	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Evans et al, 201630	✓	✓					✓			✓
Garvin et al, 201833	✓	✓		✓	✓	✓	✓	✓	✓	✓	✓	✓
Jonnalagadda et al, 201734	✓	✓		✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Kaspar et al, 201825	✓	✓	✓	✓	✓		✓	✓	✓	✓	✓
Leiter et al, 202037	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Liu et al, 201931	✓			✓	✓	✓	✓	✓	✓	✓	✓		✓
Mahajan et al, 201932	✓	✓		✓	✓		✓			✓
Patel et al, 201826	✓	✓	✓		✓		✓			✓	✓
Topaz et al, 201735	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Wagholikar et al, 201827	✓	✓		✓	✓	✓	✓	✓	✓	✓	✓		✓
Wang et al, 201528	✓	✓		✓	✓	✓	✓	✓	✓	✓	✓
Zhang et al, 201829	✓	✓		✓	✓	✓	✓	✓	✓	✓	✓
Imaging
Adekkanattu et al, 201938	✓	✓		✓	✓		✓			✓
Nath et al, 201639	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Patterson et al, 201742	✓			✓	✓	✓	✓	✓	✓	✓	✓		✓
Shi et al, 201540	✓			✓	✓	✓	✓	✓	✓	✓	✓		✓
Valtchinov et al, 202043	✓			✓	✓	✓	✓	✓	✓	✓	✓		✓
Xie et al, 201741	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Zheng et al, 202044	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Valvular Disease
Galper et al, 201845	✓			✓	✓		✓			✓

Study purposes and primary findings

By subspeciality, six studies focused on coronary artery disease (CAD), seven on electrophysiology (EP), 15 on heart failure (HF), seven on imaging, one on valvular heart disease, and one on general cardiology. Figure 3 presents a summary of the primary areas of application of the NLP methods. Supplemental File 1 reports on the purposes and study outcomes for each included study; below we describe these by subspecialty.

Figure 3.

Summary of the number of included articles by area of NLP application and cardiac disease

Coronary Artery Disease

Within CAD, most studies focused on identification and classification of disease^9–12 while two focused on prediction of major adverse cardiovascular events¹³ and inpatient admissions following cardiac catheterization.¹⁴ The studies demonstrated the ability to use NLP to identify CAD events and symptoms,⁹ Canadian Cardiovascular Society angina classification,¹⁰ symptoms and test results related to myocardial infarction,¹² and patients with familial hypercholesterolemia¹¹ with sensitivity, specificity, and positive and negative predictive values over 80% for most studies. The algorithm predicting major adverse cardiovascular events outperformed two widely used acute coronary syndrome risk score tools with an AUC of 72%.¹³ The algorithm predicting admissions following cardiac catheterization also reported an AUC of 72%, and identified age, gender, and past medical and surgical history-related factors associated with increased risk of admission.¹⁴

Electrophysiology

Study purposes within EP included identifying patients with atrial fibrillation¹⁵ and characterizing those most likely to receive guideline-directed thromboembolic prophylaxis at the time of hospital discharge,^{16, 17} an algorithm to evaluate the significance of atrial fibrillation alerts received from remote monitoring of cardiac implantable electronic devices based upon calculated CHA₂DS₂-VASc stroke risk score data obtained from the EHR,^{18, 19} extracting family history information,²⁰ and predicting cardiac resynchronization therapy outcomes.²¹ One study was able to identify patient with atrial fibrillation with F-scores of 93–94%.¹⁵ Studies aiming to characterize anticoagulant use reported sensitivities of 90–97% and found that risk scores were more accurate for the CHA2DS2-VASc than HAS-BLED,¹⁶ and models including unstructured and structured data together were more accurate than unstructured data alone.¹⁷ Studies aiming to automatically detect heart rhythm reported an accuracy of 98%¹⁸ and F-scores of 92–98%;¹⁹ they also reported that unstructured data identified rhythm and therapy delivered from ICDs more accurately than structured data alone.¹⁹ One study aiming to extract family history information found a machine learning model that incorporated unstructured data had sensitivities of 91–95% and specificities of 90–98%.²⁰ Another study predicted cardiac resynchronization therapy outcomes with a PPV of 79%, sensitivity of 26%, and AUC of 75%.²¹

General cardiology

One general study focused on extracting events from cardiology-focused notes in the Italian language, seeking to adapt existing NLP methods which have been primarily developed for the English language.²² The authors reported that models integrating recurrent neural networks with standard dictionary-lookup approaches performed better than recurrent neural networks alone.

Heart Failure

Within HF, most studies focused on either identification and classification of disease, including subtypes based on left ventricular ejection fraction and New York Heart Association (NYHA) heart failure class,^23–29 and prediction of hospital readmissions and mortality.^30–32 Other studies aimed to automate a HF quality measure,³³ automate screening for a clinical trial,³⁴ identify patients with ineffective self-management,³⁵ evaluate medication adherence,³⁶ and identify documented symptoms among patients undergoing cardiac resynchronization therapy.³⁷ The studies aiming to identify and classify HF had sensitivities of 60–100%, specificities of 96–99%, PPV’s of 71–96%, NPV’s of 87–100%, F-scores of 74–94%, and accuracies of 77–100%.^23–29 The studies aiming to predict hospital readmissions and mortality found NLP improved model performance over structured data alone^30,31 and that deep learning models outperformed other machine learning models,³¹ but performance statistics varied widely (PPV 98%,³⁰ F-score 73–76%,³¹ AUC 51–65%³²). Other studies successfully used NLP methods to characterize HF patients for quality care metrics (PPV 89–99%, sensitivity 27–100%)³³ and for clinical trials eligibility (PPV 86%, sensitivity 95%).³⁴ The study aiming to use NLP to identify patients with ineffective self-management reported a PPV of 95% and sensitivity of 79%, and identified specific types of self-management deficits that were significantly associated with readmissions.³⁵ Studies also demonstrated moderate success applying NLP to evaluate medication adherence (F-score 55–90%)³⁶ and identify HF symptoms (F-score: 72%).³⁷

Imaging

Most studies within imaging focused on extracting data elements from echocardiogram reports.^38–42 One focused on identifying patients with implantable devices prior to MRIs,⁴³ and one on interpreting exercise treadmill test results.⁴⁴ The studies aiming to extract data elements from echocardiogram reports reported widely variable PPV’s (6–100%) and sensitivities (25–100%); some studies reported NLP was reliable across concepts of interest while others reported wide variability in performance metrics between concepts.^38–42 One study found that expert-derived and ontology-derived NLP methods had similar accuracy (expert-derived: 83%, ontology-derived 91%) in identifying patients with implantable devices prior to MRIs.⁴³ One study used NLP to extract relevant information from exercise treadmill test results with a sensitivity of 96% and specificity of 95%, and was able to associate test results with risk of severe 30-day outcomes (myocardial infarction, death).⁴⁴

Valvular disease

The one article that was focused on valvular disease aimed to extract and analyze adverse events from transcatheter aortic valve replacement (TAVR) and MitraClip procedures.⁴⁵ The study found that NLP found common events associated with TAVR and MitraClip procedures with high correlation to structured data alone (R² 0.86).⁴⁵

Indicators of quality

Table 2 reports on indicators of quality^{6, 8} across the included studies. The following indicators were met by the fewest studies: discussion of model costs and other implementation considerations (n=9; 24%), availability of code and datasets for reproducibility (n=13; 35%), and patient demographic information reported (n=18; 49%).

Discussion

In this systematic review, we found the majority of NLP work in cardiology has been concentrated in a small number of clinical domains (primarily HF) and NLP methods (primarily rule-based). The most common applications were extracting information for disease identification and classification purposes; these studies reported fairly high accuracy, indicating that NLP algorithms are well developed towards this goal in cardiovascular disease.^{9–12, 16–20, 23–29, 38–45} Fewer studies used machine learning to predict outcomes and reported moderate predictive abilities (AUC 51–75%).^{13, 14, 30,21, 31} Below we describe how cardiology researchers and clinicians interested in engaging in NLP can leverage multiple opportunities to further explore patient and disease conditions, datasets, and novel NLP applications. These future research directions are summarized in Table 3.

Table 3.

Future research directions of NLP in cardiology

1.	Applying NLP to study a broader range of cardiovascular diseases (beyond heart failure) and more diverse, representative patient samples to reduce bias in trained models.
2.	Developing and applying sophisticated NLP approaches, using machine learning, to accomplish complex tasks such as generating disease timelines, monitoring drug safety, and untangling symptom-physiology relationships.
3.	Leveraging open-source, previously developed NLP tools to study portability and reliability of tools across health systems and use cases.
4.	Exploring the value of other types of unstructured health data for cardiology beyond inpatient physician notes, such as nursing progress notes, primary care notes, patient-generated health data and social media content.
5.	Conducting rigorous evaluations identifying strategies to improve explainability and address other challenges surrounding implementation of NLP algorithms in clinical practice (i.e., costs, clinical workflows, time burden).

Our review showed that NLP research in cardiology has been concentrated on a few disease areas, potentially because author lists suggest very few research groups are working at the intersection of NLP and cardiology. In future work NLP may be used to address a broader range of cardiovascular diseases, especially those that are growing in prevalence such as atrial fibrillation, and study more diverse patient samples. In this review, patient samples were predominantly middle- to older-age, male, and Caucasian. This may be explained by the underlying patient populations seeking care. Nonetheless, increasing attention has been paid to the bias in machine learning; training models with unrepresentative patient samples causes them to work less effectively for, and potentially harm, underrepresented patient groups.^{46, 47} Future studies should consider the importance of diverse, representative samples when training, validating, and implementing NLP methods to ensure they work well for the diverse range of patients receiving cardiac care in the US and globally.

Another relevant issue is explainability of predictive models. Our review demonstrated that very few articles (24%) addressed implementation considerations such as explainability. Clinicians frequently mistrust machine learning-based predictive models because of challenges understanding how machine learning-based models generate outputs, which has slowed adoption of these models in clinical practice.⁴⁸ Recognizing this problem, novel interpretability methods, such as SHAP (SHapley Additive exPlanations) values,⁴⁹ have improved insight into predictions, but much work to improve explainability and usefulness of these models in clinical practice remains.

More than half of the included studies focused on formative methods development versus evaluation of previously developed tools, and only two studies described NLP methods being adopted in routine clinical care.^{28, 30} Most of the previously developed NLP tools employed by the included studies are now publicly available (Table 4). However, portability and reliability of previously developed tools are major concerns, as performance often differs between institutions and EHR systems with different document structures and linguistic expressions. Lack of portability remains a significant challenge in NLP research, and may explain why the majority of studies in this review developed novel NLP methods rather than reuse existing algorithms and tools. There are also opportunities to explore novel unstructured data sources, such as outpatient and primary care notes,¹² nursing documentation, which has been used to predict mortality,⁵⁰ and even non-clinical unstructured data sources, such as social media posts and patient-generated health data.

Table 4.

Portfolio of open-source NLP tools and datasets applied in cardiology contexts

Name and Origin	Description	Accessibility
NLP tools
clinical Text Analysis and Knowledge Extraction System (cTakes); Mayo Clinic	A modular pipeline of components using both rule-based and machine learning methods to support information extraction; based on UIMA (Unstructured Information Management Architecture) standards.	Open-source at http://www.ohnlp.org
EchoExtractor; Veterans Affairs	An application which extracts Concept-Value pairs for metrics measured during an echocardiogram study.	Open-source at https://github.com/department-of-veterans-affairs/EchoExtractor
Leo; Veterans Affairs Informatics and Computing Infrastructure (VINCI)	A set of services and libraries that leverages UIMA standards to enable rapid creation and deployment of NLP analysis tools and incorporation of previously developed tools.	Open-source at https://department-of-veterans-affairs.github.io/Leo/userguide.html
MedTagger; Mayo Clinic	A set of tools developed for indexing based on dictionaries, information extraction based on patterns, and machine learning-based named entity recognition to support information extraction; based on UIMA standards.	Open-source at https://github.com/OHNLP/MedTagger
pyConText; University of Utah	A Python implementation of ConText, a simple text processing algorithm for identifying a large number of features and relationships between features.	Open-source at https://pypi.org/project/pyConTextNLP/0.6.0.5/
semEHR; King’s College London, UK	A general-purpose search and analytics tool that processes heterogeneous data sources, covers a range of biomedical concepts, and captures context to support information extraction in study-specific or case-specific contexts.	Open-source at https://github.com/CogStack/CogStack-SemEHR
Datasets
The Medical Information Mart for Intensive Care III (MIMIC III), Massachusetts Institute of Technology	Deidentified, freely available, critical care database of over 60,000 intensive care unit admissions.	https://mimic.mit.edu/
Electronic Medical Records and Genomics (eMERGE) network, National Human Genome Research Institute (NHGRI)	Combines DNA biorepositories with EHR data from several clinical sites nationally, and has been extensively used to develop phenotyping algorithms.	https://emerge-network.org/
Integrating Biology and the Bedside (i2b2), Partners Healthcare	A dataset of deidentified patient discharge summaries made available for research purposes.	https://www.i2b2.org/

Finally, there are novel areas of NLP application that this review suggests are underexplored in cardiology. The few studies developing predictive algorithms demonstrated that unstructured data greatly improves algorithm performance,^{17, 19, 25, 26, 30, 32, 45} suggesting opportunities for greater use of NLP for prediction tasks in future work. Deep learning models incorporating NLP methods have been applied extensively in oncology to identify temporal events to generate clinical timelines, extract highly detailed tumor information, match patients to clinical trials, and conduct drug-safety surveillance.⁵¹ NLP has also been used in clinical decision support systems in other clinical contexts but was largely unexplored in the included articles. Finally, NLP may be applied to untangle symptom-physiology relationships, study symptom assessment and management practices, and support interventions to improve patient quality of life.⁶ In this review, few of the studies focused on symptoms, potentially because of the high degree of symptom overlap between conditions, which obfuscates a symptom’s etiology, and the lack of normalized symptom concepts in controlled vocabularies.⁶

Limitations of this review include the inability to aggregate findings across studies due to vastly different methods, evaluation, and reporting around NLP. Similarly, studies reported evaluation metrics for NLP with varying degrees of detail. We reported ranges of metrics for brevity, which removed detail necessary to understand nuanced differences between specific methods and concepts being extracted. Additionally, to maintain a focused scope of this review, we excluded some studies in interesting, related areas, such as prediction of cardiovascular disease among patients without existing disease, and cerebrovascular disease.

Finally, for conciseness, we identified several studies from the same author groups reporting on the same or highly similar research studies. In these cases, we included only the most recent study under the assumption that they represented the most evolved NLP methods from that project or body of work; however, this may have biased findings towards improved performance.

Conclusion

NLP is an underutilized method for unlocking information from unstructured notes in EHRs. This systematic review of the state of the science of NLP in cardiology identified several areas of success with NLP in cardiology, specifically the identification and classification of disease phenotypes and the augmentation of predictive outcome models through the inclusion of unstructured data. It also points to opportunities for future research and clinical opportunities, including novel patient and disease conditions, datasets, and applications.