Defining Phenotypes from Clinical Data to Drive Genomic Research

Abstract

The rise in available longitudinal patient information in electronic health records (EHRs) and their coupling to DNA biobanks has resulted in a dramatic increase in genomic research using EHR data for phenotypic information. EHRs have the benefit of providing a deep and broad data source of health-related phenotypes, including drug response traits, expanding the phenome available to researchers for discovery. The earliest efforts at repurposing EHR data for research involved manual chart review of limited numbers of patients but now typically involve applications of rule-based and machine learning algorithms operating on sometimes huge corpora for both genome-wide and phenome-wide approaches. We highlight here the current methods, impact, challenges, and opportunities for repurposing clinical data to define patient phenotypes for genomics discovery. Use of EHR data has proven a powerful method for elucidation of genomic influences on diseases, traits, and drug-response phenotypes and will continue to have increasing applications in large cohort studies.

1. INTRODUCTION

The widespread adoption of electronic health records (EHRs) has raised the possibility of using these data in clinical research. Abundant evidence now supports the idea that the EHR repurposed for research represents a rich data set of a patient’s health trajectory, including diseases, laboratory and radiology tests, and medications and their response, much of which can be hard to acquire in a research setting. Simultaneously, high-throughput methods for genotyping have considerably decreased the cost of genetic discovery while also improving in accuracy and availability. A key component of these are genome-wide association studies (GWAS) and whole genome and exome sequencing technologies, which systematically analyze variation across the genome. Since 2005, over 3000 GWAS have identified almost 40,000 unique SNP-trait associations (1).

The vast majority of early genomic research studies before 2010 were performed using observational cohorts or randomized controlled trial data. Perhaps in part as a result, some of the largest GWAS published have been traits that are common to many studies, such as height and body mass index. Nearly coinciding with the growth of genetic investigation has been the national adoption of EHRs across the United States. The national adoption rate among non-federal acute care hospitals was only 9.4% in 2008 but reached 94% by 2013 (2). The formation of EHR-linked DNA biobanks that repurpose EHR data from their clinical data stores to research-oriented databases, combined with advances in informatics tools and terminologies, led to the beginning of successful EHR utilization for genetic studies in 2010 (3–7) The first EHR-based studies recapitulated non-EHR GWAS by focusing on well-defined clinical phenotypes to test against variation in the human genome but later gave rise to other innovative, reverse-genetics approaches such as phenome-wide association studies (PheWAS), which provide a systematic approach to the analysis of many phenotypes potentially associated with a specific genotype (8).

Critical to discovery through genomic and phenomic investigation is the accumulation of sufficiently large sample sizes with well-defined phenotypes. These phenotypes include specific diseases or observable traits and are used to decipher the genetic determinants of human diseases, physiologic attributes, and medication response. In this review, we highlight the growth of and approaches to phenotyping using clinical data within EHRs as it pertains to genomics research, limitations of these techniques, and future opportunities for integration of genotypic and phenotypic clinical data.

2. USE OF CLINICAL DATA IN RESEARCH

2.1. Cohorts Available for Genetic Study

Traditional genetic studies have used population- or clinical trial-based cohorts with prospective participant enrollment and questionnaires to gather data on specific aspects. While this approach can result in high-quality phenotypes, there are significant challenges due to the time and monetary expense required for participant accrual, retention, questionnaire completion, and validation (9, 10). Patient accrual can take months to years. Further, long-term follow-up and patient retention can be tedious or unfeasible, with exclusion of populations of patients important to include, such as those too ill to participate or at the extremes of age. Prospective cohorts are also generally guided by a particular clinical research question, and thus phenotype data may be limited to those conditions, hindering reuse of these data for future studies examining different phenotypes.

Accordingly, EHRs have emerged as an efficient method for obtaining dense patient information for research over the last two decades. Historically, clinical data and documentation has been collected primarily to support patient care and administrative functions, such as billing. Thus, cross-sectional aggregation and querying of EHR data was not a priority. In 2003, the National Academy of Medicine (then the Institute of Medicine) released a report on the key capabilities of an EHR, noting that facilitating research is an important secondary use of EHRs (11). Early studies utilizing aggregated EHR data often consisted of epidemiological studies performed at early EHR adoption sites with well-maintained databases such as the Veterans Health Administration or UK general practice research database (12–14). Recent efforts bringing together diverse international healthcare data have been able to study treatment protocols in as many as 250 million individuals (15). Some examples of network initiatives utilizing EHR data for genomic research are detailed below.

2.2. Current Biobanking Efforts

Biobanking of genetic data linked to the longitudinal patient data available within the EHR aggregates otherwise disparate information, potentially making it available to both clinicians and researchers. Some of the earliest biobanks linked to EHR data derived their samples from left over blood collected as part of clinical care, and would have otherwise been discarded (9). These include the Harvard Crimson, which pursued as-needed sample collection for phenotypes of interest, and Vanderbilt University Medical Center’s BioVU, which started as a prospective collection of all individuals who did not opt-out of DNA collection as part of their routine consent to treatment (16, 17). Since 2015, BioVU has converted to an opt-in consent model (electronically requested at the point of care) due to changes in the National Institutes of Health (NIH) Genomic Data Sharing policy requiring subject consent for data sharing (18).

Several other initiatives have been launched across the world for the development of very large EHR-linked biobanks which have started to deliver biomedical data sets comprising extensive phenotype and genotype information on hundreds of thousands of subjects. In the United States, the Electronic Medical Records and Genomics Network (eMERGE) and the Million Veteran Program (MVP) represent examples (19, 20). The MVP has recruited more than 580,000 participants and is establishing a longitudinal study of Veterans for future genomic and clinical research that combines data from survey instruments, the EHR, and biospecimens (20).

One of the more robust EHR phenotyping efforts has been performed by the eMERGE consortium, a national network organized and funded by the National Human Genome Research Institute with the goal of combining DNA biorepositories with EHRs for high-throughput and generalizable genomic discovery. An integral aim of eMERGE is to support the creation, validation, and dissemination of phenotype algorithms by providing tools that guide the user through the stages of development, public sharing, and reuse (21).

In Europe, the UK Biobank is a large prospective study of more than 500,000 individuals to investigate the role of genetic factors, environmental exposures, and lifestyle in the causes of major diseases (22). Participants aged 40–69 years were enrolled over 4 years in 22 recruitment centers, each completing questionnaires and donating biospecimens. The emphasis is now on further phenotyping of participants and ascertaining their health outcomes through follow-up and linkages to healthcare-based datasets (23). In Asia, the China Kadoorie Biobank also has more than 500,000 individuals and has prospectively linked genomic information to both EHR data and participant surveys.

The All of Us Research Program^a is a NIH-funded initiative to build a United States research cohort of more than one million individuals, including prospective participant provided information, molecular data (including genomics), and linkage to health information in EHRs (24). Participants will be recruited from diverse healthcare centers located across the country and as “direct volunteers”, individuals who may not have a direct connection with a recruiting healthcare system.

3. REPURPOSING OF EHR DATA

3.1. Rationale for use of Clinical Data for Genomics Research

EHRs offer longitudinal patient information in a form that is relatively unbiased to particular diseases or research agendas, allowing for study of diverse genomic risk, diseases, and outcomes. The rich phenotypic clinical documentation coupled with laboratory data, medication receipt, family history, and environmental exposures, makes the EHR a practical data source for reuse in genomic studies. The key advantage of repurposing EHRs for research is that they are already created and maintained for healthcare delivery and prospectively accrue clinical observations and costly tests at regular intervals driven by an individual’s health trajectory.

To quantify the density of information in the EHR, we explored the data available for BioVU participants (Figure 1). We found that individuals had an average of 7.8 years of clinical data, including many years before enrollment and up to 33 years of longitudinal recorded information. Much of these data are costly and may be infeasible to collect for a research trial. For example, consider the cost to obtain and clinically interpret the 3.4 million radiology tests (~14 radiology tests per patient) presented in Figure 1; these included over 628,000 computed tomography (CT) scans and nearly 192,000 magnetic resonance imaging (MRI) scans, all with clinical interpretations. Assuming a conservative estimate of $500 per test, these CT and MRI scans alone would cost $410 million. Thus, there is a significant reduction in research time and expense for accrual of large sample sizes with a breadth of clinical data (9, 25).

Figure 1. Density of data in the Vanderbilt EHR-linked biobank, BioVU.

While enrollment in BioVU and accrual of samples for DNA analysis started in 2007, clinical data within the electronic health record on the individuals enrolled dated as far back as 1984. Data points were transformed by taking the square root and dividing by 20.

Another potential benefit of linking DNA repositories to EHRs is the inclusivity of EHRs in comparison to traditional population-based cohorts, which often will exclude certain diseases, children, minority or poor populations, and the elderly. This is critical for both identification of a range of cases as well as controls. Because EHR-based cohorts provide significant variability in phenotypic traits, a single cohort can be reused many times for many phenotypes or genetic variants examined (9, 26). Once the genetic data have been collected, the majority of cost and effort is thus expended at defining, refining, and validating phenotypes of interest.

Large sample sizes for modern genetic research methods, such as GWAS and PheWAS, are critical to the discovery of novel findings and afforded by the EHR (27). For both GWAS and PheWAS, there is a need to correct for multiple comparisons, increasing the threshold for statistically-significant results. For GWAS, the threshold has been established at 5×10⁻⁸ (28). While the threshold for statistical significance is less well-established for PheWAS, a Bonferroni correction is often applied in these analyses, resulting in a conservative significance level that assumes independence across all phenotypes, which is unlikely given that many phenotypes of human diseases and traits are closely related. Since statistical power is a function of number of tests performed, effect size (often low for many variants), and minor allele frequency, use of this stringent threshold has necessitated use of larger and larger cohorts to enable identification of significant associations, especially those that are rare or low-frequency variants of moderate-to-large effect.

3.2. Classes of Data Available in EHRs for Phenotype Curation

EHR phenotyping is the process of identifying individuals with an explicit observable trait from large quantities of imperfect clinical patient data (Figure 2) (29). The earliest approach to phenotyping is via manual chart review, typically performed by thorough searching of clinical documents, laboratory, and medication information by individuals with medical domain knowledge. For automation of phenotyping, EHR data, which is stored in both structured and unstructured formats, is extracted and utilized for analysis (Table 1). Structured data is typically easier for computerized extraction, with examples being billing codes, laboratory results, vital signs, and often medications. A large proportion of the EHR is relatively unstructured, including almost all clinical documentation, radiology reports, and some test or laboratory results. For many phenotypes, accurate case selection is best achieved through use of a combination of structured and unstructured data (10).

Figure 2. Methods of electronic health record (EHR)-based phenotyping.

Clinical concepts are in both structured and unstructured (require machine-extraction) in the EHR. These concepts can be used for various forms of phenotyping.

Table 1.

Common data classes within electronic health records (EHRs)

	Demographics	Diagnoses	Procedures	Medication records	Laboratory results	Imaging	Clinical text documentation
Data format	Structured	Structured	Structured	Partially structured	Mostly structured	Partially structured	Mostly unstructured
Data standard	None	ICD9/10	CPT	RxNorm	LOINC	DICOM for images	SNOMED-CT
Query method	Simple	Simple	Simple	Simple, text searching, NLP	Simple, text searching	Simple, text searching, NLP	NLP
Recall	Moderate	Moderate	Variable	Moderate	Moderate	Moderate	Moderate
Precision	Moderate	Low	High	Moderate	High	High	High
Affected by healthcare fragmentation	No	Low/moderate (chronic) to high (acute)	High	Moderate	High	High	Moderate
Strengths	Easy to query	Easy to query	Easy to query	High validity for inpatient setting	High validity	High validity	Most dense clinical information; can capture out-of-hospital history
Weaknesses	Variable recall and precision based on demographic	Susceptible to inaccuracies	Susceptible to missing data	Susceptible to missing data	Variable recall and precision based on test; susceptible to missing data	Susceptible to missing data; difficult to process raw images	Most difficult to process and interpret at scale
Used in Phenotyping	Most common	Most common	Common	Common	Somewhat common	Somewhat common	Less common

^*CPT, Current Procedural Terminology; DICOM, Digital Imaging and Communications in Medicine; ICD-9/ICD-10, International Classification of Diseases, Ninth Revision/Tenth Revision; LOINC, Logical Observation Identifiers Names and Codes; SNOMED-CT; NLP, Natural Language Processing; Systematized Nomenclature of Medicine-Clinical Terms

Billing data are the most commonly used resource for identifying phenotypes in both clinical and genomic research (21, 30). This structured data typically consists of International Classification of Diseases (ICD) and Current Procedural Terminology (CPT) codes. The ICD coding system classifies diseases, symptomatology, and procedures based on a hierarchical terminology structure maintained by the World Health Organization (WHO). CPT codes were created by the American Medical Association, and are used to bill for clinical services, such as an imaging study or surgical procedure. Both classes of billing data are ubiquitous and easily queried within EHRs, making them highly utilized as at least a portion of most phenotyping algorithms. A query of the data types used in phenotyping algorithms in the Phenotype Knowledgebase (www.pheKB.org) shows that 122 of 154 (73%) algorithms used ICD codes (Table 2), all of which used ICD-9 codes and 26 that used both ICD-9 and −10 codes. This demonstrates one challenge of using billing codes within longitudinal patient information as coding systems change over time, such as the migration from ICD-9 to ICD-10 coding in 2015 in the US, resulting in the need for mapping strategies to combine codes from different systems. Sensitivity and specificity of billing codes alone are variable across phenotypes, with one study showing a range of positive predictive values (PPVs) for ICD codes from 0.12 to 0.56 across ten diseases (29). ICD codes generally have low specificity but are highly sensitive for diseases, as a clinician may bill an ICD code for a diagnosis based upon clinical suspicion rather than confirmation of disease (31). CPT codes tend to have higher specificity, as procedural coding is quite accurate, but lower sensitivity in comparison to ICD codes due to procedures being performed at other institutions, demonstrating fragmentation of EHR data (32).

Table 2. Data modalities used in phenotyping algorithms available on PheKB.

Data as of 10/15/2017. Non-public algorithms include algorithms in development and those whose performance has not yet been validated.

	Public (n = 44)	Non-public (n = 110)	Percent of total
ICD-9 or −10 codes	39	73	73%
Medications	31	51	53%
CPT codes	23	44	44%
Natural language processing	28	36	42%
Laboratory test results	21	37	38%
Vital signs	5	14	12%

^*ICD-9/ICD-10, International Classification of Diseases, Ninth Revision/Tenth Revision; NLP, Natural Language Processing; CPT, Current Procedural Terminology

Other types of structured data within the EHR, such as laboratory results and medications, are also often used to identify phenotypes (Table 2). In particular, 53% of algorithms available on PheKB utilize medication data. Record of medication receipt can be in various forms in the EHR; however, inpatient computerized provider order entry systems and outpatient drug prescribing systems have increased the availability of drug exposures as structured data. Medication data in the absence of corroborating evidence has not been found to be especially useful, with area under the receiver operator characteristic curve (AUC) of 0.54 for 10 diseases (29). Importantly, the capture of medications is essential to provide exposure data for pharmacogenomic studies (31, 33). Challenges to analyzing laboratory data in particular include the repeated measures, naming conventions, and various specimen sources, resulting in difficult interpretation. While laboratory and medication data must be placed into appropriate clinical context with careful selection, they can improve phenotyping accuracy for many conditions.

The main source of unstructured data within the EHR is clinical documentation, consisting of the most accurate record of the providers’ thoughts and richest information for phenotype algorithms. To be useful for electronic-based phenotyping, clinical documentation must be in a format that is computable. The majority of clinical notes consist of narrative text, lacking uniform format or structure, thus they must be processed with either basic keyword searching or modern tools such as natural language processing (NLP), discussed further in the next section.

3.3. Processing of EHR Text Data

Narrative clinical documentation includes a wealth of information about diagnoses, signs and symptoms, risk factors, treatments, family history, exposures, and clinical decision making, many of which are not well captured by structured information in most EHRs. Indeed, the gold standard in validating whether a patient has a diagnosis or a given trait generally involves a review of the clinical notes (30). NLP is a tool for producing computable representations from this narrative unstructured text (34). There are many approaches to NLP, ranging from rule, grammar, and machine learning (ML)-based approaches for producing comprehensive “understandings” of the text (so-called “general-purpose” NLP systems (35–40)) to focused applications applied to particular tasks (e.g., identifying medications and their features (41), or left ventricular ejection fraction (42)). General-purpose NLP systems often seek to parse unstructured text documents into phrases that can be mapped to concepts within controlled terminologies such as the Unified Medical Language System (UMLS), the Systematized Nomenclature of Medicine-Clinical Terms (SNOMED-CT), or RxNorm (for medications) (37, 41, 43–46). Mapping the textual elements within clinical free-text documents to a semantic terminology provides a standardized method to represent the data for downstream computation. Other tasks that often improve the performance of NLP systems include identification of negation and qualifiers (47), semantic role labeling (48), word sense disambiguation (49), and temporal analysis (50).

Some important clinical features often have historically only been found in narrative data, such as family history and smoking status. Smoking status has been the focus of many dedicated rule-based, ML, and hybrid NLP systems with F-measures ranging from 84–97% (38, 51–53). NLP also has the capacity to identify family history information with variable accuracy in prior studies (54–57). Identifying these important features within clinical documents allows the features to be reused and add meaningful information to diverse research studies. The introduction of Meaningful Use Stages 2 and 3 with the requirement for structured data entry for smoking status and family history will make these data more available over time with NLP tools (58, 59). Further, functionality such as Duke University’s MeTree, which allows a patient to complete his or her own family history, are providing new avenues for increased entry of structured information in the EHR (60, 61).

Performance for concept extraction varies with recall and precision ranging from 70–90%, depending on the specific application of the NLP (36–38, 44, 62). For this reason, EHR-based phenotyping algorithms have typically combined NLP features with other data components. Specific phenotypes such as adverse events and diseases can be successfully extracted using NLP (63–68), with several studies reporting higher predictive capability for case identification through NLP (either alone or in combination with other methods) in comparison to use of billing codes alone (19, 32, 34, 64, 69, 70). Researchers have used both general-purpose NLP systems and focused applications, but the latter have become more common, since most phenotype algorithms do not need to identify all concepts but instead require a high-precision text mining approach to identify a set of terms applicable to specific concepts. For example, nearly all Vanderbilt phenotypes involving medications have employed a general-purpose NLP tool for medication extraction (MedEx) (41), but we have used a mix of regular expressions, general-purpose NLP tools (66, 71), or ML-based approaches for specific phenotypes (72). While general-purpose NLP systems can be utilized across a range of phenotypes, they also generally have inferior recall and precision to purpose-built approaches for defined phenotypes. Further, while NLP methods have significantly improved in the ability to identify negation and sentence structure, these algorithms remain imperfect and secular shifts in documentation, such as the transition from use of dictation to typed templates to “point-and-click” note writers, can result in instability of algorithm performance.

3.4. Challenges of Repurposing EHR Data for Research

While EHRs contain a wealth of extractable information for phenotype classification, their interface and the data generated within them are used primarily for clinical care and reimbursement, typically with little consideration towards research impact. The secondary use of EHRs for clinical, genomic, and pharmacogenomics discovery can be challenged by variable accuracy, lack of standardization, irregular follow-up, incompleteness of patient records, and significant amounts of unstructured information. Inaccuracy within EHRs can result from clinical uncertainty, omissions, or billing errors. Omission can occur due to provider workload and perceptions on what is deemed to be clinically relevant to report at the time of the encounter. Due to the lack of EHR centralization, the length and depth of a patient’s record can vary greatly due to when a patient inhabits a region, what insurance the patient carries and the hospital accepts, and where a patient receives his or her care, with patients often seeing multiple disconnected providers within a region. One study to evaluate the effect of potential data fragmentation on the accuracy of a phenotyping algorithm for type 2 diabetes found that almost one-third of cases were missed if EHR data from only a single site was used (73). While completeness of the EHR is difficult to define, it is important for researchers to understand the likely limitations of the data and how it may affect study findings (74).

4. APPROACHES TO IDENTIFYING PHENOTYPES IN THE EHR

Large-scale, efficient phenotyping methods that utilize data within the EHR provide benefits that are not limited to genomic studies. Phenotyping is crucial to the identification of a population of patients that satisfy a set of criteria, which is important for clinical trial recruitment, retrospective cohort or outcomes studies, and cost analyses, among others. However, identification of patients that belong in a particular cohort is time-consuming and challenging. Different approaches to automated phenotyping have been pursued; we describe the most frequent and promising (Figure 3, Table 3).

Figure 3: Phenotyping methodological approaches differ in their clinical and informatics resources needed.

The width/height of each oval represents the range of that type of resource needed.

Table 3.

Strengths and Weaknesses of Current Phenotyping Approaches using Electronic Health Records (EHRs)

	Manual chart review	Rule-based computerized phenotyping	Supervised machine learning	Unsupervised machine learning	Phenome-wide approaches (PheWAS)
Description	Clinician review of medical records	Manually-created Boolean logic	Classification based on a set of features	Phenotypes learned through clustering of computer-identified features	Systematic analysis of many phenotypes, usually using very simple rule-based approaches
Data variables	Can be used to narrow records for manual review	Structured data, NLP	Structured data, NLP	Any	Most commonly billing codes but can be lab or NLP
Recall	High	Variable	Variable	Variable	High
Precision	High	High	High	Variable	Variable
Costs	High	Moderate; major cost is in algorithm development	Moderate; major cost is in creating training set	Moderate; usually needs very large data collections and computing resources	Low
Scalability of cohort size	Poor; costs directly proportional to N	High	High	High (large cohorts required)	High
Scalability of phenotype number	Poor	Poor	Poor	High	High
Transportability	High	Variable but often high	Variable; often less than rule-based	Variable; often less than rule-based	High
Expertise Required	Clinical domain knowledge	Clinical domain knowledge and informatics support	Clinical domain knowledge and informatics support	Little to no clinical domain knowledge, significant informatics support needed	Little to no clinical domain knowledge, minimal informatics support needed
Strengths	High validity, considered gold-standard; little informatics support needed	High validity; Ease of interpretation	Typically less costly than manual review	Can identify unspecified phenotypes; Unbiased; No domain expertise or manual review	Ascertain broad range of phenotypes; can be used for very large populations
Weaknesses	Significant knowledge and time required	Requires iterative algorithm development and manual review; Domain and informatics expertise needed	Require manually classified training sets; may require expert feature selection	Potentially poor interpretability or irrelevant findings if not coupled with supervised approaches	Often phenotype detail and accuracy limited by billing codes
Current use	Idea for small cohorts and validation for computational methods	Often used in GWAS for single phenotypes	Often used in GWAS for single phenotypes	Rare, but increasing for identifying phenotypes from unstructured data	Hypothesis generation; drug repurposing; interpretation of novel genomic loci

^*PheWAS, Phenome-wide association studies; NLP=natural language processing

4.1. Logical Constraint-Based Approaches

The earliest automated approach applied to recognize patients with a particular phenotype of interest is through the use of Boolean logic. The simplest example of this is utilizing hierarchical billing code structures to determine cases and controls, as is performed in a standard PheWAS analysis. However, more commonly these phenotype algorithms are more complex, manually-curated, and based upon rules applied in a step-wise fashion (75). Similar to the manner in which a content expert would determine case status, the logical constraint-based algorithms incorporate information from various sources of the EHR, including billing codes, clinical documents, laboratory data, and medication exposures.(19) Structured data, either data that is extracted from the EHR in structured way or is processed into a standardized format using methods such as NLP, are necessary as input for the algorithm.

Construction and validation of Boolean algorithms are typically an iterative process with collaborations between clinical domain experts, bioinformaticians, clinical informaticians, NLP experts, and genomics researchers. Clinical experts are typically required for creation of the algorithm itself, and manual effort is required for review of at least a subset of the case and control cohorts classified by each algorithm to ensure the algorithm’s accuracy. The time and effort required is extremely phenotype dependent as the algorithms can vary from fairly simple to very complex. For example, the number of rules in an algorithm can differ dramatically: a 2012 review of 9 phenotypes found that algorithms contained between 8 and 174 rules for case identification (76).

Logical constraint-based approaches to phenotyping have several advantages. The most profound benefit is that they are the simplest to interpret and implement, allowing them to be more readily replicable and transferable across different EHRs or clinical enterprises (21, 26, 66). In 2011, a GWAS was performed to identify associations with primary hypothyroidism using a phenotyping algorithm developed at a single site and implemented to be transportable across EHRs and institutions (26). This algorithm incorporated billing codes, laboratory values, text queries, and medication records within five separate EHRs. Overall, the algorithms’ PPVs were 92.4% and 98.5% for cases and controls, respectively. For the controls, PPVs at all sites were above 95%, while PPV for cases as varying sites ranged from 82–98%, with the lowest PPV mainly due to misclassification of individuals who had undergone thyroidectomy elsewhere or in the distant past. Experience has shown that algorithms developed for identification of rare phenotypes or those including only billing codes for case determination typically have the poorest performance.

These algorithms work well for circumstances in which high validity is needed for a single phenotype for a disease or simple trait that may need to be applied across multiple institutions. The ease of interpretation makes logical constraint-based algorithms attractive to clinicians, and thus research results potentially more translatable to clinical practice, and they can easily be applied to large data sets. The principal limitation is that a new algorithm needs to be created for each new phenotype pursued.

4.2. Machine Learning (ML) Approaches to Defining Phenotypes

ML-based algorithms have been proposed as a method to achieve the improved accuracy and breadth needed to scale phenotype annotation. ML approaches can automate the identification of complex patterns to classify individuals into different groups, such as a case or control for a given phenotype. The traditional approach to ML is that of supervised learning, in which an expert creates a “gold standard” of classified individuals and a feature set used for determination and then the trained algorithm can be used to make predictions on unlabeled examples (77). When individuals are not labeled into groups, an unsupervised learning approach can be applied, which attempts to find natural clusters or patterns of data and individuals. An unsupervised approach can also be used for feature extraction and then tested against an annotated outcome of interest for creation of the classifier (which can be viewed as a type of supervision). Fully unsupervised ML requires no need for domain expert annotation or feature selection, thus this approach is high-throughput and scalable but also can lead to greater difficulty in interpretation.

4.3. Supervised Machine Learning

Supervised ML requires a set of training examples belonging to either a phenotype case or control and then can build a model that can be used on other examples for classification. Several different methods can be used for supervised ML, such as support vector machines (SVM) (66, 72, 78, 79), logistic regression (66, 80), random forests (70), or neural networks (81, 82).

While supervised ML has been shown to be extremely effective for individual tasks, the requirement for manual annotation, a time-consuming and costly process, and feature selection results in limited scalability (83). The conventional approach to building a set of annotated samples is to select a random pool of individuals to manually classify. Active learning approaches to ML-based phenotyping can potentially overcome the need for large annotated datasets. Chen et al. demonstrated the use of an uncertainty sampling algorithm to find and annotate only the most informative samples, those samples with the most uncertain features (83). This approach achieved similar classification results with annotation of a fraction of samples compared to the use of a randomly annotated set. ICD-9 codes have been used as a surrogate for defining training cases and controls to limit manual input; however, this confines phenotypes to those defined by billing codes (79, 84).

Feature selection is the technique of selecting a subset of potential terms or features to use in the ML model. ML-based classification algorithms can incorporate a range of features, including billing data, clinical documentation extracted with NLP, semantic terminologies, medication exposures, and laboratory data. Features can either be chosen from a set of all potential clinical concepts or from a refined set through application of algorithms (e.g., using univariate statistical associations, ML approaches, or penalized regression models) or by domain expert curation. Potential features within the EHR are vast, and for many phenotypes, it is impractical to find an optimal subset of features using manual approaches. Prior studies have suggested the use of unrefined feature sets including all clinical concepts to reduce the domain knowledge required for phenotyping (72). While this provides effective performance for some phenotypes, algorithms using unrefined feature sets typically have lower accuracy than those using expertly curated features, unless very large training sets are used. Bejan et al. proposed statistical feature selection by ranking features based upon their association with a case or control to improve algorithm performance for phenotype identification of pneumonia (85). Yu et al. created the Automated Feature Extraction for Phenotyping (AFEP) approach for disease phenotyping by using NLP applied to knowledge sources (such as Medscape and Wikipedia) to develop medical concepts relevant to a disease followed by concept screening, in which concepts that were either too rare, too common, or not relevant enough were excluded from the feature sets in automated ways (86). AFEP algorithms achieved accuracy comparable to the use of expert-curated feature sets for rheumatoid arthritis and coronary artery disease.

Another approach to limit the need for manually engineered features is to use deep learning. Deep learning allows for the construction of a hierarchy of progressively complex feature layers, with transformation into more abstract representations at higher levels by training a neural network. The key aspect of deep learning is that these layers of features are not designed by domain experts, and rather they are learned from the data (87). Gulshan et al. applied supervised deep learning to more than 128,000 images of patients with diabetic retinopathy along with controls, with the development of algorithms that identified diabetic retinopathy with similar performance to ophthalmologists (AUC of 0.99) (81). Similarly, Esteva et al. used deep neural networks for automated classification of skin lesions, demonstrating the capability of identifying skin cancer with performance comparable to dermatologists (82).

In an early demonstration of deep learning neural networks applied to longitudinal health data, Lasko et al. applied an unsupervised feature learning approach to produce phenotypic features from uric-acid measurements for inputs in a supervised logistic regression classifier, and the model was capable of accurately distinguishing (with an AUC of 0.97) between gout and acute leukemia (88). A unique aspect of this study was managing the variable representation of time intervals, since labs are taken at irregular intervals for patients. Lab values for unobserved times were estimated using Gaussian probability distributions to allow drawing of continuous curves that formed the learned features from the deep learning model.

An important limitation of supervised ML is that the methods require training sets labeled with the phenotypes that they will “learn” to find. Thus supervised learning can be successful in finding patterns that explain phenotypes we have the knowledge to label, but not when we don’t have the knowledge or ability to label the phenotypes and rather aim to discover phenotypes from the data (88).

4.4. Unsupervised Machine Learning

Unsupervised ML has demonstrated success using an unbiased approach towards phenotype discovery. This method could conceivably identify all phenotypes in a data set, including disease subtypes, medication response or adverse events, and previously unrecognized disease patterns. There are several types of unsupervised ML methods, including clustering, dimensionality reduction techniques, and tensor factorization. Prior studies have used unsupervised ML and hierarchical clustering to learn subphenotypes (or sets of comorbidities that group together) of autism (89, 90). Others have used clustering methods to identify distinct phenotypes of bicuspid aortic valves (91) and bronchiectasis (92). Nonnegative tensor factorization has also been shown to have the ability to capture diverse multi-attribute phenotypes consisting of combinations of diagnoses and medications with over 80% found to be clinically meaningful (93).

Unsupervised ML-based approaches may have the potential to scale phenotype classification to thousands of phenotypes in a computable and accurate way. The limitation of unsupervised ML models is their difficulty with interpretability, necessary for wide adoption by clinical providers (94). Further, ML-based algorithms are only as useful as the features used for classification, thus ontologies and advancement of NLP techniques will continue to improve their predictions as well.

5. PHENOTYING FOR GENOMIC RESEARCH

5.1. Phenotype Creation, Validation, and Implementation for Genomic Research

Phenotype creation and validation is often an iterative process and critical step in determining the success and scalability of the phenotype as shown in Figure 4 (30). Phenotype accuracy has been shown in several studies to improve with combinations of data within phenotype algorithms (29, 34, 70). Even simply adding the requirement of 2 or more ICD codes can significantly increase PPV and sensitivity of algorithms; however, inclusion of EHR components, such as medication records and clinical notes, often results in further phenotyping performance improvement (29). Following initial creation and execution of the algorithm at the development site, it is important to validate the results at the primary site before executing at subsequent sites. Validation is typically performed by manual review of a subset of clinical records, and can be achieved by either an “expert review” by a seasoned clinician who reviews the clinical record to determine holistically if the patient meets the case definition or not (e.g., does the patient clinically appear to be diagnosed by appropriate clinicians with the disease of interest) or using formal case review algorithms, which specify finding a number of elements in the chart to verify the case is truly a case. The latter may be particularly important for more precise phenotypes, such as defining resistant hypertension or a subtype of eye disease, or when a clinical expert is not available for chart review.(30) The scope of review can vary based on the phenotype, for example while some algorithms may only require review of 1 year of a clinical record, others may require a full review of patient’s health record. After validation and tuning at one site, then the algorithm is executed and validated at secondary sites, preferably occurring across multiple EHRs and institutions, ensuring its reproducibility and transportability.

Figure 4. Phenotype development and validation.

A primary site first develops and executes the phenotype, and then secondary sites execute the phenotype. At each step feedback to primary and secondary sites may lead to revisions in the methods (arrows).

Curation and validation of a single or few phenotyping algorithms using the above described method to ensure high PPV is feasible and remains a highly-utilized method for GWAS.

5.2. Scalability and Portability of EHR-derived Phenotyping Algorithms

For multi-site collaboration and accrual of large datasets, it is important to consider the portability of phenotypes across different institutions with varying EHRs, infrastructures, clinical domain knowledge, and informatics support. Researchers within eMERGE and other networks have demonstrated the portability of well-defined logical constraint-based phenotyping algorithms and have fostered sharing through public availability on PheKB. Kirby et al. described the results of 43 phenotyping algorithms including multisite validation data in PheKB, with a median of 3 (range 1–8) external validations per algorithm (21). Performances on case and control algorithms for development-site evaluations were similar to performance by external-site evaluations, with median case and control PPV over 95% for both. PheKB currently contains 154 phenotypes, 44 of which are publically available (Table 2). As there is presently no uniformly-adopted method for data representation and extraction across varying institutions and EHRs, the algorithms are typically represented as ‘pseudocode’ to improve transportability and guide other sites at implementation, containing all necessary variables and the rules to combine them (30). Pathak et al. suggested that standardization of phenotype data dictionaries in these phenotyping algorithms using common data elements and biomedical ontologies could also help facilitate multi-site and cross-study collaborations (95); these ideas were later expanded into a desiderata for phenotype algorithms which includes logic formalism, use of ontologies, and the need for a common phenotype language that removes the need for human reimplementation from pseudocode (96).

In 2012, Carroll et al. utilized a previously published logistic regression phenotyping algorithm for identification of cases and controls for rheumatoid arthritis as a model to demonstrate portability of a ML algorithm across institutions (66). Features obtained from structured data were comprised of billing codes, laboratory results, medication orders, and use of general-purpose NLP systems. Applying the previously published regression phenotyping model to 2 external institutions with different EHRs and NLP systems showed an AUC to be similar to that obtained at the development site (92% and 95% at external sites compared to 97% at development site).

Logical constraint-based algorithms are largely rule-based, relying heavily on domain experts for curation. Similarly, supervised ML algorithms require annotated sets and often defined features for a specific phenotype. While these algorithms achieve high accuracy and portability across institutions, their scalability to a phenome-wide approach remains limited. In addition, many traits are not dichotomous but present along a continuum in a population. A recent study by Wells et al. demonstrated the use of left ventricular function determined by systolic ejection fraction, a continuous measure, to analyze associations with drug side effects in a GWAS analysis (97). Other opportunities for defining phenotypes include incorporating ontologies such as SNOMED-CT or the human phenotype ontology (HPO), which was originally designed to capture phenotypes related to Mendelian disease but has grown to encompass an increasing representation of common diseases (98).

As genotyping costs continue to decline and efforts become more widespread, the limiting factor to identification of genotype-phenotype associations will be accurate labeling of phenotype cases and controls. Research questions that require identification of many cohorts or phenotypes can become exponentially more challenging, with the resultant need for high-throughput phenotyping methods. Unsupervised ML algorithms to broadly define phenomes may have potential to scale phenotype discovery; however, their current use in genetic association analyses is limited.

5.3. Phenome-wide Association Studies

Analogous to a GWAS, PheWAS leverages the breadth of phenotypes in the EHR to perform systematic interrogation for associations with an independent variable, typically a genotype. The first PheWAS was performed in 2010, in which four known SNP-disease associations were replicated and several new proposed (99). Since then, dozens of studies have used PheWAS to explore both genetic and phenotypic associations to specific traits.

The PheWAS method requires a broad set of phenotypes collected in an unbiased approach to create a complete phenome of diseases and traits. PheWAS can use thousands of phenotypes; thus manual curation and validation is not practical (99–101). Thus to define a complete phenome across a large cohort, many PheWAS have used phenotypes derived from custom groupings of ICD-9 codes, also referred to as phecodes (99, 102). Typically, 2 or more ICD-9 codes are required for mapping to a single phecode. While the phecode groupings have been shown to better align with clinical diseases in practice, other methods of phenotype classification also are effective for PheWAS studies (100, 101, 103–105). Several studies have reported success with a PheWAS method using raw ICD-9 codes and parent ICD-9 three-digit groups as phenotypes (100, 103, 104). Leader et al. compared five gold standard phenotypes to ICD-9-Clinical Modification (CM) 5-digit and 3-digit diseases and phecodes (103). They found that phecodes may not be granular enough for some phenotypes. Others have also used Agency for Healthcare Research and Quality (ARHQ) Clinical Classification Software for ICD-9-CM (CCS), which reorganizes disparate ICD-9-CM codes into a smaller number of clinically meaningful categories (105, 106). Similar to three-digit ICD-9-CM codes or phecodes, CCS provides a hierarchical grouping of ICD-9-CM or ICD-10-CM codes but at a more aggregate level than other approaches. Wei et al. recently compared different diseases studied in prior GWAS to determine which phenotyping method better aligned with clinical practice and prior genomic association results and found that phecodes replicated more known SNP-phenotype associations (153 SNP-phenotype pairs) than use of ICD-9-CM (143) or CCS (139) (105).

Finding specific phenotypes that are not captured by billing codes or uncommon disease associations is crucial and will require increased granularity and accuracy of phenotyping techniques. An example of a trait rarely captured by billing codes alone is that of drug response, important for pharmacogenomic studies (107). Hebbring et al. has shown that PheWAS can be performed by defining the phenome solely on textual data within clinical documentation (101). Using clinical text extracted using NLP along with billing codes and other data available in the EHR could help automate and refine phenotypes at the phenome-wide scale.

PheWAS has shown the feasibility of analyzing genomic associations with thousands of phenotypes across a cohort of individuals and finding novel associations (99, 100, 102, 108). This approach can also be applied using various predictive attributes in the PheWAS analysis, including genetic risk scores, a set of SNPs aggregated into a single continuous score, or gene expression data. Krapohl et al. demonstrated how a genetic risk score could be applied to a phenome consisting of behavioral or psychiatric traits to explain some of the phenotypic variation seen in a population (109). Gamazon et al. have described PrediXcan, a method that predicts tissue-specific gene expression and could be used in PheWAS to measure trait associations as well (110). Mosley et al. applied generalized linear mixed models in a phenome-wide approach to estimate the additive genetic variance underlying phenotypes to prioritize those diseases and traits more likely to have genetic drivers, identifying a few conditions for which novel genetic signals were discovered via subsequent GWAS (111).

The flexibility and ease of application of the PheWAS approach makes the methods highly generalizable. Expansion of PheWAS-like methods with other high-throughput phenotyping techniques that capture the breadth of phenotypes in clinical practice has the potential to further advance discovery.

6. CHALLENGES AND OPPORTUNITIES OF EHR PHENOTYPES FOR GENOMICS RESEARCH

6.1. Challenges Learned from Current Phenotyping Efforts

Several challenges remain to optimizing use of EHR data. In addition to the inherent limitations of reusing EHR data for research, such as data fragmentation, variable accuracy, and extent of unstructured data, there are significant challenges to phenotype algorithm creation, stability, and scalability. Evolving billing standards, such as the transition to ICD-10, changing documentation practices, and advancing EHR systems all weaken the stability of phenotyping algorithms. EHR-linked biobanks introduce further challenges. In particular, there can be significant loss to follow-up as individuals change providers or institutions. Further, many EHR-linked biobanks prevent the ability to recontact patients due to de-identification of the EHR prior to the conduct of genotyping and research.

One major challenge both within EHR-linked biobanks as well as with combining genomic data from different institutions in multi-site collaborations is the usage of selected patient cohorts genotyped on varying platforms. Often genotyping of all consented individuals at an institution is not financially or logistically feasible. Thus, priorities are made for genotyping of individuals with known phenotypes of specific research interest or external funding for genotyping expenses. This has resulted in relatively large sample sizes of specific phenotypes at any one site, which can also influence what other phenotypes are observed in the genotyped population. In addition to population selection for genotyping, a variety of different genotyping platforms may be used. These constraints result in specific biases within certain biobanks that must be considered by researchers.

Another challenge is variable success of some algorithms. For example, type 2 diabetes has shown replicability with PPV greater than 98% for cases across institutions while the PPV for accurate identification of dementia ranged from 73–90% at different sites (30). For dementia in particular, cases were compared to a research-quality dementia diagnosis at a specific site obtained through a longitudinal cohort study, showing that billing codes and medications alone were not sufficient for accurate prediction of a dementia diagnosis. Unfortunately, there is currently no reliable method of determining preemptively which algorithms will perform well across different sites; thus validation, an iterative and time-consuming process, remains a crucial step.

6.2. Future Directions

Collaborations between researchers and clinicians could allow for integration of data obtained in prospective research initiatives into EHRs. In addition, linkage of EHR data to medical claims data and nationally public databases, such as prescription data and national death indices could also improve accuracy of phenotyping algorithms (30). In contrast to EHRs, which are used primarily by healthcare providers, participant-facing collection of health data has grown dramatically. While not a focus of this review, phenotypes for genomic research obtained through consumer-facing organizations such as 23andMe and the upcoming All of Us Research Program also have the potential to accrue a wealth of phenotypic information for genomic research (24, 112–114). Further, as personal health records and consumer health tracking applications on smartphones or other consumer devices increase in functionality and integration with other aspects of patient health information, these have the potential for the development of more refined phenotype definitions (18, 107–109).

Although use of ML and novel statistical methods for phenotype classification are growing, rule-based approaches continue to dominate. Use of standard terminologies within phenotype algorithms improves portability. Globally, there are challenges with collaboration due to differences in billing standards, such as the use of ICD-9, ICD-10, or ICD-10 with Clinical Modifications in the US. For example, while SNOMED-CT is the most common emerging terminology used in the United States, the United Kingdom has historically used Read codes in general practice clinics.

With no current unified programming language or phenotype implementation approach, the actual code for phenotyping algorithms remains largely non-portable today (118). The Phenotype Execution Modeling Architecture (PhEMA) project has the goal of developing reusable and machine-executable phenotype algorithms across sites and EHR systems (119). A key challenge, however, is not the programming language itself but the semantic representation of the data within the EHR. The Shared Health Research Information Network (SHRINE) has allowed for shared phenotype queries across systems implementing the Integrating Biology and the Bedside (i2b2) platform (120). Another promising approach to harmonizing data from heterogeneous EHRs is the use of a common data model (CDM) to create a common format for the data elements. Several different CDMs are in widespread use, including the Observational Medical Outcomes Partnership (OMOP) CDM (121) and the Patient-centered Clinical Research Network (PCORnet) CDM (122). When transformed into a CDM, ideally the same code can be executed across disparate sites and EHR systems.

The creation of precision medicine initiatives through biobanks allows for discovery into phenotypes across populations of patients with diverse genetic and environmental backgrounds. Genomic discovery has historically lacked in ancestral diversity (123, 124). Research collaboration using these large cohorts will help to better understand the role of genomics in phenotype variation internationally.

The overall goal of integration of genomic and phenotypic information with the EHR is not only to foster advances in research, but also to drive clinical decision making to support precision medicine (125). For example, implementation of algorithms to predict who may be at risk for being placed on particular medications could guide who is genotyped prior to medication therapy (126). Phenotyping algorithms could improve identifying populations for public health measures and decision support.

The National Academy of Medicine has recognized that healthcare systems are falling short it the ability integrate the wealth of knowledge and innovation into improvement in quality, outcomes, and cost (127). Much of the genetic information from genome-wide sequencing or genotyping is either not accessible to clinicians, not actionable at this time, or not in a computable form available for clinical decision support and future research (128). Providing accessibility of genomic information to clinicians through the EHR comes with many challenges, including data storage, structuring, and visualization, and can be overwhelming to a clinician (129). A vital step in achieving a learning healthcare system that incorporates genomic knowledge is the development of genomic clinical decision support for clinicians. The ability to integrate genomic information into the EHR in a way that is accessible, logical, interpretable, and reusable will leverage the translation of genomic discovery to the bedside.

7. CONCLUSIONS

While integration of genomic information into EHRs has yet to fully reach its potential, significant work in phenotyping and biobanking efforts are providing an improved understanding of potential challenges to guide the future. The capability of data collection and extraction in the EHR has made it feasible for genomic and phenomic studies to have significant impacts on modern genomic research. Efforts towards the centralization of information and design of systems that meet the needs of collaborative research along with clinical and billing requirements will greatly facilitate advancement in phenotyping studies. Current successful initiatives worldwide provide the framework for phenotyping and genomic progress and innovation. Continued advances and standardization in EHR data abstraction and visualization, along with structured methods for integrating and representing genomic information will afford generalizable approaches to promote precision medicine.