Standardising terminology in routine electronic health records for respiratory endpoints: why we still can't agree

Introduction

Clinical practice guidelines and healthcare policies increasingly leverage real-world data, often retrieved from electronic healthcare records (EHRs), to optimise disease prevention, diagnosis and management.¹ Traditionally, randomised controlled trials (RCTs) have been regarded as the gold standard for establishing causal relationships between interventions and outcomes, primarily due to their control over confounding factors. However, RCTs face what has been called the clinical trial paradox wherein their strict inclusion criteria and tightly controlled conditions improve internal validity but reduce generalisability to routine care settings.^{2 3} It is well known that clinical trial populations exclude people with complex morbidity.^{4 5} Additionally, different research designs answer different research questions and RCTs are not always fit for purpose to answer many important clinical questions. While strict criteria and conduct of RCTs indeed ensure internal validity on drug efficacy, external validity on drug effectiveness in general practice is often constrained, and ethical considerations sometimes preclude an ability to mimic all real-life conditions. Most RCTs are short in follow-up but also lose randomisation advantages after longer follow-up. Finally, high costs associated with measuring clinically relevant hard outcomes over prolonged timelines under strict conditions have spurred a growing interest in real-world evidence, derived from routine EHRs, registries and other observational data sources.⁶

Unlike efficacy RCTs, observational studies using EHRs can support evaluation of delayed or long-term and rare safety outcomes.⁷ Moreover, studies using observational causal inference methods can provide rigorous evidence when randomised assignment is not feasible or ethical.⁸ New methodologies, including target trial emulation, are one way of addressing causality using observational data.⁹ Real-world data are also used to assess the effectiveness of clinical practice guidelines along different healthcare settings and populations not commonly included in clinical trials.¹⁰ Examining a broader range of clinical settings and treatment effects among larger and more heterogeneous populations supports evidence-based clinical decision-making and public health. EHRs, embedded in the fabric of routine clinical care, offer an efficient, cost-effective avenue for large-scale observational studies that reflect real-world conditions more accurately than many RCTs. Researchers, clinicians and regulatory agencies such as the US Food and Drug Administration (FDA) and European Medicines Agency (EMA) increasingly rely on real-world data from EHRs to conduct rigorous research on drug effectiveness and safety, underscoring their growing utility across clinical, policy and public health domains, with examples including respiratory diseases such as chronic obstructive pulmonary disease (COPD) and asthma.^11–13

Challenges of using EHR to study respiratory diseases/the current landscape

Data generated in real-world settings inherently reflect the complexity of clinical practice, including inconsistent diagnosis setting and coding arising from variation across clinicians, site-specific conventions, temporal changes in coding practices and incomplete health records. In this regard, the major advantage of more representative patient populations and longer follow-up, captured in EHR data, is countered by concerns over data quality, bias and the ability to control for confounding.¹⁴ Biases arise in EHRs for a variety of reasons, including prevalent user or immortal time bias. Countries, moreover, differ in healthcare systems, referral practices and recording procedures at the different levels of healthcare contacts (eg, primary vs specialist care). People may also have varying inclinations to seek care for cultural, social or economic reasons. Therefore, design-related biases need to be avoided and assumptions validated.

Respiratory care is clinically complex, operationally heterogeneous and historically under-standardised. For example, respiratory symptom recording (eg, breathlessness) is often subjective, context-dependent and episodic.^{15 16} Even objective measures demonstrate high degrees of variability both due to quality and to the intrinsic test properties (eg, pulmonary function test data). This means clinicians often default to free text, local shorthand or broad Systematised Nomenclature of Medicine Clinical Terms (SNOMED CT) codes that do not map neatly to research endpoints. Depending on the jurisdiction, there are also implicit financial incentives to use terminology or coding that results in higher compensation. Even when there is broad stakeholder agreement that standardisation is useful for research or quality improvement and for artificial intelligence applications, there can be reluctance to implement or follow suggested ways forward.¹⁷

Coding systems themselves were not built for modern respiratory research. The International Classification of Diseases (ICD), SNOMED CT and current common data models (CDMs) are great initiatives to standardise terminology in health data, to improve data quality, facilitate interoperability and foster primary and secondary analysis. However, adequate adoption by healthcare professionals and coders lags behind due to the potential for ambiguity in concept definitions and practical difficulties in translating the heterogeneous clinical presentation of patients into uniformed codes. The most common systems used, SNOMED CT, ICD-10 and Read Codes, contain overlapping concepts (eg, ‘acute exacerbation of COPD’ vs ‘COPD with acute lower respiratory infection’), lack granularity for physiology-based endpoints (eg, oxygen requirement trajectories) and do not always clearly align with trial-grade definitions (eg, what counts as a ‘COPD exacerbation’). It is not very often straightforward to simply lift the same outcome from a clinical trial and replicate it in EHRs.

The other issue is that EHR systems are designed to optimise clinical workflow, prioritising speed of documentation and clinical utility, or in the case of claims data, billing capture and administrative efficiency rather than the semantic precision required for research. This leads to drop-down lists that hide coding complexity and the introduction of local custom codes and templates that vary by organisation, specialty or even individual clinicians. This ultimately results in a patchwork of semistructured data that is difficult to harmonise. This is also problematic with the expected and growing Open Science culture for databases that should be interpretable between and across clinical research communities for effective collaboration and enhanced productivity and impact. A significant challenge in using EHRs for respiratory research is the lack of standardised clinical definitions and outcomes across different healthcare settings and data settings.¹⁸ Compared with other major disease entities, such as defining ketoacidosis in diabetes or heart attacks in cardiovascular disease, respiratory disease often has no specific clinical measures included in the definitions of key clinical outcomes such as exacerbations. Respiratory endpoints, increasingly collected digitally, as used in trials or observational studies, often require physiological data at prespecified time points, clear thresholds (eg, fraction of inspired oxygen changes) and confirmation of clinical intent (eg, ‘initiated for respiratory failure’).¹⁹ This leads researchers to create derived endpoints/operational definitions, which differ across studies, making consensus even harder. As clinical data are collected in varied databases with different coding practices, terminologies and diagnostic tools and criteria, discrepancies in how outcomes are measured can contribute to inconsistent results across studies, making it more difficult to draw meaningful conclusions or to compare results across different studies.

While standardisation of definitions might benefit all areas of respiratory disease, attempts to standardise disease and exacerbation definitions in COPD, for example, have been difficult, as EHR-based studies are often limited by data availability. Many of the expert consensus-based definitions of poor disease control or exacerbations include patient-reported outcome measures (PROMs), not consistently or routinely captured in coded EHR data.^{20 21} In lieu of PROMs, proxies such as EHR codes for admissions with an acute exacerbation, prescriptions or dispensing of short-acting beta-2-agonists for breathlessness or oral corticosteroids for exacerbations may be used.²² Not all studies, however, include the code lists or timing used to define disease and/or outcomes investigated, challenging reproducibility.²³ In the case of EHR data used for patient care, it is rich in longitudinal information yet often suffers from scattered, incomplete, and often contradictory documentation, adding further complexities to using patient care data longitudinally in epidemiological research.²⁴ Finally, given the plethora of EHR systems available, even within individual countries, interoperability and data pooling often require standardised mapping for reliable cross-system research, making CDMs of value.^{25 26} However, in respiratory disease, standardised code lists are often lacking, not used, not demanded by editors/reviewers, or not transparently reported. This is mainly a concern for studies where databases used are not mapped to a CDM, which, although not the solution for everything, is a first step into interoperability. Importantly, this heterogeneity in definitions and lack of standardisation may lead to different or even contrasting answers to the same research question, depending on definitions chosen or data available.¹⁵ The impact of differential disease and outcome definitions is highlighted by large discrepancies in estimated incidence and prevalence across studies in COPD and asthma, with estimates depending heavily on definitions used (eg, clinician-reported and diagnosed asthma versus patient-reported clinician-diagnosed and treated asthma).²⁷ In another example, a high variation in COPD exacerbation frequency was seen, depending on different definitions used, and this impacted the results related to exacerbation outcomes comparing people with COPD with and without heart failure risk.¹⁵ The difficulty is that incentives for standardisation are weak.

Solution

Unlike other disease areas such as cancer or cardiovascular disease, respiratory care lacks strong national registries, mandated minimum datasets, consistent commissioning requirements and a unified professional push for standardisation across Europe. Therefore, to unlock its full potential of generating supportive evidence at low cost, a critical step is to improve consistency and reliability across EHRs, particularly through the standardisation of clinical definitions and outcomes. As a community, we need to have agreement on definitions for diseases and outcomes that we are going to use (ie, the phenotype definitions) and, from that, decide which data sources are best used to answer a specific research question. We will unlikely have a situation where every GP or specialist records the data based on the clinical definition we agreed on, and we cannot aim for full consistency—reliability across data sources, but we can aim for consistency—reliability on definitions across studies and researchers.

Herein, we propose five key principles for the development of standard definitions in EHR-derived endpoints (figure 1).

Figure 1. Summary of STANDOUT. ¹Stakeholders include clinicians, trialists, data scientists, epidemiologists, patients/people with lived experience and regulatory bodies (eg, the European Medicines Agency, the US Food and Drug Administration, the National Institute for Clinical Excellence and the Medicines and Healthcare products Regulatory Agency); stakeholder mapping ensures representative voices are included in all discussions. ²Definitions must work across health systems (national coding practices and local software), disciplines (primary care, pharmacy and specialist care) and patient populations; where definitions differ by setting, transparent reporting of methodology is essential. ³Real-world settings refer to EHRs and other data sources from routine clinical care; feasibility relates to whether patients and outcomes of interest can be identified within available data sources (fit-for-purpose). ⁴In line with findability, accessibility, interoperability and reusability principles, code scripts should be freely available in open-source repositories (eg, GitHub), ideally in multiple programming languages. ⁵The evolving clinical landscape refers to shifts in best evidence, clinical practice and policy; definitions should be versioned, validated with each update and revised as infrequently as possible to maintain comparability across studies. STANDOUT, STANDardisation of respiratory Definitions and OUTcomes.

Relevance

First, it is essential that a wide variety of stakeholders, including clinicians, trialists, data scientists and epidemiologists, are consulted during the development process to ensure that the finalised definitions are clinically relevant; practical to implement, evidence-driven, flexible with regard to anticipated policy and practice changes and meaningful to those with lived experience of the conditions. As such, stakeholder mapping—that is, understanding who the stakeholders next to patients and healthcare providers are, for example, EMA, FDA, National Institute for Clinical Excellence and Medicines and Healthcare products Regulatory Agency—is a crucial early step in planning and it must be ensured that representative voices are included in all discussions. Unless this is implemented, variations of definitions are likely to be required for different individual use cases and user groups.

Consistency

It is required that definition sets can be consistently extracted in a variety of settings. As such, they must be as inclusive as possible across health systems (including national coding practices and local software systems), disciplines (such as pharmacy, primary care and specialist care) and patient populations. Including in the consensus exercise, stakeholders with experience in working within these domains, is of great benefit to this end. Terms must also be clearly labelled with regard to their intended use, such as demarking between direct outcomes or surrogate outcomes. This prevents multiple concepts from being lumped into one term and ensures that different measures are labelled according to the concepts they represent. We are aware of the complexity of this task given that the way healthcare is recorded and delivered varies by country and that the same amount of detail may not be available everywhere at every level. There are clearly instances where different definitions may be required in different settings, making transparency of reporting methodology even more important.

Feasibility

Comprehensive data dictionaries for each data source are essential to facilitate the mapping of corresponding elements of health system and discipline processes. However, user experience is still required to highlight potential hurdles associated with how systems are pragmatically used compared with their intended design. For example, some data entry fields are sparsely completed. Feasibility relates to whether it is possible to identify, within the data sources that we have available for our respiratory epidemiological research, those patients and outcomes that are important. This is an issue that is mainly encountered when for instance, we want to replicate/emulate RCTs and experience the boundaries of our EHRs. In current work for EMA, a lot of activity is spent on assessing whether the data sources that are accessible are ‘fit for purpose’. The ‘fitness-for-use’ assessment of a data source should also be considered as part of the data quality framework.²⁸ The other issue is that while the importance of initiatives such as this is recognised, they do not attract sustained funding easily, which is paramount in maintaining definitions and ensuring consistency, reproducibility and adaptability.

Reproducibility

In line with FAIR principles, definitions must be transparent and reproducible. This facilitates ongoing usage by all parties and allows the strengths and weaknesses of operational definitions to be examined towards future refinement. Any code scripts which are used to derive endpoints should be made freely available in an open-source repository, for example, GitHub and if possible, provided in multiple programming languages to promote their uptake.

Adaptability

Finally, it is required that updated definitions be anticipated and versions be clearly labelled and recorded accordingly. Validation and calibration are essential with each iteration to ensure reliability. This allows shifts in best evidence and clinical practice (including policy changes) to be accommodated into definitions and ensure they remain relevant. However, it is also important that definitions are updated as infrequently as possible to mitigate excess versions which undermine the value of standardisation when comparing results between studies.

To help put these principles into action and bring the community together, STANDardisation of respiratory Definitions and OUTcomes (STANDOUT) (figure 1) a newly adopted European Respiratory Society Clinical Research Collaboration, has been developed. STANDOUT aims to establish standardised definitions for respiratory endpoints in Real-World Data (RWD) for use across observational studies and clinical trials and to improve the mapping of respiratory RWD to CDMs in all age groups.²⁹ STANDOUT has funding for 3 years in the first instance, with the aim of financial and collaborator expansion, ensuring stakeholders are representative. Our steering group will help ensure this. While our current network consists predominantly of researchers from Europe and North America, we would encourage individuals from the Global South to reach out, given these are areas where EHRs are being implemented and/or large-scale EHR data is increasingly being used. We look forward to facilitating a united effort to improve the quality and use of EHRs in the respiratory field and are delighted to hear from anyone interested in being involved.