Development of phenotype algorithms for the detection of adverse events in electronic health record data: a multicentre study

Abstract

Objectives

To develop phenotype algorithms for the detection of adverse events (AEs) or AE-proxies in electronic health records (EHRs), accounting for varying data availability.

Design

Multicentre study conducted as part of the Use Case POLAR_MI (POLypharmacy, drug interActions, Risks) of the German Medical Informatics Initiative (MII).

Setting

Germany.

Participants

Multidisciplinary teams from 10 German university sites within the MII.

Interventions

Not applicable.

Main outcome measures

Literature- and consensus-based development and operationalisation of AE algorithms using structured EHR data, including a standardised, multicentre expert review process. Data categories used: International Classification of Diseases, 10th Revision (ICD-10) codes for diagnoses; Anatomical Therapeutic Chemical (ATC) codes and ‘Pharmazentralnummern’ (PZN; German eight-digit identification code for pharmaceutical products) for medications (used in the treatment of AEs); Logical Observation Identifiers Names and Codes (LOINC) for laboratory values and medical findings; and ‘Operationen- und Prozedurenschlüssel’ (OPS; German procedure classification) codes for medical and surgical procedures.

Results

We developed 82 algorithms for 48 AEs. Algorithms for the same AE varied by data categories or code selections. At the AE level, 31 AEs were covered exclusively by newly developed algorithms, and 17 AEs by at least one modified algorithm.

Overall, 52 algorithms were based on a single data category, while 30 required multiple categories. ICD-10 codes were most commonly used (n=65 AE algorithms), followed by LOINC (n=27), ATC codes (n=18), OPS codes (n=11) and PZN (n=2). All phenotype algorithms were semantically modelled and can be executed using the publicly available Terminology- and Ontology-based Phenotyping (TOP) Framework, which supports export in various formats.

Conclusion

We present a peer-reviewed set of algorithms for a large number of AEs, which can be implemented in structured routine electronic data sources and (pending validation studies) may support pharmacoepidemiologic research. The algorithms will be implementable across all 39 participating sites of the German MII. As a next step, we will empirically validate the algorithms against all information (including free text) contained in EHRs.

Trial registration

Not applicable.

STRENGTHS AND LIMITATIONS OF THIS STUDY:

• The development of phenotype algorithms followed a rigorous, multicentre methodological approach incorporating a structured literature review and independent multidisciplinary expert review.

• For most adverse events, multiple algorithms were developed using different combinations of data categories to account for heterogeneity in data availability across sites.

• Algorithms were operationalised using standardised terminologies (International Classification of Diseases, 10th Revision (ICD-10), Anatomical Therapeutic Chemical (ATC) classification system, Logical Observation Identifiers Names and Codes (LOINC), Operationen- und Prozedurenschlüssel (OPS)) to enable consistent implementation in structured electronic health record data.

• All algorithms were modelled within the Terminology- and Ontology-based Phenotyping Framework supporting execution across heterogeneous source systems and export in multiple formats.

• Empirical validation of the phenotype algorithms against a gold standard is still pending.

Background

Regulatory and clinical decision making on medicines requires valid data on their desired and undesired effects. Although randomised controlled trials (RCTs) serve as the gold standard for assessing the efficacy and risks of medicines, they typically only provide average effect estimates in narrow and homogeneous segments of the population.¹ Especially older people with multimorbidity and polypharmacy, who are the main consumers of medicines, are commonly underrepresented in RCTs.² Observational pharmacoepidemiologic studies can be highly valuable for answering questions on medication effectiveness and safety that RCTs cannot address due to either practical or ethical constraints.^{3 4}

A key issue in quantifying drug effects is the detection of adverse events (AEs) as clinical endpoints, where misclassification may lead to biased risk estimates. Pharmacoepidemiologic studies most frequently rely on electronic databases containing structured data, such as claims or electronic health record (EHR) data.

In observational studies using claims or administrative data only, clinical endpoints are most commonly defined using International Classification of Diseases (ICD) codes. However, underdocumentation is a common problem,^{5 6} potentially resulting in underestimation of disease incidence and absolute risk estimates. For example, in one study of community-acquired acute kidney injury (AKI) associated with the use of non-steroidal anti-inflammatory drugs, only 43.4% of patients with laboratory-defined AKI had an explicitly documented AKI diagnosis.⁷ In other cases (such as falls), codes are missing in the German version of the ICD-system or are hardly used, implying the need for AE-proxies. Therefore, to increase the sensitivity and/or specificity of endpoint detection, other data categories typically available in EHRs, such as results of clinical or laboratory investigations, may sometimes be valuable.⁸

Several phenotype libraries and repositories have been developed to support EHR research, such as the UK Health Data Research (HDR) Phenotype Library⁹ and the US Phenotype KnowledgeBase (PheKB),¹⁰ which provide phenotype definitions for disease-based cohort identification using structured diagnosis and procedure codes. However, only a subset of clinically relevant AEs is currently represented, and alternative phenotype definitions that accommodate heterogeneous data availability at different sites remain limited.

The German Medical Informatics Initiative (MII) aims to develop infrastructure and tools to facilitate observational research using EHRs. This work was conducted as part of the POLypharmacy, Drug interActions and Risks (POLAR_MI) project, a use case of the MII. An overview of the POLAR_MI project is available via the project website,¹¹ and detailed information on the project has been published previously.¹² The focus of this work was on detecting AEs from inpatient EHR data, irrespective of prior drug exposure. Since data availability may differ across sites, it was intended to provide (whenever reasonable) multiple detection algorithms based on different data categories to allow for adaptation to each site’s circumstances and to potentially increase sensitivity. Although this work is motivated by drug safety research, the present study deliberately focuses on AEs, regardless of confirmed causality. The algorithms can serve as a basis for subsequent analyses linking AEs to specific exposures, including medications. The specific objectives were (i) to systematically develop a set of algorithms for clinically relevant AEs or AE-proxies using one or more data categories and (ii) to make them publicly available through a repository.

Methods

Study design

The AEs to be operationalised were based on a previously conducted expert consensus process using a modified RAND Corporation/University of California, Los Angeles (RAND/UCLA) appropriateness method.^{13 14} Each AE was operationalised by multidisciplinary teams from at least two of 10 participating German university sites. For each AE, an appointed lead centre first identified any previously published algorithms via structured literature search and either adopted or modified them or (if not available) designed new algorithms using data categories available in EHRs. A second centre was responsible for review and revision. In the last step, the algorithms were made available to all participating centres via SharePoint with a request for commentary, and were finalised based on the feedback received.

Selection of AEs

An AE was defined as any abnormal sign, symptom, laboratory test, syndromic combination of such abnormalities, untoward or unplanned occurrence (e.g., an accident or unplanned pregnancy) or unexpected deterioration in a concurrent illness.¹⁵ As part of this study, AEs were operationalised as clinically relevant conditions or deteriorations that are identifiable in structured EHR data, regardless of confirmed causality or a specific triggering exposure. The selection of AEs was based on the above-mentioned RAND survey, in which two expert panels of medical doctors and pharmacists assessed a list of potentially drug-related events (‘candidate ADEs’), in terms of their overall importance for medication safety.¹³ Across both expert panels, a total of 65 distinct candidate ADEs were assessed, of which 42 were prioritised by at least one panel based on a median importance rating of 4 (‘very important’) or 3 (‘important’) without disagreement. A further six AEs were additionally considered important by our research group, so that a total of 48 AEs were selected for operationalisation.

Development of algorithms

The multidisciplinary expert teams included clinical pharmacists, clinical pharmacologists, medical specialists and laboratory physicians involved in the POLAR_MI project.

For each AE, the lead centre conducted a structured literature search in PubMed/MEDLINE to identify previously published algorithms for AE detection. Searches combined AE-specific terms (including synonyms) with keywords related to phenotyping and algorithm development (e.g., ‘phenotype’, ‘algorithm’, ‘electronic health records’) and were limited to studies published in English or German. An example search strategy for the AE ‘thrombocytopenia’ is provided in the online supplemental file 1.

Based on the literature review, algorithms were adapted from existing definitions or developed de novo if none were available. To ensure a standardised procedure across centres, a structured document template was used comprising summaries of existing algorithms, documented adaptations and a flow chart linking relevant data categories. The included gateway decision points were adapted from Business Process Model and Notation, the global standard for process modelling. An example of the document template with a detailed description of one of the algorithms (for the AE ‘thrombocytopenia’) is provided in the online supplemental file 1.

For operationalisation of medical diagnoses, the ICD, 10th revision, German Modification (ICD-10-GM), version 2023, was used.¹⁶

Medication data were specified using Anatomical Therapeutic Chemical (ATC) codes (German version 2023).¹⁷ In rare cases, the German PZN (‘Pharmazentralnummer’) was applied, which is a standardised identification code consisting of an eight-digit number that uniquely identifies pharmaceutical products by name, dosage form, strength and package size. Including the PZN instead of ATC codes was therefore required, if such additional information was relevant, for example, furosemide for intravenous use in the treatment of decompensated heart failure. Only drugs used to treat the respective AE were included (not those that potentially cause it).

Laboratory values were classified by Logical Observation Identifiers Names and Codes (LOINC) (V.2.75),¹⁸ and for laboratory-based AEs, Common Terminology Criteria for Adverse Events (CTCAE) severity grades were specified where applicable.¹⁹

Medical, surgical and diagnostic services and procedures were identified through ‘Operationen- und Prozedurenschlüssel’ (OPS; German procedure classification) codes (version 2023). OPS is an adaptation of the English-language International Classification of Procedures in Medicine of the WHO.²⁰

For most AEs, a single-category algorithm was created initially, typically using ICD-10 codes or laboratory values. In order to account for heterogeneous data availability across sites, at least one alternative algorithm was developed encompassing other data categories where possible. In some cases, alternative algorithms were specified to prioritise sensitivity and/or specificity.

Peer review process

Following the operationalisation of an AE by the lead centre, a standardised two-stage review process was conducted in order to improve the clinical plausibility and cross-site consistency of the algorithms. In the first stage, a review was performed by the second centre and discussed with the lead centre. A laboratory physician was involved in the operationalisation of laboratory value-based algorithms. In some cases, external specialists (not involved in the POLAR_MI project) were consulted as a third instance if ambiguities could not be clarified in the discussion between the two centres, for example, a cardiologist for the AE ‘bradycardia’ or a dermatologist for the AE ‘allergic skin reactions’. Modifications and improvements resulting from the discussion were implemented in the algorithm. In the second stage, all centres involved in the process of AE operationalisation were invited to comment on the algorithms.

Algorithm integration into the Terminology- and Ontology-based Phenotyping Framework

All final phenotype algorithms were modelled in the ontology-based TOP Framework,²¹,²³ which supports the formal representation, execution and reuse of phenotypic knowledge.^{24 25}

In the TOP Framework, phenotype models define relevant phenotype classes (e.g., ‘number of thrombocytes’, ‘diagnosis of thrombocytopenia’, ‘administration of platelet concentrate’) together with their terminology codes (e.g., ICD, OPS, ATC or LOINC), formulas (e.g., for identifying patients with thrombocytopenia based on several parameters) and value ranges (e.g., number of thrombocytes<lower limit of normal). These models enable the execution of generic phenotype algorithms across different data sources without requiring site-specific query logic.

The semantically modelled phenotype algorithms can be executed on a variety of source systems using medical terminologies and appropriate adapters (e.g., Fast Healthcare Interoperability Resources (FHIR) Search or SQL-based systems). Phenotype models are stored centrally and can be exported in different formats (e.g., JSON, Ontology Web Language, CSV or Decision Model and Notation).

All phenotype models resulting from this study are publicly accessible via the POLAR_MI repository within the TOP Framework.

Patient and public involvement

Patients or the public were not involved in the design, conduct, reporting or dissemination plans of our research. This study focused on the methodological development and expert review of phenotype algorithms using standardised medical terminologies and did not involve patient-level data, patient-reported outcomes or direct interaction with patients. Therefore, patient and public involvement was not considered essential to achieve the study objectives.

Results

Characterisation of developed phenotype algorithms

Table 1 shows an overview of all algorithms created for the selected AEs and the data categories included. For 48 AEs, a total of 82 algorithms (numbered 1, 2, etc.) were developed, of which 21 were modified versions of previously published algorithms and 61 were new developments. In terms of AEs, 17 of the 48 AEs were related to at least one modified algorithm based on existing definitions, while 31 AEs were covered exclusively by newly developed algorithms. For eight AEs, both modified and newly developed algorithms were provided. Algorithms for the same AE differed either in the type or number of data categories used (52 single-category and 30 multiple-category algorithms), or in the code selection within the same data category (for 16 AEs, named A, B and, if applicable, C), for example, different LOINC regarding the type of laboratory parameter or the specimen used (depending on data availability).

Table 1. Overview of AEs and data categories required per algorithm.

AE	Number of algorithms	Data categories required					Origin
		Coded diagnoses and procedures		Medication		Lab results and medical findings
		ICD-10	OPS	ATC	PZN	LOINC
Cardiovascular
Decompensated heart failure	1 (A, B)	X					M8 36 37
	2	X		X	X	X	M38 39
Tachycardia—ventricular	1	X					M40,42
	2	X				X	N
Bradycardia	1	X					M42
	2	X				X	N
Hypertensive crisis	1	X					N
	2	X		X		X	N
Hypotension	1	X					N
	2	X				X	N
Syncope and collapse	1	X					M43 44
	2	X	X				N
Haematological
Bleeding outside the GIT	1	X					M26 27
Anaemia	1					X	N
	2 (A, B)	X		X			N
Thrombocytopenia	1					X	N
	2	X	X	X			N
Agranulocytosis and neutropenia	1					X	M45
	2	X					N
Gastrointestinal
Bleeding/perforations of the upper GIT	1	X					M26 27
GI ulcers without bleeding/perforation	1	X					M46 47
Pseudomembranous colitis	1	X					M48,50
	2	X				X	M48 50
	3	X		X			N
	4	X				X	N
Acute pancreatitis	1	X				X	N
Diarrhoea (excl. gastroenteritis, colitis)	1	X					N
	2	(X)		X			N
Intestinal obstruction (ileus)	1	X					N
Neurological
Seizure	1	X					M51,53
Extrapyramidal disorders	1 (A, B)	X					N
	2	X		X			N
Delirium	1 (A, B)	X					M54
	2	X	X	X		X	M55 56
Dizziness	1 (A, B)	X					N
Hallucinations	1 (A, B)	X					N
Serotonin syndrome	1	X					N
	2	X		X			N
Somnolence	1	X					N
	2			X			N
	3	X		X			N
Sensory
Toxic damage to the inner ear  (excl. vertigo)	1	X					N
Respiratory
Interstitial lung diseases	1 (A, B)	X					N
Respiratory depression	1	X					N
	2	X	X				N
	3	X		X			N
Musculoskeletal
Fall (injuries)	1	X					N
	2	X	X				N
Acute gout attack	1	X					N
	2		X				N
	3	X	X	X			N
Rhabdomyolysis	1 (A, B)	X				X	N
Myopathy (without rhabdomyolysis)	1	X					N
	2	X				X	M57,59
Metabolic
Uncontrolled hyperglycaemia	1					X	N
	2	X					N
Hypoglycaemia	1					X	N
	2	X					N
	3	X		X			N
Hormonal
Disorders of the adrenal cortex	1 (A, B)	X					N
	2					X	N
	3			X		X	N
Electrolyte disorders
Exsiccosis/dehydration	1	X					N
Hypernatraemia	1 (A, B)					X	N
Hyponatraemia	1 (A, B)					X	N
Lactate acidosis	1 (A, B)					X	N
	2	X					N
Hyperkalaemia	1 (A, B, C)					X	N
Hypokalaemia	1 (A, B, C)					X	N
Hypercalcaemia	1 (A, B)					X	N
Renal
Acute kidney injury	1	X					N
	2	X	X			X	M60
Urinary retention	1	X					M61 62
	2	X	X				M63
Hepatic
Liver damage	1	X	X	X		X	M64
Immunological
Stevens-Johnson syndrome (SJS)/  toxic epidermal necrolysis (TEN)	1	X					N
Allergic skin reactions	1	X					M65,67
Anaphylactic shock	1	X					M68
	2	X	X	X	X		N
Angioedema	1	X					N
	2	X		X			N
General
Inflammation	1 (A, B)					X	N

A, B, and C indicate alternative algorithm variants for the same adverse event based on different code selections within the same data category.

AE, adverse event; ATC, Anatomical Therapeutic Chemical (Classification); GI(T), gastrointestinal (tract); ICD-10, International Classification of Diseases, 10th Revision; LOINC, Logical Observation Identifiers Names and Codes; M, modified algorithm based on previously published definitions; N, newly developed algorithm; OPS, Operationen- und Prozedurenschlüssel; PZN, Pharmazentralnummer.

When using alternative code selections or data categories, differences in sensitivity and specificity are generally to be expected. For selected AEs, we have explicitly provided additional algorithms that reflect different assumptions regarding sensitivity, specificity or severity based on the code selection (e.g., ICD-10 alone vs ICD-10 combined with ATC codes or laboratory toxin tests for pseudomembranous colitis).

ICD-10 codes featured in the majority of algorithms (n=65; 79%) and were the most frequently used data category, followed by LOINC (n=27; 33%—laboratory values: n=22; 27%, medical findings: n=5; 6%), ATC codes (n=18; 22%), OPS codes (n=11; 13%) and PZN (n=2; 2%).

As a result of the structured peer review process, most phenotype algorithms were revised at least once prior to finalisation, primarily involving refinements to terminology code selection and the inclusion or exclusion of specific data categories.

In most cases, one algorithm was created that requires only one data category (usually ICD-10 codes, or LOINC in the case of laboratory value deviations). The other data categories were typically used as part of more complex (multiple-category) algorithms. The following sections provide examples of single- and multiple-category algorithms and their rationale.

Single-category algorithms

The AE ‘bleeding/perforations of the upper gastrointestinal tract (GIT)’ represents a single-category algorithm, as ICD-10 codes alone showed sufficient sensitivity and specificity in prior studies, while adding procedure codes did not improve performance.^{26 27}

Electrolyte disorders, such as hyperkalaemia or hyponatraemia, represent another group of single-category algorithms, for which LOINC-coded laboratory values constitute the most reliable source. Depending on data availability, different specimen definitions were considered.

Figure 1 shows two algorithms for identifying the AE ‘thrombocytopenia’. The primary algorithm is based on a single data category, using LOINC-coded platelet counts with severity thresholds aligned to CTCAE grades. If no laboratory values are available, an alternative algorithm was provided that relies on ICD-10 codes and surrogate indicators of severe thrombocytopenia, such as the administration of platelet concentrates identified via ATC or OPS codes.

Figure 1. Algorithms (1: single-category, 2: multiple-category) for the adverse event ‘thrombocytopenia’. ATC, Anatomical Therapeutic Chemical; ICD-10, International Classification of Diseases, 10th Revision; LLN, lower limit of normal; OPS, Operationen und Prozedurenschlüssel.

Multiple-category algorithms

However, there are also AEs that require more complex detection algorithms. One example is ‘falls’, for which there are no specific ICD-10 codes available in the ICD-10-GM and therefore no validation studies from the literature that could have been used for algorithm development (in contrast to ‘bleeding/perforations of the upper GIT’ (see Single-category algorithms), for example). Nevertheless, fall injuries were used as proxies, stratified by their likelihood of fall-related aetiology based on published positive predictive values (PPVs).²⁸ Where only ICD-10 codes with lower probability of a fall aetiology are identified, the algorithms consider additional risk factors (e.g., age, underlying diseases or procedures) to support case identification.

Liver damage is another example for a more complex algorithm including four different data categories (see figure 2): ICD-10 codes, (hepatic) laboratory parameters and selected procedure or medication codes. Laboratory thresholds were based on the Council for International Organizations of Medical Sciences (CIOMS) criteria,²⁹ with adaptations reflecting current consensus and data availability.³⁰ Depending on analytic priorities, alternative parameter thresholds allow prioritisation of sensitivity or specificity. A detailed description is provided in the online supplemental file 2.

Figure 2. Complex algorithm for the detection of ‘liver damage’ based on the CIOMS criteria. *deviating from CIOMS 1990 (conjugated bilirubin). ALT, alanine transaminase; AP, alkaline phosphatase; AST, aspartate transaminase; ATC, Anatomical Therapeutic Chemical (Classification); CIOMS, Council for International Organizations of Medical Sciences (criteria); ICD-10, International Classification of Diseases, 10th Revision; INR, International Normalised Ratio; OPS, Operationen und Prozedurenschlüssel; ULN, upper limit of normal.

Further details on the rule generation for algorithm development and the selection of threshold values (including literature references) are provided in the ‘Descriptions’ section within the TOP Framework for each algorithm.

Algorithm modelling using the TOP framework

All algorithms were modelled using the TOP Framework and can therefore be executed on a variety of source systems and exported in different formats.

Figure 3 shows the phenotype model for the AE ‘liver damage’, including all relevant phenotype classes with defined codes and laboratory cut-off values. The definition of ‘Algorithm 1’ is displayed as a logical formula referencing other phenotype classes. In addition, a description of the algorithm with any relevant literature is provided. The modelling process is described in detail in²² using the AE ‘delirium’ as an example.

Figure 3. TOP Framework screenshot showing the repository for the AE ‘liver damage’. ALT, alanine transaminase; AP, alkaline phosphatase; AST, aspartate transaminase; ICD-10, International Classification of Diseases, 10th Revision; INR, International Normalised Ratio; OPS, Operationen und Prozedurenschlüssel; TOP, Terminology- and Ontology-based Phenotyping; ULN, upper limit of normal.

The complete set of the POLAR_MI phenotype algorithms/models is publicly accessible at: https://top.imise.uni-leipzig.de/polar.

Discussion

This study has yielded 82 peer-reviewed phenotype algorithms to detect 48 AEs with high relevance to medication safety. The algorithms are intended to be applied in structured data sources typically available in inpatient EHR data. For most AEs, we present more than one algorithm in order to account for variable data availability across different sites. Of all algorithms, 52 were based on a single data category (most commonly ICD-10 codes and LOINC), while 30 required two or more data categories. ICD-10 codes were included in the majority of algorithms, followed by LOINC, ATC codes, OPS codes and PZN.

Some AEs, such as thrombocytopenia or liver damage, may conceptually overlap with disease entities. In this study, AEs were used as an outcome-oriented concept rather than a disease classification, reflecting a pragmatic focus on clinically relevant safety outcomes and not an attempt to categorise all newly occurring diseases. Although the temporal onset of events was not explicitly modelled, care was taken during operationalisation to distinguish AEs from chronic or clearly defined disease entities, for example, by excluding diagnosis codes representing qualitative platelet disorders or non-thrombocytopenic purpura in the thrombocytopenia phenotype.

Of the 82 phenotype algorithms developed, 21 were based on previously published definitions and were adapted for use in the present study. The modifications did not generally change the clinical concept of the AE, but were necessary to ensure compatibility with the available structured EHR data and the German coding systems used at the participating sites. Common modifications included refining or replacing terminology codes, including or excluding specific data categories and aligning decision logic with routinely available inpatient data. In several cases, existing algorithms were extended to provide alternative operationalisations reflecting different assumptions regarding sensitivity or specificity. All modified algorithms were reviewed by multidisciplinary teams of experts to ensure their clinical plausibility and consistency before being integrated into the TOP framework.²² The POLAR_MI repository containing all modelled phenotype algorithms is publicly accessible through the TOP Framework.

Comparison to other phenotype repositories

To the best of our knowledge, this is the first study in Germany to provide a repository of 82 semantically modelled phenotype algorithms for implementation in EHR data. As far as we know, there are a few other platforms that provide phenotype algorithms,³¹ but the overlaps are very small. The HDR UK Phenotype Library contains definitions for hundreds of diseases from structured and unstructured data sources.⁹ Phenotypes and collections from numerous contributing organisations across the UK were integrated into the Library, such as the ClinicalCodes repository from the University of Manchester, which is accessible online and allows researchers using EHR data to upload and download lists of clinical codes.³² The algorithms provided in the HDR UK Phenotype Library primarily consist of ICD codes, Read Codes (a clinical terminology system used in general practice in the UK) and SNOMED CT codes (Systematised Nomenclature of Medicine – Clinical Terms). Although the Library already includes a large number of algorithms, only 17 of our 48 AEs are also available there.

The US PheKB website provides validated phenotype algorithms and offers the opportunity to share validated algorithms.¹⁰ However, only four of our AEs (pseudomembranous colitis, liver damage, heart failure and seizures) overlap with algorithms on the PheKB website.

Strengths and limitations

Our study has several strengths. The minimal overlap between the AE algorithms developed here with those published in existing UK and US sources demonstrates that our work addresses a critical gap in medication safety research. For example, for the AE angioedema and extrapyramidal disorders, we have found no algorithms in the literature so far. A further strength is that for most AEs, we provide two or more algorithms that vary in terms of the number and type of data categories used. This provides future users with different options depending on data availability and the relative emphasis placed on sensitivity or specificity in a given study. Moreover, these algorithms can be adapted to different data sources, such as EHR or claims data. The rigorous development process, which included structured literature reviews and independent peer reviews by clinical experts, further enhances the robustness of this work.

However, there are also some limitations. Currently, the phenotype algorithms rely solely on structured data sources like ICD, ATC and LOINC codes, whereas some information for detecting AEs in EHRs may only be found in unstructured data, such as doctors’ letters or clinical notes. Integrating unstructured data using natural language processing techniques will be crucial in the future. Regular updates to code lists of ICD, ATC and OPS codes are also necessary to align with annual revisions and to enable grading the severity of AEs, which is currently limited to AEs that rely on laboratory data. The use of OPS and PZN, specific to Germany, limits generalisability; however, OPS is comparable to international coding systems like the Current Procedural Terminology, and can be mapped accordingly, while PZNs were infrequently used.

Another limitation is that the phenotype algorithms do not explicitly model the temporal onset or transition from a disease-free to a disease state. The algorithms identify the presence of AEs based on structured EHR data, but do not distinguish between new events and pre-existing conditions. This limitation reflects the heterogeneity of timestamp availability and data completeness across various EHR systems and was accepted to ensure broad applicability of the algorithms across different sites. Furthermore, it is still essential to validate the developed algorithms against a gold standard in order to examine their performance (i.e., sensitivity, specificity, PPV and negative predictive value), including a comparison of different algorithms for the same AE. Such validation may be particularly relevant in situations where the algorithms can only detect AE-proxies (such as certain fractures and injuries as proxies for falls). As a next step, we will empirically validate the algorithms against manual chart review as a gold standard, using all available data (including free text) contained in EHR.

Implications for research and practice

The algorithms developed here may be used as independent or dependent variables in observational studies and may be particularly suited for pharmacoepidemiologic studies using inpatient EHRs pending further validation. Their implementation in EHR systems could enhance the detection of AEs that might otherwise be overlooked (e.g., delirium, inflammation, liver damage and AKI) and thus contribute to medication safety. Future efforts should focus on continuously improving the detectability of AEs in routine clinical data. A key advancement will be the implementation of ICD-11, as demonstrated by Andrikyan et al., who highlighted that ICD-11 allows for more detailed AE coding compared with earlier versions, offering a promising step forward in improving AE documentation and patient safety.³³ Although the phenotype algorithms do not include explicit temporal modelling, the temporal context can be taken into account in the study design depending on the research question. For example, AEs during hospital admission can be identified based on admission diagnoses, while events occurring during hospitalisation can be detected by limiting analyses to diagnoses, procedures or laboratory values recorded after admission. This flexible approach makes it possible to use the same phenotype algorithms either as baseline characteristics or as outcome variables without embedding rigid temporal assumptions in the algorithms themselves.

Although this project focuses on the development of AE algorithms irrespective of prior drug exposure, the algorithms can be combined with medication exposure data (e.g., ATC codes) to support the identification of potential adverse drug reactions in subsequent analyses. Such drug-event pair approaches have been described previously and applied in large-scale routine data settings.^{34 35}

Beyond the current predefined set of AEs, the underlying ontological approach and the TOP framework are not limited to these entities. The POLAR_MI repository is intended to be a living resource that can be continuously expanded or adapted as new research questions, data sources, or clinical priorities emerge, supporting scalability and long-term applicability.

Conclusion

In collaboration with numerous experts in Germany, we have developed phenotype algorithms for a large number of AEs with high relevance to medication safety, which can be implemented in routine electronic data sources and (pending validation) serve as a basis for further database-tailored medication safety research. As a next step, we will empirically validate the algorithms against all information (including free text) contained in EHR data. The POLAR_MI repository, containing all the phenotype algorithms, is publicly accessible through the TOP Framework, enabling implementation across all 39 sites participating in the German MII.

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study.