Development of an Artificial Intelligence Powered Medication Risk Score Calculator Application

ABSTRACT

The publication explores the development of the Augmented Medication Risk Score (AUGMERIS) calculator, a web application supported by artificial intelligence, designed to automate the evaluation of medication therapies with the Danish Medication Risk Score (MERIS) method. It is a tool that assesses drug combinations and kidney function in estimated glomerular filtration rate (eGFR), which helps clinical pharmacists identify high‐risk patients. To overcome the problem of processing unstructured electronic health records (EHRs), a hybrid text processing model was created by combining rigorous algorithms and Generative Pre‐trained Transformer (GPT) technology, which was integrated into a web application along with an automated risk calculation programme. Our objective was to develop and test a globally accessible calculator application with the validation of performance on poor‐quality data. Despite the validation limitations, the text processing function serves the application satisfactorily. The AUGMERIS web app is built with Python 3 and shared globally by Streamlit. Volunteer testers from eight different countries performed a total of 383 trial calculations. The application has the potential to improve global pharmacotherapy by identifying patients requiring medication reviews. Its wider adoption might enhance patient safety and optimize treatments in a variety of healthcare systems.

Summary

Imagine doctors need to know if the mix of medicines a patient takes is risky, especially considering their kidney health. This paper describes a new web tool called Augmented Medication Risk Score (AUGMERIS) that automatically calculates this risk score. Reading messy drug lists from hospital records is hard for computers. So, this tool uses a combination of normal computer rules and smart artificial intelligence (like ChatGPT's relatives) to figure out the medications. The tool was tested and seems to work reasonably well compared with expert pharmacists. The goal is to help pharmacists quickly identify patients needing a medication check‐up, making healthcare safer.

Abbreviations

AI

artificial intelligence

API

application programming interface

ATC

Anatomical Therapeutic Chemical (classification system)

AUGMERIS

Augmented Medication Risk Score

CKD‐EPI

Chronic Kidney Disease Epidemiology Collaboration

CSS

Cascading Style Sheets

DRP

drug related problem

EHR

electronic health record

GPT

Generative Pre‐trained Transformer

HTML

HyperText Markup Language

MedRev

Medication Review

MERIS

Medication Risk Score

NEAK

Nemzeti Egészségbiztosítási Alapkezeelő (National Health Insurance Fund)

NLP

natural language processing

NLTK

Natural Language Toolkit

NNGYK

Nemzeti Népegészségügyi és Gyógyszerészeti Központ (National Centre for Public Health and Pharmacy)

NSAID

non‐steroidal anti‐inflammatory drugs

PUPHA

Hungarian Public Drug Database (not an acronym)

UI

user interface

URL

uniform resource locator

WHO

World Health Organization

1. Introduction

1.1. Background

The primary objective of clinical pharmacists is the prevention of drug related problems (DRPs) [1]. There are a lot of publications about the efficacy of the interventions of clinical pharmacists and many guidelines about the therapy optimization [2, 3, 4, 5], but their implementation into daily clinical practice encounters obstacles. The competence of clinicians is different in various countries, and there are deficiencies in the documentation systems. Therefore, there needs to be a lot of support for the process of Medication Review (MedRev) [2]. According to the definition [3], the aim of the pharmaceutical review of therapy is the optimization, which can reduce the occurrence of DRP resulting from side effects or interactions, as well as the extent of financial expenditure. The effectiveness of clinical pharmacy interventions in inpatient wards has been extensively documented in the literature, highlighting their positive impact on patient outcomes, medication management and overall healthcare costs. Clinical pharmacists play a critical role in optimizing pharmacotherapy and enhancing patient safety through various interventions [4, 5]. Clinical pharmacy in Hungary has developed greatly in the last 10 years, but compared with international statistics, there are still relatively few such professionals working in hospital wards. In the United States, the average ratio of clinical pharmacists to hospital beds is approximately 1 clinical pharmacist for every 20–30 beds, depending on the hospital's size and the complexity of care provided [6]. In contrast, European countries exhibit a wider range of ratios. For instance, in the United Kingdom, the ratio of clinical pharmacists to beds can vary from 1:30 to 1:50, depending on the hospital's resources and the specific services offered. Some reports indicate that in certain regions of Europe, the ratio may be as high as 1:60 or more, particularly in smaller hospitals or those with limited pharmacy services [7]. This disparity highlights the ongoing challenges in staffing and resource allocation faced by the healthcare system in Hungary, where clinical pharmacy roles are still evolving.

The identification of high‐risk medication therapies is an unsolved problem all over the world. Some specialized techniques are employed in specific medical specialties, but none of them are suitable for general use. One of them is a geriatric risk assessment tool for Emergency Medical Unit, which included only patients with age above 70 years [8]. An article about a literature review concentrates on the hospital admissions and readmissions related to medication usage [9]. Another approach is the risk–benefit assessment of medicines, which also has a universal framework [10]. While many different approaches exist to this problem, none of them have yet provided a general solution. The American Beers Criteria [11] and the European Screening Tool of Older Persons' Prescriptions (STOPP)/Screening Tool to Alert to Right Treatment (START) toolkit [12] are used only in geriatrics, and the target of the tools is the medicines, in contrast to the Medication Risk Score (MERIS) [13, 14, 15, 16] algorithm, which focuses on the patient. Creating risk maps of hospital wards with the MERIS algorithm seems promising, especially if we make the calculation more sophisticated by increasing the number of objective parameters. However, the integration of an artificial intelligence (AI) text processing method can be a key to its widespread usage because the electronic health record (EHR) systems are storing therapy data in very different formats, which makes new conversion approaches necessary. Therefore, the standardization of the data is essential for the automated execution of the evaluation algorithms, including the MERIS calculator. With the augmentation of acceptable data sources and includable patient parameters, AUGMERIS could be the general solution for identifying patients at high risk of medication overuse. We could create risk maps of the hospital wards using this patient‐focused tool, which would assist the clinical pharmacists in concentrating on the patients who require the most medication reviews.

1.2. Aim of the Study

In order to develop an application that automatically calculates the Danish Medication Risk Score (MERIS) from real data and is accessible from anywhere in the world, our two primary objectives were to automate this risk assessment method and to develop and validate a solution for processing the low‐quality therapy descriptions. For this purpose, we had to create our own two Python libraries: one for the computation and another for the transformation of the data. We therefore set out to develop and integrate these elements into the web application in this project.

2. Materials and Methods

2.1. List of the Applied Development Syntaxes

• Python 3 (Version: 3.9.6) [17, 18]

• HyperText Markup Language (HTML5) (Version: 5.1)

• Cascading Style Sheets (CSS3)

• JavaScript (Version: ES6)

2.2. List of Required Python Libraries

• Natural Language Toolkit (NLTK, Version: 3.8.1) library [19]

• Streamlit (Version: 1.31) [20]

• Pandas (Version: 2.1.3) [21]

• GeoPandas (Version: 1.0.1) [22]

• Folium (Version: 0.19.4) [23]

• OpenAI (Version: 1.54.1) [24]

• Pillow (Version: 11.0.0) [25]

• Matplotlib (Version: 3.9.2) [26]

2.3. The Structure of the MERIS Calculator Module

The MERIS algorithm was developed and published by Eva Aggerholm Sædder, Associate Professor at Aarhus University in Denmark, and her colleagues [13, 14, 15, 16]. This tool offers a scale, ranging from 0 to 36.2 points, to assess the risk of applying medications at the same time. We consider therapies with under 14 points as low‐risk, between 14 and 26 points as high‐risk and above 26 points as extreme‐risk, as shown on the scale in Figure 1. Clinical pharmacist interventions are necessary for high‐risk therapies, and a clinical pharmacologist's review is necessary for the extreme category. Therefore, it may be a powerful tool for clinicians to identify the patients who require medication review [27] the most. The MERIS algorithm requires the estimated glomerular filtration rate (eGFR) value, which indicates the patient's kidney function, and a list of drugs used simultaneously. Recent studies indicate that eGFR, particularly when derived from the Chronic Kidney Disease Epidemiology Collaboration (CKD‐EPI) equation, is more reliable than other estimation techniques across various patient demographics and medical conditions [28]. The MERIS algorithm assigns point values to the applied medications. Table 1 displays the 53 drug classes with point values for their potential for harm and interaction ability based on the classification table summary from the original article [13]. The kidney function score, the medication quantity score and the medication quality score are all summarized by the algorithm, as shown in Figure 2 [16].

FIGURE 1.

Screenshot from the AUGMERIS calculator application.

TABLE 1.

The harm and interaction values of the drug classes.

ID	Name	Harm	Interaction
1	Allopurinol	0.5	0.25
2	Alphablockers	0.5	0.25
3	Amiodarone	1.0	0.5
4	Antiepileptics	0.5	0.5
5	Antimycotics	0.5	0.25
6	Antipsychotics	1.0	0.25
7	Antithrombotics	0.5	0.25
8	Azathioprine	1.0	0.25
9	Benzodiazepines	1.0	0.25
10	Betablockers	0.5	0.25
11	Calcineurin inhibitors	1.0	0.5
12	Calciumantagonists excl. verapamil	0.25	0.25
13	Ciprofloxacin	0.5	0.25
14	Cyclophosphamide	1.0	0.25
15	Digoxin	1.0	0.25
16	Disulfiram	0.5	0.25
17	Erythromycin	0.5	0.5
18	Fibrates	0.25	0.25
19	Flecainide	1.0	0.25
20	Fluoxetine	0.5	0.25
21	Fluvoxamine	0.5	0.25
22	Glucocorticoids	1.0	0.25
23	Insulin	1.0	0.25
24	Isoniazide	0.5	0.25
25	Lithium	1.0	0.5
26	Loop diuretics	0.5	0.25
27	Low molecular heparines	0.5	0.25
28	Macrolides excl. erythromycin	1.0	0.25
29	MAO‐inhibitors	0.5	0.5
30	Methotrexate	1.0	0.25
31	Metoclopramide	0.25	0.25
32	Nitrofurantoin	0.5	0.25
33	NSAID	1.0	0.25
34	Ondansetron	0.25	0.25
35	Opioids	1.0	0.25
36	Oral antidiabetics	0.5	0.25
37	Other antidepressants	0.5	0.25
38	Other oral anticoagulants	1.0	0.25
39	Paracetamol	0.5	0.25
40	Potassium	0.5	0.25
41	Potassium sparing diuretics	0.5	0.25
42	Propafenone	1.0	0.25
43	Protease inhibitors	0.5	0.25
44	Renin‐angiotensin system inhibitors	0.5	0.25
45	Rifamycins	0.5	0.5
46	SSRI, excl. fluvoxamine, fluoxetine	0.5	0.25
47	Statins	0.25	0.25
48	TCA	1.0	0.5
49	Thiazides	0.5	0.25
50	Verapamil	0.5	0.25
51	Viagra	0.5	0.25
52	Vitamin K antagonists	1.0	0.5
53	Xanthines	0.5	0.25

FIGURE 2.

The MERIS algorithm from the original publication [16].

The first module we built contains the Python class, whose methods can be used to perform risk assessment based on the knowledge of eGFR and the Anatomical Therapeutic Chemical (ATC) codes of concomitant medications. The ATC classification operates on five hierarchical levels, allowing for a detailed categorization of drugs. This structure enables precise identification of active substances and their therapeutic applications, thereby enhancing data standardization and comparability across different healthcare settings [29]. The ATC classification can correctly handle the definitions of the drug classes of the MERIS algorithm with their including and excluding rules. We completed the data in Table 1 with the list of ATCs to be assigned in order to facilitate import and use in the Python module we created. All of its own features make use of the native Python 3 toolkit; the only prerequisite is the Pandas library. The class object has a method to calculate the MERIS score, which requires two arguments named eGFR and the ATCs. The first one accepts an integer or a None value, and the second waits for a list of strings containing the ATC codes of the simultaneously used drugs. This function assigns a kidney subscore to the eGFR value, suspecting that if it gets none, then the kidney function is good. It adds the quantity subscore according to the length of the ATCs list object and summarizes the harm and interaction potential values of classifiable medications as a quality subscore. Although the function returns the summarized score, the log of the calculator object contains the subscores.

2.4. The Structure of the Text‐Processing Module

One of the main obstacles to the widespread application of automated MERIS evaluation is the diversity of the data structures in EHR systems. Most EHR systems' databases have only one text field to record the medication therapy of inpatients, and it contains non‐uniform, hand‐typed descriptions with many typos. This type of data may be comprehensible to humans, but it cannot be processed using conventional informatics methods. Consequently, we had to create an algorithm that could manage our low‐quality data. Initially, we attempted direct text processing based on the key set generated from the National Centre for Public Health and Pharmacy (Nemzeti Népegészségügyi és Gyógyszerészeti Központ, NNGYK) [30] drug database. Processing unstructured data is one of the main benefits of applying generative AI in the healthcare industry [31]. The invention of Generative Pre‐trained Transformer (GPT) models represents a significant advancement in the field of AI, specifically in natural language processing (NLP). This novel architecture allows models to process sequences of text in parallel, making them faster and more efficient in learning from large datasets compared with previous sequential models [32]. The application of AI seemed promising to solve the data transformation problem, so we decided to try the GPT in a retrospective examination [33]. In the experiment, we used the same cleaned data pool to test two GPT models and one rule‐based algorithm in parallel. We concluded that none of them performed well on their own. Therefore, by combining the two technologies, we created a hybrid model that merges the precision and speed of the strict matching‐based algorithm with the flexibility of the GPT‐based solution. We have integrated AI features into the module in a discrete and quick manner by using OpenAI's application programming interface (API) to access the GPT model straight from the code [24]. Thus, satisfactory text processing has been achieved regardless of the quality of the source data. In its current form, the model first searches for complex active ingredients and drug names in the text entered, based on a key set compiled using the Hungarian drug databases (NNGYK [30] and the Hungarian Public Drug Database (PUPHA) from National Health Insurance Fund of Hungary (Nemzeti Egészségbiztosítási Alapkezelő, NEAK) [34]). The recognized sequences are deleted from the text, then the text is tokenized using NLTK, and the tokens are validated to ensure that only potential drug names remain in the list. On the validated dataset, a strict algorithm performs the search for single‐word drug names, and then we attempt to recognize the remaining elements with the help of GPT‐4o. By processing the inputs, we registered the tokens that the integrated cleaning algorithm did not remove from the source to discover its improvement possibilities. Figure 3 shows the text processing procedure.

FIGURE 3.

The text processing procedure.

2.5. The Validation Experiment for Text‐Processing Module

As the development progressed, it became necessary to test whether the text‐processing module provides reliable data for the evaluation. The two hypotheses we attempted to prove were that AI text processing can extract data from even the worst quality data source, and that the weighted number of the clinical pharmacist's interventions correlates with the MERIS value of the case. Following the original MERIS algorithm [14], we determined the medication risk scores for every record using the same risk categories as the original article [15]. Two independent clinical pharmacists were asked to evaluate the sample events in compliance with the objective criteria used in the MERIS algorithm. The experts categorized the drugs they took into consideration during the evaluation manually into the MERIS risk classes. Therefore, none of our self‐developed algorithms were used during the professional assessment procedure. In the experiment, we also attempted to measure the subjective opinions of experts. The first subjective value was the number of interventions the professional was supposed to take. We used a two‐dimensional scale of four because it would have been too challenging to use any validated weighting tool in the retrospective analysis: the values we asked for were the relevance and the urgency of the clinical pharmacist's intervention in the actual case. The subjective score we used in the statistics was calculated by multiplying the three values. The MERIS values determined using the AI‐based text processing method were then compared with the results of the validation experiment we talked about above. It was also investigated whether the MERIS values of the cases correlate with the metric of professionals' subjective opinions.

2.5.1. Setting and Data Collection

The data pool for the validation experiment of the text processing algorithm was a set of anonymized case descriptions from Karolina Hospital in Hungary, which contained the description of 6049 inpatient events from the year 2022. After data cleaning, where the selection criterion was that the file contained both blood count results and therapy descriptions, the size of the dataset included in the experiment was narrowed to 2999, from which we mined out the therapy description sections with the names of the simultaneously used drugs and the eGFR value. We created the sample for the human evaluation by randomly choosing 15 cases from all three risk groups (low, high and extreme) based on the automated assessment of the case descriptions.

2.5.2. Outcomes

The primary outcomes were the MERIS scores calculated by the AI‐powered module and those determined by the clinical pharmacists. A secondary outcome was the subjective, weighted score of professional interventions, which was used to explore its correlation with the MERIS values.

2.5.3. Data Analysis

For the evaluation, we used the Spearman correlation [35, 36] calculation with a p < 0.05 significance level.

2.6. The Description and Functionality of the Web Application

We combined the text processing module and the calculator module into a web application using Version 1.31 of the Streamlit framework, whose main benefit is that it saved us a great deal of time by allowing us to build the app in pure Python 3 code. The majority of the software's components were created in Python 3. The client‐side languages (HTML5, CSS3 and JavaScript) were used only to increase the user experience and to collect the country data. The application is running on a public Streamlit server and available at the https://meriscalculator.streamlit.app/ Uniform Resource Locator (URL). The Python 3 files are stored in a private repository on GitHub, from where they are automatically deployed to the server.

To increase the calculator's functionality, we created a database on a private, secure server to track application usage. In this database, we record input values, the country of access, the return values of evaluation functions and the response time of the application. The usage of the application is voluntary, and the data the testers provide are predominantly fictitious. However, even with a genuine therapy description and eGFR value, the stored data is insufficient to identify the patient. We use this data to calculate statistics and assess response accuracy through human review to identify unforeseen errors and make algorithmic corrections. We are searching for medications that have been misidentified and are attempting to determine the cause of the error and find a solution to avoid them in the future.

Although the tool is primarily designed to assist clinical pharmacists, we have included relevant references on the interface and provided various supplementary information to make the application usable by anyone. The main page of the user interface (UI) is easy to use and lets users enter the therapy description and eGFR value in free‐text format. The application automatically processes the entered data, calculates and displays the MERIS value and visually illustrates the distribution of subscores and the MERIS category. It also provides a view of the drugs that have been identified, indicating the type of processing method. Figure 1 displays the application UI filled with fictitious data. The other pages of the interface provide some statistical information, such as a usage timeline for the last 60 days and a feature named ‘Activity Atlas’, where users can see the actual distribution of application accesses by continent. The tool we used to show the activity on a static map was GeoPandas, and the interactive map on the application interface was made in Folium, which are geographic information libraries for Python 3.

2.7. The Global Usage Analysis of the Web Application

This section outlines the methodology for the prospective analysis of data collected from the public, globally accessible AUGMERIS web application.

2.7.1. Setting and Data Collection

Data was collected voluntarily and anonymously from users of the AUGMERIS web application across eight different countries. The dataset includes 383 trial calculations, capturing input values (therapy descriptions, eGFR), application response times and the country of access.

2.7.2. Outcomes

The outcomes of this analysis were the application's performance metrics (e.g., average response time for AI vs. local processing), the proportion of AI‐assisted drug recognition across different countries and usage statistics, including the number of calculations per country and the distribution of MERIS risk categories.

2.7.3. Data Analysis

Descriptive statistics (mean, standard deviation) were used to summarize the performance and usage data. A two‐proportion Z‐test [37] was performed to statistically compare the rate of AI‐assisted drug identifications in the global dataset versus the domestic EHR dataset. Additionally, a Spearman rank‐order correlation test [35, 36] was conducted to evaluate the correlation between the medicine usage frequencies in the two datasets.

2.8. Policy of the Study

The study was conducted in accordance with the Basic & Clinical Pharmacology & Toxicology policy for experimental and clinical studies [38]. Furthermore, it complies with Hungarian Act CLIV of 1997, §157 ‘Medical research conducted on humans’ and Regulation (EU) 2016/679 Article 9. Our study is licensed by the Hungarian Medical Research Council under licence number ‘BM/8122‐1/2025’.

3. Results

3.1. The Outcome of the Validation Experiment for the Text‐Processing Module

From the randomly selected life descriptions (N = 45 cases), 11 cases ended with the patient's death, which our two invited experts handled differently. Our first expert processed the last assessable medication of the deceased, while our second expert disregarded these cases. We conducted the correlation assessment accordingly. We applied the Spearman correlation [35, 36] calculation for the evaluation. In the case of our first expert, the correlation coefficient is 0.463 (p < 0.05), while for the second expert, the correlation coefficient is 0.496 (p < 0.05). The correlation is moderate in both cases, but there is no significant difference between the opinions of the two experts (p = 0.846). Figure 4 shows the plot of the professionals evaluated MERIS values compared with AI‐processed MERIS values.

FIGURE 4.

The MERIS scores of the evaluation of professionals compared with the AI calculation.

3.2. The Analysis and Statistics of the Web Application's Usage Data

To find the abnormal behaviour, we created a database to record the calculator's activity for statistical and development purposes. In this database, there are 383 records from different places in the world. The GPT model was involved 244 times by a 3.00 s average response time, and the pure strict matching algorithm responded in 89 cases with an average of 0.62 s. The mean response time of the applications was 2.38 s. Reduced kidney function was found in 277 of the trial calculations, with 65 of those having critically low values. The average MERIS value is 11.87 points, with 228 low‐risk (mean = 6.24, SD = 4.30), 137 high‐risk (mean = 18.81, SD = 3.66) and 18 extreme (mean = 29.93, SD = 1.35) cases. On average, there were 6.35 simultaneously administered medications, and we recognized 339 different drug names referring to 319 unique medicines in the therapy descriptions. Germany, with 82 calculations, Hungary, with 193 calculations, and New Zealand, with 89 calculations, were the three most active nations. In all three active countries, the percentage of typos and foreign drug names that needed AI assistance to recognize was between 20% and 30%. The concrete percentile distribution of drug names recognized by AI in the three most active countries was as follows: Hungary 20.57% (N = 525), Germany 26.26% (N = 811), New Zealand 29.64% (N = 904). These data are collected in Table 2. Figure 5 shows the activity of the AUGMERIS calculator on the map.

TABLE 2.

Summary of data metrics from the AUGMERIS calculator.

Metric	Value/description
All calculations	383 instances
MERIS risk categories	Low risk: 228 (mean = 6.24, SD = 4.30) High risk: 137 (mean = 18.81, SD = 3.66) Extreme: 18 (mean = 29.93, SD = 1.35)
GPT model usage	244 instances (average response time: 3.00 s)
Strict matching algorithm	89 cases (average response time: 0.62 s)
AI recognized drug rate (Top 3 countries)	Hungary: 20.57% (N = 525) Germany: 26.26% (N = 811) New Zealand: 29.64% (N = 904)
Mean response time (applications)	2.38 s
Reduced kidney function	277 cases (65 critical values)
Average MERIS value	11.87 points
Average simultaneous medications	6.35
Recognized drugs in therapy	339 different drug names referring to 319 unique medicines

FIGURE 5.

The activity map of AUGMERIS calculator.

3.3. Comparison of the Mining Results of the Two Datasets

In contrast to 32.32 (SD = 80.25) in the EHR data, the application had an average token number of therapy descriptions of 12.89 (SD = 12.01). Furthermore, the algorithm discovered 1618 unique junk tokens in the 2999 instances of therapy descriptions from EHR data, compared with 86 distinct junk tokens by 402 processing procedures in the global tests, which greatly represents the difference in the data quality. We conducted a comparative Z‐test to examine the proportion of AI‐assisted drug identifications across the two datasets we negotiate in this article. As illustrated in Figure 6, in the dataset from the international test of the web application, 25.82% of the identified drugs (N = 2386) were recognized via the AI method, while in the domestic dataset that we used in the validation experiment of the text‐processing module, only 10.34% (N = 28 269) were AI‐identified. According to a two‐proportion Z‐test [37], the difference was statistically significant (Z = 8.67, p < 0.0001), indicating that processing the global dataset required more AI recognition. Table 3 displays the detailed frequency of identified drugs based on the ATC level 2 groups, which were represented in both data sets. The Spearman rank‐order correlation test revealed a strong, positive and statistically significant correlation between the medicine usage frequencies from the two sources (ρ = 0.8047, p < 0.0001). This indicates that as the rank of medicine usage increases in one data source, it also increases in the other.

FIGURE 6.

The ratio of AI‐based drug identifications in the two datasets.

TABLE 3.

Frequency of identified drugs in the two datasets according to the common ATC Level 2 groups.

ATC code	Group name	HER (N = 27 905)	APP (N = 2500)
A01	Stomatological preparations	19 (0.07%)	2 (0.08%)
A02	Drugs for acid related disorders	2228 (7.98%)	111 (4.44%)
A03	Drugs for functional, gastrointestinal disorders	271 (0.97%)	14 (0.56%)
A04	Antiemetics and antinauseants	23 (0.08%)	6 (0.24%)
A05	Bile and liver therapy	119 (0.43%)	2 (0.08%)
A06	Drugs for constipation	132 (0.47%)	31 (1.24%)
A07	Antidiarrheals, intestinal, antiinflammatory/antiinfective agents	448 (1.61%)	26 (1.04%)
A09	Digestives, incl. enzymes	37 (0.13%)	3 (0.12%)
A10	Drugs used in diabetes	1227 (4.40%)	134 (5.36%)
A11	Vitamins	426 (1.53%)	99 (3.96%)
A12	Mineral supplements	1001 (3.59%)	30 (1.20%)
A16	Other alimentary tract and metabolism products	102 (0.37%)	2 (0.08%)
B01	Antithrombotic agents	3240 (11.61%)	115 (4.60%)
B02	Antihemorrhagics	235 (0.84%)	1 (0.04%)
B03	Antianemic preparations	265 (0.95%)	53 (2.12%)
B05	Blood substitutes and perfusion solutions	2058 (7.38%)	6 (0.24%)
C01	Cardiac therapy	849 (3.04%)	37 (1.48%)
C02	Antihypertensives	224 (0.80%)	26 (1.04%)
C03	Diuretics	1892 (6.78%)	172 (6.88%)
C05	Vasoprotectives	203 (0.73%)	12 (0.48%)
C07	Beta blocking agents	1531 (5.49%)	141 (5.64%)
C08	Calcium channel blockers	315 (1.13%)	71 (2.84%)
C09	Agents acting on the renin‐angiotensin system	1574 (5.64%)	197 (7.88%)
C10	Lipid modifying agents	801 (2.87%)	171 (6.84%)
D01	Antifungals for dermatological use	2 (0.01%)	3 (0.12%)
D02	Emollients and protectives	3 (0.01%)	1 (0.04%)
D07	Corticosteroids, dermatological preparations	41 (0.15%)	19 (0.76%)
D08	Antiseptics and disinfectants	1 (0.00%)	1 (0.04%)
D10	Anti‐acne preparations	4 (0.01%)	4 (0.16%)
G03	Sex hormones and modulators of the genital system	6 (0.02%)	9 (0.36%)
G04	Urologicals	193 (0.69%)	36 (1.44%)
H02	Corticosteroids for systemic use	701 (2.51%)	21 (0.84%)
H03	Thyroid therapy	228 (0.82%)	43 (1.72%)
H05	Calcium homeostasis	1 (0.00%)	1 (0.04%)
J01	Antibacterials for systemic use	1978 (7.09%)	110 (4.40%)
J02	Antimycotics for systemic use	13 (0.05%)	1 (0.04%)
J05	Antivirals for systemic use	169 (0.61%)	6 (0.24%)
L01	Antineoplastic agents	27 (0.10%)	21 (0.84%)
L02	Endocrine therapy	9 (0.03%)	15 (0.60%)
L04	Immunosuppressants	21 (0.08%)	10 (0.40%)
M01	Antiinflammatory and antirheumatic products	172 (0.62%)	52 (2.08%)
M02	Topical products for joint and muscular pain	10 (0.04%)	4 (0.16%)
M03	Muscle relaxants	42 (0.15%)	3 (0.12%)
M04	Antigout preparations	213 (0.76%)	55 (2.20%)
M05	Drugs for treatment of bone diseases	5 (0.02%)	10 (0.40%)
M09	Other drugs for disorders of the musculo‐skeletal system	1 (0.00%)	1 (0.04%)
N01	Anaesthetics	4 (0.01%)	21 (0.84%)
N02	Analgesics	920 (3.30%)	226 (9.04%)
N03	Antiepileptics	705 (2.53%)	51 (2.04%)
N04	Anti‐Parkinson drugs	66 (0.24%)	8 (0.32%)
N05	Psycholeptics	1163 (4.17%)	71 (2.84%)
N06	Psychoanaleptics	704 (2.52%)	100 (4.00%)
N07	Other nervous system drugs	103 (0.37%)	13 (0.52%)
P01	Antiprotozoals	3 (0.01%)	1 (0.04%)
R02	Throat preparations	2 (0.01%)	1 (0.04%)
R03	Drugs for obstructive airway diseases	793 (2.84%)	64 (2.56%)
R05	Cough and cold preparations	147 (0.53%)	4 (0.16%)
R06	Antihistamines for systemic use	73 (0.26%)	21 (0.84%)
S01	Ophthalmologicals	136 (0.49%)	22 (0.88%)
S03	Ophthalmological and otological preparations	1 (0.00%)	2 (0.08%)
V03	All other therapeutic products	25 (0.09%)	7 (0.28%)

4. Discussion

The purpose of this study was to create and validate the AUGMERIS web application, a tool that processes unstructured, real‐world therapy descriptions to automate the Medication Risk Score (MERIS) calculation. Our primary findings show that this effort was successful. First, the human validation experiment confirmed that our novel, hybrid text‐processing module performs effectively, showing a significant (p < 0.05), moderate correlation with MERIS scores calculated by clinical pharmacy experts. Second, the usage of the web application was beyond expectations, confirming the tool's necessity. Its statistics revealed worldwide interest. The areas of Germany and New Zealand were very active in testing our application, demonstrating that identifying patients who are at high risk of medication errors is just as important there as it is in Hungary or Denmark. Finally, the fact that AI was required to identify 20%–30% of medications in the worldwide test versus the approximately 10% necessity in the domestic dataset underscores that flexibility is crucial in handling the linguistic diversity inherent in real‐world clinical data.

4.1. Performance and Validation of the Hybrid Text‐Processing Module

Our first hypothesis in the validation experiment was that the AI text processing can extract data from even the worst quality data source, just like professionals. The findings mentioned above allow us to keep this hypothesis. As we see in Figure 4, the AI and the professionals evaluated some cases very differently. These cases needed long hospitalization, and the source file contained more than one therapy description and eGFR value. While the computer used the worst values for the evaluation, the experts selected the most pertinent ones. This difference might be substantial if the patient's condition significantly improved while they were in the hospital.

The second hypothesis of the experiment was that the weighted number of the clinical pharmacist's interventions is correlated with the MERIS value of the case. However, as discussed in the limitations, the subjective nature of the intervention scoring did not allow for this hypothesis to be confirmed, highlighting an area for future methodological improvement.

4.2. Experience With the Web Application

We developed the web application initially as an internal tool to allow the research team to test and refine the core programme modules. However, its public introduction revealed a noteworthy, unanticipated level of international interest. The significant use by nations like Germany and New Zealand (Figure 5) highlights the need for objective instruments to categorize medication‐related risks among clinicians worldwide. We regularly reviewed the application records to identify any errors, and if required, we updated the backend. For example, we found that the system frequently only took the first word into account when identifying drug names that were composed of multiple words. After that, a distinct programme section was added to the text processing to incorporate complex key search. The data provided by testers may not have clinical context, but it does reveal information about the overall medication therapy habits.

4.3. The Performance of the Hybrid Text‐Processing on the Different Datasets

Our findings confirm that the ratio of AI recognition increases with distance from Hungary because we declared the key set of the strict algorithm using data from the Hungarian national databases [30, 34]. The AI‐assisted drug identification ratio was 25.82% in the global dataset, while it was only 10.34% in the domestic dataset, despite the fact that the Hungarian dataset was of poor quality. Therefore, we can conclude that the AI recognition ratio depends much more on the language and location than on the quality of the data. We can infer from the strong, positive and statistically significant correlation between the two sources' medicine usage frequencies that clinical experts tested the application using actual case therapy descriptions. Global guidelines dictate the therapies for the majority of diseases, so professionals choose similar medications for similar conditions.

4.4. Strengths and Limitations

The primary strength of our novel tool is to address a real‐world clinical need, providing a flexible and objective approach in the risk assessment of medical therapies. It could be a powerful tool in the hands of clinical professionals to identify the patients who need the therapeutic intervention the most. The majority of EHR systems worldwide contain the data needed by the algorithm, and our AI‐supported data transformation technique can standardize the data for the computation regardless of the source's quality. Therefore, we must request less data in a less structured format, which is crucial for user satisfaction. In Figure 1, we can see that, thanks to its flexibility, the application can handle the original medicine names, active ingredient names and typos as well. We consider this result a great success because the method we used can be applied to the screening, anamnesis taking and patient follow‐up as well [39, 40, 41]. This flexibility makes our solution well‐suited for future integration into diverse healthcare IT systems, including those in inpatient care institutions, primary outpatient care and community pharmacies. Although the rule‐based algorithm is not perfect, we were able to correct some of its mistakes, such as misidentifying combined agent medications, by using activity tracking. Given the project's aims, the main advantages are clear, but because of the multi‐phase development and evaluation process, it is important to look at each component's limitations independently. Limitations associated with the use of AI in text processing.

Because the GPT is based on statistical data, it performs best on the most frequently discussed topics and less well on the less frequently mentioned ones. The ATC categorization of medicines belongs to rarely mentioned topics, so we have to treat the results with caution. It is worth noting that the GPT will try to figure out the answer even if it does not know the correct ATC code, and the answer may be correct. But that flexibility can also cause errors in even the clearest cases. Therefore, it is important to use mostly the traditional text processing method instead of AI, which permanently has the ability to make mistakes [42].

4.4.1. Limitations Related to the Text‐Processing Validation Experiment

The human validation of our text‐processing module, while yielding positive results, has several limitations. Firstly, the sample size used for the evaluation by the experts was relatively small (N = 45), which limits the statistical power of the findings. The sample size was determined based on the tight budget and the limited capacity of the two invited experts. Secondly, the source data for this validation was drawn from a single Hungarian hospital. While the data was of poor quality and thus suitable for testing robustness, it may not fully represent the linguistic diversity and formatting quirks of EHR data from other countries. Finally, our attempt to find the correlation between the MERIS scores and the subjective measure of clinical interventions was not successful. We deduced from the results that the experiment's subjective evaluation rules were not precisely defined enough to support the hypothesis. This part of the experiment highlights the difficulty of retrospectively assessing the necessity of the clinical interventions.

4.4.2. Limitations Related to the Public Web Application Data

Although the data gathered from the public AUGMERIS web application is useful for gauging interest worldwide, its authenticity is questionable. The primary limitation is the lack of a ‘ground truth’. We are unable to confirm the actual clinical context or the accuracy of the input therapy descriptions or eGFR values because the data was provided voluntarily and anonymously. Therefore, this dataset demonstrates the tool's real‐world usage patterns and technical performance rather than its clinical accuracy. The user base was also self‐selected, so the usage data might not accurately reflect the population of clinical pharmacists in general.

4.5. Implications for Clinical Practice and Future Directions

From a clinical practice perspective, the immediate impact of a tool like AUGMERIS is its potential to enhance efficiency and patient safety. It can assist clinical pharmacists in prioritizing their caseloads by giving them a quick, objective risk score. This allows them to concentrate on patients with the most dangerous drug regimens who would most benefit from the medication review. Due to its flexibility, the solution can be integrated into clinical workflows as a component of institutional EHR systems, as a locally connected module or as a web service. Our future research should measure outcomes like an official cost‐effectiveness analysis, changes in prescribing patterns after the tool's implementation or decreases in medication‐related adverse events to quantify this impact.

Building on our experiences so far, our future work will focus on overcoming the identified limitations to enhance the robustness and clinical validity of the instrument. To enhance the reliability of the subjective intervention scoring, our next investigation will implement a validated classification system, such as the Act‐IP [43], DokuPIK [44], GSASA V2 [45] or Vi‐Med [46]. This will enable a prospective study to objectively assess the relationship between the AUGMERIS score and the type and severity of clinical pharmacist interventions that are required. To obtain a larger number of validation samples, we would like to include active inpatient wards in an upcoming multicentre study. In addition to increasing statistical power, this will enable the tool to be evaluated in various clinical settings. Based on our current experience, we can offer a quick tool for creating ward risk maps, and when paired with the selected method of intervention classification, we can obtain a representative dataset for assessing the correlations. The phenomenon of MERIS alteration during prolonged hospital stays may be used to rank the clinical pharmacists' interventions in the future.

Furthermore, to reduce the reliance on the AI component and improve the accuracy of the rule‐based algorithm, we will expand its key set with non‐European drug names by integrating international drug databases, such as those from the World Health Organization (WHO) [47], ResourcePharm [48] and Drugs.com [49]. As the key dictionary of the strict algorithm improves, the proportion of AI‐identified medications will decline. This will enhance the tool's speed and reliability for global users by the next phase of validation.

Because EHR systems store relevant information in a semi‐structured form, allowing us to access higher quality data, we do not have to process the worst quality data sources in clinical practice. However, the data mining technique we employed may be crucial in the data migration to the national centralized EHR systems.

5. Conclusion

The results of our study effectively illustrate the substantial global demand for instruments capable of effectively identifying patients at high risk and expediting medication reviews. A central achievement of this study is the creation of a hybrid text‐processing model that effectively combines the rule‐based algorithm with GPT technology. This model successfully addressed a significant obstacle to automated risk assessment by processing unstructured and low‐quality data from EHRs. The validation experiment confirmed that the AI‐driven data extraction performs comparably to that of clinical professionals' evaluations, affirming its utility for processing complex real‐world data. The AUGMERIS application represents a reliable solution that can be integrated into clinical workflows to generate ward risk maps and prioritize pharmacist interventions. By continuously refining the algorithm based on usage data and expanding its core dataset, the tool's accuracy and speed can be further enhanced. Ultimately, AUGMERIS is a significant step toward a universally applicable system for improving pharmacotherapy safety.

Conflicts of Interest

The authors declare no conflicts of interest.

Bertalan Á., Angyal V., Domján P., et al., “Development of an Artificial Intelligence Powered Medication Risk Score Calculator Application,” Basic & Clinical Pharmacology & Toxicology 137, no. 4 (2025): e70109, 10.1111/bcpt.70109.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.