. 2023 Oct 21;12:100346. doi: 10.1016/j.rcsop.2023.100346

Table 1.

Summary of findings.

Adverse drug reaction (ADR) detection
Author, Year	Country	Objective of the Study	Study design	Participants	Main findings
Mohsen A, 2021¹⁰	Japan	Using various machine learning methods, estimating the likelihood of adverse drug reactions or events (ADRs) during drug discovery.	Database study	Open TG-GATEs (Toxicogenomics Project-Genomics Assisted Toxicity Evaluation Systems) for drug-induced gene expression profiles and FAERS (FDA [Food and Drug Administration] Adverse Events Reporting System) database for ADR occurrence information	A total of 14 predictive models were built using this framework and Deep Neural Networks (DNN), with a mean validation accuracy of 89.4%, indicating that the approach successfully and consistently predicted ADRs for a wide range of drugs. As case studies, researchers looked at how prediction models performed in the context of Duodenal ulcer and fulminant Hepatitis, highlighting mechanistic insights into those ADRs. The developed predictive models will aid in assessing the likelihood of ADRs when testing new pharmaceutical compounds.
Yalçın N, 2022¹¹	Turkey	The primary goal of this study was to generate objective risk categories by incorporating severity with NAESS and probability with the ‘Du'ADRs algorithm into the risk matrix analysis performed by a multidisciplinary team that included a clinical pharmacist. The next goal was to create a machine learning-based clinical decision support tool (risk score) that predicts whether these identified ADRs will occur.	Prospective cohort study	The study included all admitted neonates, but those with preexisting hepatic or renal impairment were excluded.	Enoxaparin, dexmedetomidine, vinblastine, dornase alfa, etoposide/carboplatin, and prednisolone were identified as high-risk drugs. According to the random forest importance criterion, the independent variables included in the risk score to predict ADR presence were: systemic hormones (2 points), cardiovascular drugs (3 points), circulatory system diseases (1 point), nervous system drugs (1 point), and parenteral nutrition treatment (1 point) (cut-off value: 3 points). This risk score correctly classified 91.1% of the test set observations (c-index: 0.914).
Hammann, F, 2010¹²	Switzerland	To conduct a comprehensive survey of ADR reports for a wide range of clinically used drugs and to develop computational models for understanding and predicting such reactions	Database study	Structure-activity relationship analysis of adverse drug reactions (ADRs) in the CNS, liver, and kidney, as well as allergic reactions, for a wide range of drugs (n = 507) from the Swiss drug registry	For allergic, renal, CNS, and hepatic ADRs, the models had high predictive accuracies (78.9–90.2%). The feasibility of predicting complex end-organ effects using simple models that do not require expensive computations and can be used for (i) compound selection during the drug discovery stage, (ii) understanding how drugs interact with target organ systems, and (iii) generating alerts in post marketing drug surveillance and pharmacovigilance.
Cami A, 2011¹³	USA	(i) To create a predictive approach that integrates various data types, such as structural network properties, drug intrinsic properties, and drug and adverse drug event (ADE) taxonomies, and introduces several previously unexplored covariates. (ii) Using a simulated prospective approach, evaluate network-based predictive models.	Prospective evaluation	Based on a snapshot of a widely used drug safety database from 2005, drug-ADE associations were created for 809 drugs and 852 ADEs and supplemented these data with additional pharmacological information.	The proposed model had an AUROC (area under the receiver operating characteristic curve) statistic of 0.87, with a sensitivity of 0.42 and a specificity of 0.95. These findings imply that predictive network methods can be used to forecast unknown ADEs.
Rahmani H, 2016¹⁴	Not mentioned	To develop a novel network approach for ADR prediction, called Augmented Random-WAlk with Restarts (ARWAR).	Database study	The side-effects of each drug in the DrugBank database were extracted from the SIDER (Side Effect Resource) database, and 146 drugs with at least 5 target proteins and 5 side-effects were chosen.	According to the empirical results, the ARWAR method outperformed the existing network approach by 20% in terms of average Fmeasure. ARWAR was also capable of generating novel hypotheses about drugs in terms of novel and biologically meaningful ADR.
Bresso E, 2013¹⁵	France	To develop a method for identifying and characterizing side-effect profiles (SEPs) shared by several drugs.	Database study	Drug annotations from SIDER and DrugBank databases	Cross-validation and direct testing with new molecules were used to assess learning efficiency. A comparison of two machine-learning techniques: decision trees and inductive logic programming. Demonstrated that the inductive-logic-programming method was more sensitive than decision trees and could successfully exploit background knowledge such as functional annotations and drug target pathways, producing rich and expressive rules.
Bean DM, 2017¹⁶	United Kingdom	To construct a knowledge graph containing four types of nodes: drugs, protein targets, indications, and adverse reactions	Database study	Public data on drug targets, indications and ADRs	The developed machine learning algorithm based on a simple enrichment test and first demonstrated how well this method performed at classifying known causes of adverse reactions (AUC 0.92). A cross validation scheme in which 10% of drug-adverse reaction edges were systematically deleted per fold revealed that the method correctly predicts on average 68% of the deleted edges.
Onay A, 2017¹⁷	Turkey	To develop computational classification methods that can distinguish between approved drugs and withdrawn ones.	support vector machines (SVMs) and ensemble methods (EMs)	6 data sets with a total of 110 approved and withdrawn drugs for all and nervous system diseases.	The descriptors number of total chemotypes and bond CN amine aliphatic generic were the most significant. On the test set for drug data set including all diseases, the developed Medium Gaussian SVM model achieved 78% prediction accuracy. For phycholeptics and psychoanaleptics drugs, the bagged tree and linear SVM models achieved 89% accuracy. In nervous system withdrawn drug (NSWD) data sets, a set of discriminative fragments was obtained.
Dandala B, 2019¹⁸	USA	(i) To detect mentions of medication name and attributes (dosage, frequency, route, and duration), as well as ADEs, indications, other signs or symptoms (SSLIF), and severity. (ii) To identify the characteristics of a medication, the relationships between medications and ADEs (referred to as “adverse” relations), medications and indications (referred to as “reason” relations), and the severity of an ADE, sign, or symptom. (iii) To develop an integrated system of the two tasks, in which entities recognized by the system in task 1 are used to identify relationships.	Natural language processing (NLP) techniques	1089 de-identifed clinical notes of 21 cancer patients, of which 213 were the unseen test dataset and 876 training dataset	The accuracy analysis of the three methods revealed that the joint modelling technique improved performance (F measure) by nearly 3% points (4.5% relative) over the traditional approach, and the addition of FAERS information improved system performance by another 1% point (1.4% relative)—achieving an overall F measure of 0.661.
Dey S, 2018¹⁹	USA	To develop machine learning models, including a deep learning framework, that can predict ADRs and identify the molecular substructures associated with those ADRs without having to define the substructures beforehand.	Database study	Shortest-path, PubChem, MACCS, CDK Standard, CDK Graph, Klekota-Roth (KR), E-State, CDK Hybridization, CDK Extended, and ECFP6 were the ten popular chemical fingerprints used for ADR prediction tasks.	The model's performance was compared with ten different state-of-the-art fingerprint models, the neural fingerprints from the deep learning model outperformed all other methods in predicting ADRs. Important molecular substructures were associated with specific ADRs using feature analysis on drug structures and statistically assessed their associations.
Yang X, 2019²⁰	USA	To develop a machine learning-based clinical NLP system - MADEx for detecting medications, ADEs and their relations from clinical notes.	Database study	A corpus of 1089 de-identified clinical notes was used to extract clinical NER and relations.	On the validation set, the RNN-1 model outperformed the CRFs model with an F1-score of 0.8897. On the test set, RNN-2 had the highest F1-score of 0.8233, outperforming RNN-1 (0.8134) and CRFs (0.7250).
Chapman AB, 2019²¹	USA	To develop a natural language processing (NLP) system that will identify mentions of symptoms and drugs in clinical notes and label the relationship between the mentions as indications or adverse drug effects (ADEs).	Database study - Named Entity Recognition (NER)	Clinical notes from the UMASS hospital were de-identified and manually annotated into categories.	The NLP system validation was carried out against the evaluation set provided by the MADE 1.0 challenge, and the performance of our system was compared to that of other submitted systems. The micro-averaged F1 score for NER was 80.9%, RE was 88.1%, and the final system was 61.2%.
Duan L, 2012²²	USA	To develop methods for identifying the associations that the observational medical outcomes partnership (OMOP) defined in order to simulate data from the observational simulated dataset.	Database study	The simulated dataset contains ten million people, 90 million drug exposures from 5000 different drugs, and 300 million condition occurrences from 4500 different conditions from a period of over a ten-years.	The experimental results show that the proposed pattern discovery method improves the standard baseline algorithm—chi-square—by 23.83% on the simulated OMOP dataset.
Huang LC, 2011²³	USA	To develop a computational systems pharmacology framework consisting of statistical modelling and machine learning to predict ADR of drugs.	Database study	Clinical observation data combined with drug target data, PPI networks, and gene ontology (GO) annotations.	An in-silico model based on this framework could predict cardiotoxicity ADRs with reasonable accuracy (median AUC = 0.771, Accuracy = 0.675, Sensitivity = 0.632, and Specificity = 0.789). The findings also highlighted the importance of using prior knowledge, such as gene networks and gene annotations, to improve future ADR assessments.

Community pharmacy
Takase T, 2022²⁶	Japan	To assess the impact on medication dispensing of automated dispensing robots and collaborative work with pharmacy support staff.	Prospective study	Prescriptions filled with each dispensing device during the study periods	The total incidence of prevented dispensing errors was significantly reduced after the robotic dispensing system was introduced (0.204% [324/158,548] to 0.044% [50/114,111], p < 0.001). The total number of unpreventable dispensing errors was reduced significantly (0.015% [24/158,548] to 0.002% [2/114,111], p < 0.001). The number of cases of wrong strength and wrong drug, which can have serious consequences for a patient's health, had reduced to almost zero. Pharmacists' median dispensing time per prescription was significantly reduced (from 60 to 23 s, p < 0.001).

Computerized physician order entry
Jungreithmayr V, 2021³⁴	Germany	To investigate the distinct effects of a CPOE system implemented on general wards in a large tertiary care hospital on the quality of prescription documentation.	Retrospective analysis	Two groups of 160 patients' prescriptions	The overall mean prescription-Fscore increased from 57.4% ± 12.0% (n = 1850 prescriptions) prior to implementation to 89.8% ± 7.2% (n = 1592 prescriptions) after (p < 0.001). Individual criteria-Fscores improved significantly in most criteria (n = 14), with 6 criteria achieving a total score of 100% after CPOE implementation. While the implementation of a CPOE system generally improved the quality of prescription documentation, certain criteria were difficult to meet even with the assistance of a CPOE system.

Dose recommendation
Blasiak A, 2022³⁷	Singapore	To develop CURATE.AI, a small data, AI-derived platform that harnesses only a patient's own prospectively/longitudinally acquired data to dynamically identify their own optimal and personalized doses.	Open-label, multi-center, single-arm, prospective feasibility trial	Patients with advanced solid tumours who were treated with single-agent capecitabine, XELOX, or XELIRI (plus/minus biologics).	When compared to the projected SOC dose, the prescribed dose was reduced by 20% (13.8%) on average. The nine patients who were reported completed 3.9 cycles (2.2 cycles), with the longest participation lasting 8 cycles. CURATE. AI recommendations were considered in 27 of the 40 total dosing decisions, and 26 of those decisions were accepted for prescription.
Wang Z, 2022³⁹	Singapore	To develop a machine learning algorithms to recommend vancomycin dosage in tertiary general hospital patients.	Retrospective analysis	Inpatients, who received at least one vancomycin injection during the period from January 1, 2017 to December 31, 2019, were selected.	Only a small proportion (34.1%) of current injection doses could achieve the desired vancomycin trough level (14–20 μg/ml) in the 3-year data. The machine learning models outperformed the traditional pharmacokinetic models in terms of PAR and MAE. In the test data, the model outperformed the other previously developed machine learning models.
Hu YH, 2018⁴⁰	Taiwan	To predict the appropriateness of initial digoxin dosage using machine learning techniques.	Retrospective analysis	Patients who had been hospitalized and had their conditions treated with digoxin between 2004 and 2013	Six machine learning techniques were considered: decision tree (C4.5), kNN, classification and regression tree (CART), randomForest (RF), multilayer perceptron (MLP), and logistic regression (LGR). The area under the ROC curve (AUC) of RF (0.912) was excellent in the non-DDI group, followed by MLP (0.813), CART (0.791), and C4.5 (0.784); the remaining classifiers performed poorly. The AUC of RF (0.892) was the best for the DDI group, followed by CART (0.795), MLP (0.777), and C4.5 (0.774); the other classifiers performed poorly.
Roche-Lima A, 2020⁴¹	Puerto Rico	Using genetic and non-genetic clinical data, compare seven ML methods for predicting stable warfarin dosing in Caribbean Hispanic patients.	An open-label, single-center, population-based, observational, retrospective cohort study	Participants were recruited from an anticoagulation clinic affiliated with the Veteran's Affairs Caribbean Healthcare System (VACHS) in San Juan, Puerto Rico.	Random forest regression (RFR) outperformed all other methods, with a mean absolute error (MAE) of 4.73 mg/week and 80.56% of cases falling within ±20% of the actual stabilization dose. RFR performance is also superior to the rest of the models with “normal” dose requirements (MAE = 2.91 mg/week). Support vector regression (SVR) outperforms the others in the “sensitive” group, with a lower MAE of 4.79 mg/week. Finally, multivariate adaptive splines (MARS) performed best in the resistant group (MAE = 7.22 mg/week) with 66.7% of predictions within ±20%. Models generated by the RFR, MARS, and SVR algorithms predicted weekly warfarin dosing significantly better than other algorithms in the studied cohorts.

Drug-drug interactions
Mei S, 2021⁴³	China	Based on potential drug perturbations on associated genes and signaling pathways, an attempt was made to simplify computational modelling for drug-drug interaction prediction.	Database study	Only drugs that have been discovered to target at least one human gene were represented in the drug target profile.	The SP, SE, and MCC metrics on the two classes show that the proposed framework is less biased, with 0.9556 on the positive class, 0.9402 on the negative class, and 0.9007 overall MMC. These findings show that a drug target profile alone can accurately separate interacting drug pairs from non-interacting drug pairs (accuracy = 94.79%).
Van Laere S, 2022⁵⁰	Belgium	To compare the performance of conventional statistical methods (CSM) and machine learning techniques (MLT)	Database study	Retrospective data of 512 and 102 drug-drug interactions with possible drug-induced QTc prolongation	In a hold-out dataset, random forest and Adaboost classification performed best, with an equal harmonic mean of sensitivity and specificity (HMSS) of 81.2% and an equal accuracy of 82.4%. Both sensitivity and specificity were high (respectively 75.6% and 87.7%). All CSM performed similarly, with HMSS ranging from 60.3 to 66.3%. The logistic regression overall performance was 62.0%. In terms of predicting drug-induced QTc prolongation, MLT (bagging and boosting) outperformed CSM.
Song, D, 2018⁵¹	China	To develop a machine learning model using support vector machines (SVMs) based on a previously reported set of similarity measures and extensive training data sets.	Database study	DrugBank provided 10,705 DDI that were associated with 1162 drugs.	The predictive performance of AUROC the 10-fold cross-validation studies was >0.97, which is significantly better than the AUROC of 0.67 of an analogously developed machine learning model. The pairwise kernel SVM model outperformed previous works in terms of accuracy, and it can be used as a pharmacovigilance tool to detect potential DDI.

Electronic Health Records
Dalal AK, 2019⁵⁴	USA	To describe the experience of the systems engineering (SE) and human factors (HF) core team to support individual projects during each phase of a suite of novel digital health tools integrated with the electronic health record (EHR) across the 5 phases of AHRQ's SE lifecycle: problem analysis, design, development, implementation, and evaluation	Case report	Patient Safety Learning Laboratory (PSLL) members	Of the 29 participants, 19 and 16 took part in surveys and focus groups about their perceptions of SE and HF, respectively. Over the course of the four-year project, we identified seven themes in the application of the 12 SE and HF methods. Qualitative methods (interviews, focus groups, observations, and usability testing) were the most used, typically by individual project teams, and produced the most insight. The SE and HF core teams typically used quantitative methods (failure mode and effects analysis, simulation modelling), but the results were variable.
Balestra M, 2021⁵⁶	USA	To develop a predictive model for identifying orders that require intervention based solely on the ordering provider's interactions with the EHR.	Database study	Data from the EHR system on provider actions and pharmacy orders	In both the area under the receiver-operator (AUROC) and precision-recall (AUPR) curves, the XGBoost algorithm outperformed both logistic regressions and the random forest algorithm by a significant margin. The area under the receiver-operator characteristic curve was 0.91, and the area under the precision-recall curve was 0.44.

Potentially inappropriate medications
Xingwei W, 2022⁶⁰	China	To evaluate the data on potentially inappropriate prescribing (PIP), potentially inappropriate medications (PIM), and potential prescribing omissions (PPO) in elderly patients with cardiovascular disease, and to develop a prediction platform using multiple machine learning algorithms to predict the risk of PIP, PIM, and PPO in elderly patients with cardiovascular disease.	Retrospective analysis	This study included participants who were discharged from the Department of Geriatric Cardiology at Sichuan Provincial People's Hospital between January 2017 and June 2018.	The study included 404 patients in total (318 [78.7%] with PIP; 112 [27.7%] with PIM; and 273 [67.6%] with PPO). Following data sampling and feature selection, 15 datasets were obtained, and 270 risk warning models based on them were built to predict PIP, PPO, and PIM, respectively. The AUCs of the best model for PIP, PPO, and PIM were 0.8341, 0.7007, and 0.7061, respectively, according to external validation. The findings indicated that angina, the number of medications, the number of diseases, and age were the most important factors in the PIP risk warning model. The risk warning platform was developed to predict PIP, PIM, and PPO, with acceptable accuracy, prediction performance, and clinical application potential.
Tai, C.-T, 2020⁶¹	Taiwan	To predict the risk of high-alert medication treatment (digoxin) using machine-learning techniques	Retrospective analysis	This study included patients who had accepted digoxin therapy while hospitalized between January 2004 and December 2013.	AUC values ranged from 0.551 to 0.836. The RF classifier performed the best (0.836; excellent discrimination), followed by C4.5 (0.719) and ANN (0.688); the remaining classifiers performed poorly. This study found that machine-learning techniques can improve prediction accuracy for high-alert medication treatment, lowering the risk of ADEs and improving medication safety.
Wongyikul P, 2021⁶²	Thailand	To develop a novel approach that employs machine learning models to predict the appropriateness of high alert drugs (HAD) use for a specific patient visit.	Retrospective analysis	Patient data from the Maharaj Nakorn Chiang Mai Hospital's outpatient and inpatient departments in 2018	The machine learning algorithm identified over 98% of actual HAD mismatches in the test set and 99% in the evaluation set when screening drug prescription events with a risk of HAD inappropriate use. This study demonstrates that machine learning plays an important role in screening and reducing errors in HAD prescriptions.
Patel J, 2021⁶³	USA	To examine the prevalence and leading predictors of potentially inappropriate NSAIDs use among older adults with OA using real-world data from nationally representative commercial health insurance claims with the help of machine learning approaches.	Retrospective cohort study	Older adults with OA were identified using one inpatient or two outpatient claims at least 30 days apart that consisted of OA diagnosis codes (ICD-10 codes M15–M19) during the baseline year and were required that these adults be enrolled in Medicare Advantage plans with medical and pharmacy benefits during 2015 and 2016 (i.e., 24 months).	XGBoost and CVLR- both models had an AUROC value of 0.92 (95% CI: 0.91–0.93) and 0.91 (95% CI: 0.90–0.92), respectively. While both models had similar accuracy and specificity, CVLR had better precision (0.83 vs. 0.81). On the other hand, XGBoost performed better on all other metrics being compared, including recall, F1 score, and kappa statistic.

Medication adherence
Brath H, 2013⁶⁸	Austria	To test a remote medication adherence measurement system (mAMS) based on mobile health (mHealth) in elderly patients with high cardiovascular risk who were being treated for diabetes, high cholesterol, and hypertension.	Randomized single-blinded (doctor blinded), controlled, single centre study with crossover design	150 patients with a known risk of cardiovascular disease (Type 2 diabetes, hypertension, hypercholesterolemia)	A comparison of medication adherence in the monitoring and control phases for the four different medications revealed a significant difference in metformin intake (P = 0.04) favouring the MON phase. This result did not consider the two study groups separately. There was no significant difference between the other three medications.
Wiegratz I, 2015⁶⁹	Five European countries (France, Germany, Italy, Spain, UK)	To assess the effect of an acoustic alarm function on adherence to ethinylestradiol (EE) 20 g/drospirenone 3 mg in a flexible extended regimen (EE/drospirenoneFlex) among women seeking oral contraception in five European countries (France, Germany, Italy, Spain, and the United Kingdom).	Randomized, parallel-group open-label study	Women between the ages of 18 and 35 (smokers up to the age of 30) in good general health who want contraception	Dispenser data revealed a daily delay in pill release of 88 (126) minutes in group A vs 178 (140) minutes in group B (P < 0.0001). The median (lower quartile, Q1; upper quartile, Q3) number of missed pills in group A was 0 (0; 1) vs 4 (1; 9) in group B (P < 0.0001). The results of the diary cards revealed similar trends; however, underreporting of missed pills was evident in both groups. During the 424 woman-years of exposure, no pregnancies were reported. The mean (SD) EE/drospirenoneFlex cycle length was 51.0 (31.8) days across the two groups, with significant regional differences, and the mean (SD) number of bleeding/spotting days was 50.4 (30.0). EE/drospirenoneFlex was well tolerated, with 80% of women satisfied with the treatment.
Wang R, 2014⁷⁰	USA	To assess how wireless wearable devices equipped with a tri-axial accelerometer can be used to detect and classify user hand gestures during solid-phase medication administration.	Prospective observational study	Twenty-five subjects, aged 21 years and older	Using hand gesture signals, the true positive rate was 84.17% and the false alarm rate was 13.33%, demonstrating that hand gestures could be used to effectively identify pill taking activity.
Bilodeau GA, 2011⁷¹	Canada	To develop and test a computer vision system for monitoring medication intake in the context of home care services.	Prospective observational study	Not mentioned	Consistently low false positive and false negative values for skin detection was obtained. The algorithm struggled most with the Guillaume and PierLuc video sequences. Again, the results for face and hand tracking were generally good. TPface and TPhands had high values of 98% and 94%, respectively.
McCall C, 2010⁷²	USA	To develop an economical and marketable RFID-based Medication Adherence Intelligence System (RMAIS) that will allow patients to adhere to prescribed medication schedules with minimal effort.	Prospective study	Not mentioned	By reminding a patient of the prescribed time for medication and dispensing it in a fully automatic and error-proof manner, the system is patient-centered and user-friendly. It is a novel motorized rotation platform design with the smooth integration of a scale, an RFID reader, and the rotation platform. This system also includes an Internet-based notification function that alerts the patient when it is time to take medicine and reports deviations from the prescribed schedule to primary care physicians or pharmacists.
Shtrichman R, 2018⁷³	Israel	To assess ReX feasibility through human factor studies that include assessing ReX safety, acceptance, and usability, as well as ReX efficacy in providing pills according to a preprogrammed dose regimen, managing reminders and adherence data, and increasing adherence rate compared to the standard of care.	Self-controlled study	59 human subjects (29 males and 30 females) ranging in age from 18 to 92 years.	81% (48/59) of subjects rated the ReX device as simple to use. The 4-day home-use study assessed the ReX system's safety, efficacy, and usability. There was no adverse event; no pill overdose or pill malformation was reported. Overall adherence in the ReX test was 97.6% versus 76.3% in the control test (P < 0.001). In the event of a missed pill, real-time, personalized reminders contributed to 18.0% of doses taken during the ReX test. 87% (35/40) of subjects found the ReX system simple to use, and 90% (36/40) felt comfortable using it for medication.

Medication errors
Segal G, 2019⁷⁹	Israel	To assess the precision, validity, and clinical utility of medication error alerts generated by a novel system that employs outlier detection screening algorithms.	Prospective study	Patients admitted to Sheba Medical Center's single 38-bed internal medicine department	The system's alert burden was low, with alerts generated for only 0.4% of all medication orders. 60% of the alerts were raised after the medication had already been administered due to changes in the patients' status that necessitated medication changes (eg, changes in vital signs). 85% of the alerts were clinically valid, and 80 % were clinically useful. 43% of the alerts resulted in changes in subsequent medical orders.
Dos Santos HD, 2018⁸⁰	Brazil	To develop a Density-Distance-Centrality (DDC) unsupervised method for detecting potential outlier prescriptions.	Database study	Dataset containing 21 different medications prescribed at Hospital Nossa Senhora da Conceic¸ao	When compared to other methods for detecting overdose and underdose in medical prescriptions, this approach yielded better results. Furthermore, most of the false positives detected by the algorithm were potential prescription errors.
Nagata K, 2021⁸²	Japan	To detect extreme overdose and underdose prescriptions that occur very rarely in clinical practice using unsupervised machine learning algorithms.	Retrospective analysis	Retrospective analysis	The model identified 27 out of 31 clinical overdose and underdose prescriptions as abnormal (87.1%). The OCSVM models developed performed well in detecting synthetic overdose prescriptions (precision 0.986, recall 0.964, and F-measure 0.973) as well as synthetic underdose prescriptions (precision 0.980, recall 0.794, and F-measure 0.839). In a comparative analysis, OCSVM performed the best. The models correctly identified the majority of clinical overdose and underdose prescriptions and performed well in synthetic data analysis.
Yalçın N, 2023⁸³	Turkey	To develop models that predict the presence of medication errors (MEs) (prescription, preparation, administration, and monitoring) using machine learning in NICU patients.	Randomized, prospective, observational cohort study	Neonates admitted to a 22-bed capacity NICU in Ankara, Turkey, between February 2020 and July 2021.	The prevalence (the ratio of drug errors) was comparable between the train and test sets (64% for the train set and 59% for the test set). The performance measures were calculated as follows: accuracy 0.919 (95% CI 0.858–0.956), sensitivity 0.918 (95% CI 0.844–0.964), specificity 0.922 (95% CI 0.829–0.973), PPV 0.944 (95% CI 0.884–0.974), NPV 0.887 (95% CI) 0.804–0.937), AUC 0.920 (95% CI 0.876–0.970), and F 1 score 0.931. A higher AUC indicated that the model correctly classified 92% of the patients as having physician- or nurse-related MEs.
Corny J, 2020⁸⁴	France	To test the accuracy of a hybrid clinical decision support system in prioritizing prescription checks to improve patient safety and clinical outcomes by lowering the risk of prescribing errors.	Retrospective analysis	Retrospective analysis	The pharmacist analyzed 412 individual patients (3364 prescription orders) in an independent validation dataset, our digital system's areas under the receiving-operating characteristic and precision-recall curves were 0.81 and 0.75, respectively, demonstrating greater accuracy than the CDS system (0.65 and 0.56, respectively) and multicriteria query techniques (0.68 and 0.56, respectively).

Medication Therapy Management (MTM)
Kessler, S, 2021⁸⁵	USA	To evaluate the impact of a novel artificial intelligence (AI) platform that identifies members and provides decision support to clinicians performing telephonic interventions similar to MTM and CMM with high-risk Medicaid members on actual medical claims.	Retrospective observational study	2150 Medicaid members, primarily middle-aged (aged 40–64 years), with an average of 10 chronic condition medications among a total of 25 medications.	Receiving interventions was found to have statistically significant correlations with lower costs and utilisation. The economic study discovered a 19.3% reduction in the TCoC (P < 0.001), which, when applied to a preintervention monthly cost of $2872, resulted in a $554 per member per month savings (PMPM). Medication costs were reduced by 17.4% (P < 0.001), resulting in a savings of $192 PMPM when compared to the preintervention cost of $1110. The utilisation study discovered a 15.1% decrease in ED visits (P = 0.002), a 9.4% decrease in hospital admissions (P = 0.008), and a 10.2% decrease in bed days (P = 0.01). Based on TCoC savings and programme costs, the return on investment is 12.4:1.
Bu F, 2022⁸⁶	China	During the COVID-19 pandemic, to establish an internet hospital pharmacy service mode based on artificial intelligence (AI) and provide new insights into pharmacy services in internet hospitals.	Prospective study	Users who benefit from Shanghai medical insurance settlement.	The AI preview qualified rate was 83.65%. Among the 16.35% of inappropriate prescriptions, 49% were accepted and modified proactively by physicians, while 51% were passed after pharmacists intervened. For collecting their medication in the internet hospital, 86% of patients preferred the “offline self-pick-up” mode, which allowed the QR code to be fully utilized. There were 426 medication consultants served, with 48.83% of them consulting outside of working hours. As a result, when pharmacists were unavailable, an AI-based medication consultation was proposed.

Telemedicine
Schiff GD, 2019⁹⁰	USA	To evaluate a novel interactive voice response (IVR) platform for detecting patient-reported symptoms.	Cluster randomized controlled trial	Adult primary care patients seen at Brigham and Women's Hospital and North Shore Physician's Group practices.	320 patients were transferred to the pharmacist and discussed 1021 potentially drug-related symptoms based on positive symptom responses or requests to speak with a pharmacist. Of these, 188 (18.5%) were determined to be probably related to the medication, while 479 (47.1%) were determined to be possibly related to the medication. Intervention patients were significantly more likely than control clinic patients to have adverse effects documented in the medical record by a physician (277 vs. 164 adverse effects, p < 0.0001, and 177 vs. 122 patients discontinued with documented adverse effects, p < 0.0001).