Table 3.

Classifications of discrepancies between manual and automated detection of medications. Issues were generally categorized into one of 3 groups: human error, ETL or mapping error, and abstraction-query mismatch. Descriptions and examples of each issue are provided.

Common Issues	Description	Example
Human Error
Abstractors overlook desired information (false negative)	In 16% of discrepancies, the manual abstractors overlooked a medication that should have been included based on instructions	Patient received hydroxychloroquine during admission, but was categorized by the manual abstractor as not having received hydroxychloroquine
Abstractors include inappropriate information (false positive)	In 10% of discrepancies, complex drug classes or questions led manual reviewers to classify patients as having been exposed to a medication when they were not	Patient classified by manual abstractor as exposed to NSAIDs despite only receiving acetaminophen (a non-NSAID drug) during hospitalization
ETL/Mapping Error
Missing data leading to query error	In 31% of discrepancies, data missing in the EHR led to the query incorrectly categorizing patients as having continued exposure to a given drug	Outpatient medications were only included by manual abstractors if the patient was exposed based on admission documentation, but many orders in the outpatient EHR lack end dates, requiring further work for appropriate automated calculation
Local errors	In 5% of discrepancies, issues with missing reference terminology in source systems caused failure to detect some medications during automated extraction of data	Remdesivir and sarilumab were not coded to RxNorm vocabulary due to investigational status and exposures to these drugs were not detected in the automated extracted data
Patient identifier inconsistency	In 4% of discrepancies, patient identifiers were either missing or incorrect, leading to discrepancies in specific drug exposures	Two patients shared the same enterprise master patient index, resulting in conflation of their data
Cross institutional differences	In 1% of discrepancies, data were not mapped correctly between differing hospital campuses, which led to incorrectly classified drug classes in the data extracted by automated methods	The formulary from one hospital was mapped to the formulary from another, yielding incorrect classes for some drugs
Abstraction-Query Mismatch
Mismatch between query and data format	In 30% of discrepancies, for inpatient medications where duration of administration was important, the query overlooked medications that were ordered daily as opposed to ordered continuously as order duration was used to measure duration of administration.	Diuretics were commonly ordered as single doses each day, thus although a patient could receive diuretics for consecutive days, the query only detected doses as having a 24-hour duration when instructions for manual abstraction asked for a minimum 48-hour duration.
Complex instructions/confounding medications	In 3% of discrepancies, lack of clarity in some special instructions for specific medication categories created challenges in developing the query	Manual abstractors were instructed to only capture protease inhibitor exposure if the drug was part of an HIV regimen – the automated extraction method and query did not take this into account. Confounding medication names also led to inappropriate inclusion.