Prevalence and Clinical Significance of Discrepancies within Three Computerized Pre-Admission Medication Lists

Abstract

Inaccurate records of pre-admission medication exposure have been identified as a major source of medication error. Authors collected records of patients’ pre-admission medications: 1) the most recent outpatient medication list (“EMR”), 2) the medication list recorded by admitting providers (“H&P”), and 3) a list generated by a medication reconciliation process conducted by nursing staff (“PAML”). Forty-eight sets of pre-admission records composed of 1087 medication entries were compared to a reference standard generated by trained study staff conducting an independent interview. Sensitivity was greatest for PAML (85%), compared to EMR (76%) and H&P (76%) sources. However, positive predictive value was greatest for the H&P source at 96% vs 88% and 91% for PAML and EMR sources respectively. Potentially harmful medication discrepancies were found within all lists. The authors concluded no single list was sufficiently accurate to avoid serious medication errors.

Introduction

Medication errors have been linked to transitions in care, such as admission or discharge from the hospital ¹. Errors at these “interfaces” of health care frequently result from poor communication or inadequate documentation of medication exposures. Failure to correctly construct a complete medication history can delay recognition of adverse drug events, cause under- and over-dosing, duplicate therapy, and lead to omissions of therapy ². In 2005, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) mandated that an explicit medication reconciliation process occur at the time of admission to an acute care facility within accredited organizations³. The authors’ institution, Vanderbilt University Hospital (VUH), implemented the JCAHO medication reconciliation process in 2006 using a new, web-based computer application. As a result, VUH had three potentially authoritative sources of pre-admission medication data: the most recent outpatient EMR medication list (“EMR”), the current medication list recorded by the admitting physician (“H&P”), and a list generated by the medication reconciliation process (“PAML”). Authors studied the accuracy of the three sources of pre-admission medication data by comparing each to a “gold standard” comprehensive medication history created by trained study staff. Subsequently, we categorized medication discrepancies with potential for harm using two physician reviewers.

Methods

Setting:

Vanderbilt University Hospital (VUH) is an academic, tertiary care facility with 832 adult beds and locally developed and maintained inpatient CPOE and inpatient/outpatient EMR systems.

Medication Data Sources:

1) Electronic Medical Record (EMR): Clinical staff maintain medication lists within an EMR-based universal problem list at all VUH inpatient and outpatient settings. Medication entries up to the time of the study were manually entered in free-text; a new locally developed outpatient electronic prescription writer that produced structured pharmacy data had just begun operation prior to the study.

2) Admission History and Physical (H&P): Inpatient H&P notes are dictated or typed within 24 hours of admission and appear within the EMR as electronic text documents with embedded medication lists. Clinicians generating an H&P can import medication list information directly from the EMR list (#1 above), but institutional policy and common practice require providers to at a minimum perform appropriate editing, or to collect data independently.

3) Computerized Medication Reconciliation to create a Pre-Admission Medication List (PAML): a web-based application accessible within the care provider order entry (CPOE) system, the EMR, and the Emergency Room Triage (“ED Triage”) application, facilitated the entry of medication name, dose, frequency, route, and approximate duration of exposure at the point of hospital admission. Previous published design specifications informed the features and architecture of the application⁴. Medication entries could be imported from the outpatient list (EMR) and from any previous discharge mediation list. The tool was implemented and mandated for use by the nurse staffing the Emergency Department triage (ED), or by nursing staff performing an intake assessment for a direct hospital admission.

Study Design:

The overall design was a prospective cohort study. Patients were randomly sampled from the census of recent (<24 hours) admissions to VUH using an electronic patient enrollment tool. The tool enforced the following inclusion criteria: age ≥18, admission to a medical or non-obstetric surgical service, admission to a non-intensive care unit, and prior completion of a reconciled medication list (using the computerized PAML tool). Study personnel selected patients sequentially from the enrollment tool and called the patient’s nurse to obtain permission to approach the patient to consent for an interview. Comatose, non-communicative (e.g. severe dementia) or non-English speaking patients were excluded, unless a surrogate was available who had managed all pre-admission medications for the patient.

Medication History Interview:

After informed consent, trained study personnel (two trained pharmacists and a senior medical resident MD) conducted a detailed interview with the patient and any family members the patient identified as assisting with home medication management. The objective was to identify all prescription and over-the-counter medications taken by the patient within the previous 7 days prior to admission. Intake of herbal medications or other alternative therapies were not assessed as part of the interview. All medication bottles brought to the hospital by the patient or family were reviewed. Study members obtained with permission faxed refill records from the patient’s outpatient pharmacies used within the previous 6 months.

Creation of Reference Standard Medication List:

Study personnel recorded all medication data on a secure, web-based data collection form. The reviewers explicitly identified discrepancies between sources (e.g. entries on one list not included on another list) and resolved all inconsistencies through interaction with the enrolled patient on a second in-person visit. The “Reference Standard List” contained all medications confirmed by the patient to have been taken at least once in the 7 day window prior to admission. All other medication entries, including those where exposure was uncertain, were excluded. For patients who did not personally organize or self-dispense medications (e.g. a pillbox was maintained by family member), the responsible care giver provided all the reconciliation information.

Outcome assessment:

The primary outcome, medication discrepancies, was defined as omissions (medications in the Reference Standard set, but missing on at least one of the three electronic lists) or false positive entries (medications on one of the three electronic lists not present on the Reference Standard). Dose, route, and frequency discrepancies were not considered for this analysis. Discrepancies were classified according to potential for harm and severity of harm by adapting a previously published taxonomy for errors of inpatient medication reconciliation ⁵. Specifically, reviewing physicians (JD, MM) determined using a 6-point confidence scale whether the medication discrepancy had the potential to cause patient harm if the discrepancy was not intercepted and corrected by the admitting provider on admission or discharge. Secondly, the reviewers determined the potential severity of harm, if it were to occur, on a 3 point scale (significant, serious, and life-threatening). The study defined “serious medication errors” as having a prespecified score of 5 or 6 which corresponded to “strong confidence” and “virtually certain confidence” combined with a severity score of “serious” or “life-threatening”. Reviewer disagreements for both potential for harm and severity of harm were resolved in a consensus conference with a third physician reviewer (JP) acting as tie-breaker. Dose, frequency, and route discrepancies are not reported. Study MDs reviewed all discrepancies involving omissions or false-positive entries of over-the-counter medications (excluding non-steroidal anti-inflammatory drugs – NSAIDs) and of topical medications and judged them as a class to be non-serious.

Analyses:

The study determined the number and type of medication discrepancies for each of the three electronic sources of medication lists. Entries where the patient was uncertain of pre-admission exposure were not analyzed (n=12 of 1087). Derivation of sensitivity and positive predictive value involved comparing entries on each electronic medication list with the Reference Standard. Comparisons between test characteristics utilized McNemar chi-square test. Agreement between physician reviewers prior to consensus scoring was represented with the kappa statistic. The study was approved by the local institutional research board (IRB).

Results:

Forty-eight patients met inclusion and exclusion criteria and completed an in-person interview. Two patients were unable to answer questions directly but an eligible surrogate was available. Fifteen additional family members involved in the organization or delivery of outpatient medications to the patient including 11 spouses, and 4 other family members were also interviewed. Twelve patients (25%) brought medication bottles to the hospital for review, and an additional 12 (25%) provided a self-maintained medication list.

Similar to the general VUH census, the majority of patients was Caucasian, and attained a minimum of a high school education (Table 1). The number of outpatient prescribing physicians per patient varied widely from 0 to 7 with a median of 1. A minority of patients (29%) were followed in primary care outpatient practices affiliated with VUMC. Slightly less than half (44%) of outpatient pharmacies responded to repeated requests for refill records.

Table 1:

Characteristics of Patients (n=48)

Characteristic	Result
Age, mean ± SD	58 ± 16
Female	54%
Race
Caucasian	88%
African-American	12%
Highest Education
Grade school	8%
High school	54%
College	23%
Graduate school	12%
Family Member Interviewed	31%
Pharmacy Refill Records Obtained	44%
Medication Bottles Reviewed	25%
Patient Medication List Reviewed	25%
Primary provider at VUMC	29%
Outpt. Providers, median (range)	1 (0,7)

Medication Discrepancies

Among 1087 medication entries across all medication sources, 360 (33%) were found to be discrepant with the Reference Standard list. Discrepancies were clustered, with 43 of the 48 patient records found to have a median of 10 discrepancies (range: 1–27). The majority of discrepancies (74%) were omissions as opposed to false-positive entries. Discrepancies within PAML and EMR sources were more commonly false positive entries compared to the H&P list. The majority of discrepancies were prescription medications, although over-the-counter drugs comprised a larger proportion of discrepancies in the EMR list. Table 2 presents discrepancies associated with high-risk drug classes. Low-risk medications of various classes are grouped in the “Other” category.

Table 2:

Characteristics of Medication Discrepancies Across Three Medication Lists

	PAML (n=392 entries)	H&P (n=331 entries)	EMR (n=364 entries)
Discrepancies, n	100	116	144
Type, n (%) of discrepancies
Omission	64 (64)	102 (88)	99 (69)
False Positive	36 (36)	14 (12)	45 (31)
Source, n (%) of discrepancies
Over-the-counter	29 (29)	41 (35)	62 (43)
Prescription	71 (71)	75 (65)	82 (57)
Drug Class, n (%) of discrepancies
Antimicrobials	8 (8)	15 (13)	19 (13)
Anticoagulants	5 (5)	3 (3)	6 (4)
Anti-platelets	2 (2)	3 (3)	3 (2)
NSAID	6 (6)	7 (6)	9 (6)
Sedative/hypnotics	4 (4)	6 (5)	10 (7)
Opiates	6 (6)	5 (4)	13 (9)
Hypoglycemics	1 (1)	2 (2)	4 (3)
Antihypertensives	9 (9)	3 (3)	12 (8)
Other	59 (59)	70 (61)	73 (51)
Potential for Harm, n (%) of discrep.
Low (score ≤ 4)	93 (93)	106 (91)	118 (82)
High (score ≥ 5)	7 (7)	10 (9)	26 (18)
Severity, n (%) of discrepancies
Significant	82 (82)	93 (80)	96 (67)
Serious	15 (15)	17 (15)	40 (28)
Life-threatening	3 (3)	6 (5)	7 (5)
Serious Medication Errors n (%)**	8 (7)	10 (9)	21 (18)
Comparison to Reference Standard
Sensitivity, % (95% CI)	85 (81–88)	76 (71–80)	76 (72–80)
PPV, % (95% CI)	91 (88–94)	96 (93–98)	88 (84–90)

^*Potential for harm adjudicated using 6-point confidence scale;

^**Serious medication errors defined as the intersection of high potential for harm and serious or life-threatening severity; PAML = Pre-admission Medication List; H&P = History and Physical; EMR = Electronic Medical Record; NSAID = non-steroidal anti-inflammatory drug; PPV = Positive predictive value

While discrepancies between one of the three lists and the Reference Standard were frequent, fewer medication entries were discrepant across all three lists. Twenty-five medications reported by patients to the study interviewers were omitted on all lists, while 6 medications were false-positives on all three lists. An additional 68 medications were either omitted or a false-positive on both of the two admission lists: the PAML and H&P.

Serious Medication Errors

The two physicians who reviewed all medication discrepancies for potential for harm initially disagreed on 28 (7.8%) of 360 potential for harm ratings (dichotomized between scores 1–4 and 5–6) and 41 (11%) of severity ratings. The resulting kappa values were 0.27 and 0.61 respectively. After final adjudication, 7%, 9%, and 18% of discrepancies were judged to be serious medication errors for PAML, H&P, and EMR sources respectively. The overall error rates using total medication entries as a denominator were 2.0% (PAML), 3.0% (H&P) and 5.8% (EMR). Serious medication errors were largely related to discrepant entries regarding the anticoagulants enoxaparin and warfarin in the setting of major surgery or a procedure (n=5), omitted records of pre-admission antibiotic exposure in the setting of major infections (n=8), omission of clonidine during admissions for hypertension (n=1), omission of antiviral therapy for HIV (n=5), and discrepant recording of insulin (n=3).

Comparison to the Reference Standard List

The H&P and PAML lists routinely generated during admission demonstrated differing sensitivity and positive predictive value when compared to the Reference Standard. The sensitivity (probability that a medication exposure was documented) of the PAML was higher at 85% vs. 76% for H&P (p<0.001), while the positive predictive value (probability an entry on a list was a true medication exposure) was lower for PAML at 91% vs 96% for H&P (p<0.001).

Discussion:

Accurate documentation of pre-admission medications is designed to avoid three classes of medication errors: 1) omission of clinically important therapies while the patient is hospitalized, 2) omission of therapies which should be resumed at discharge and 3) a false assumption of medication exposure where none has occurred. As demonstrated by this analysis, none of the three routinely collected, electronically available medication lists approached the accuracy of the Reference Standard generated by in-depth medication history taking. All of the lists demonstrated a low but significant rate of serious medication errors that could potentially cause patient harm. Each medication list source proved to have significant weaknesses. The PAML process produced a list with higher sensitivity (fewer omissions), but generated more false positive entries when compared to the H&P. One potential explanation is that medication reconciliation was implemented as a computerized application, which allowed importation of other medication lists including the outpatient EMR list and the discharge medication list from a previous admission. Such a feature may lead to entries that are not carefully cross-checked with other medication data sources. Additionally, the medication reconciliation process at the study institution occurs upon initial evaluation of patient acuity by nursing staff; their numerous other clinical care responsibilities may detract attention from the task. Time pressures are also heightened in ED triage where the goal is to complete the triage process in less than 5 minutes. A measure of the time required to complete the Reference Standard list for each patient was not completed for the study but was estimated by the participating pharmacists to be approximately 60 minutes.

Admitting providers appeared to generate more accurate H&P entries than the PAML, but providers either missed pre-admission medications during history taking or patients were less forthcoming with recent compliance. Regarding the false positive entries, providers generating the H&P may have chosen to list medications that represented what the patient should have been taking or medications taken more than one week prior to admission. Such differences in objectives could not be evaluated in this study because the clinicians who generated H&P and PAML lists were not interviewed. The lower performance of the EMR list generated by outpatient clinicians was expected since it was typically generated prior to the patient presenting for acute evaluation.

In 2005–2006, JCAHO mandated medication reconciliation for every acute care admission but has since removed the procedure from hospital accreditation scoring due to a lack of proven strategies. Few published studies have examined how well the process works and how it compares to the previous standard of care, an admitting provider generated history and physical exam note. A recent randomized study of a computerized PAML application showed a reduced risk of potential adverse drug events (RR 0.72)⁶. The benefit was restricted to one of the two studied hospitals. As the authors point out, successful use of the PAML application likely relied on additional technical (communication with the discharge application) and socio-technical (training and publicity) factors present in one hospital and not in the other.

Medication reconciliation processes are a natural fit for clinical informatics, yet the application domain remains problematic. Even in highly computerized and interconnected environments, disparate systems may store partially overlapping, incomplete medication information, especially at care transition points. In most settings, reconciliation remains a largely manual process, although software features to improve reconciliation accuracy (for example to highlight list differences or recent changes) are conceivable. Previous studies have demonstrated that pharmacists are the most capable profession for conducting accurate medication histories. However, accurate generation of a medication list needs to be balanced with timely generation of a pre-admission medication exposure list, and using triage nursing staff ensures a list is available immediately to assist ED providers and admission teams in making initial treatment decisions. A refined process should allow triage staff to highlight PAML elements needing clarification for subsequent clinical teams.

Several limitations are apparent. The study was conducted at an academic, tertiary care medical center with internally developed information systems, and the findings may not generalize to all settings. The number of discrepancies may be higher or more clinically significant in other settings where there is no central repository of medication related records or no technical ability to integrate different information sources. Secondly, there is no absolute gold standard for medication reconciliation as it relies on patient recall, which is imperfect. The study may have misclassified omissions and false positives. However, the Reference Standard construction used methods similar to other published studies⁶ and the study methods successfully discovered medication exposures not documented in the clinical sources evaluated. Finally, the agreement between reviewers was low prior to the consensus conference due to the difficulty in estimating the future likelihood of harm. The final adjudication adhered to the guidelines and examples described in previous medication error publications.

In conclusion, a computerized PAML application and a provider-generated H&P both failed to generate a highly accurate pre-admission medication list. New approaches incorporating technical innovations and human expertise are needed to integrate multiple, potentially discrepant medication lists.