Accuracy of the medication list in the electronic health record—implications for care, research, and improvement

Abstract

Objective

Electronic medication lists may be useful in clinical decision support and research, but their accuracy is not well described. Our aim was to assess the completeness of the medication list compared to the clinical narrative in the electronic health record.

Methods

We reviewed charts of 30 patients with inflammatory bowel disease (IBD) from each of 6 gastroenterology centers. Centers compared IBD medications from the medication list to the clinical narrative.

Results

We reviewed 379 IBD medications among 180 patients. There was variation by center, from 90% patients with complete agreement between the medication list and clinical narrative to 50% agreement.

Conclusions

There was a range in the accuracy of the medication list compared to the clinical narrative. This information may be helpful for sites seeking to improve data quality and those seeking to use medication list data for research or clinical decision support.

Introduction

With the surge of distributed research networks such as the national Patient-Centered Clinical Research Network, the Accrual for Clinical Trials Network,^1–3 and other learning health systems which review advance population health through research and quality improvement,^4–6 there is a growing need to aggregate and leverage data from the electronic health records (EHRs) of patients from multiple care centers. Within these networks EHR data are used for many different purposes, including cohort identification, patient phenotyping, observational, and comparative effectiveness research,⁷ process and outcomes measurement,^8,9 clinical decision support, and patient care management through activities like pre-visit planning.¹⁰

The large-scale adoption of EHRs and computerized physician order entry means that most healthcare centers now have a digital record of the medication orders for each patient. While the medication list is generated through the course of clinical care, having this data in a discrete format provides an opportunity for data utilization in the activities described above. The medication list is a list, within the ambulatory clinic EHR medical record, of all prescription and over-the-counter medications the patient takes generated by new prescriptions in addition to medications the patient reports taking at home. Before using the medication list in this way; however, it is important to understand the accuracy and completeness of the information. This is especially important as the contents of the medication list can also be accessed through messaging standards such as the Consolidated Document Architecture,¹¹ and the Fast Healthcare Interoperability Resource,¹² which support the interoperable exchange of EHR data between organizations. These standards and interfaces are viewed as enabling large-scale decision support and analytics in support of a learning health system. Data is generated at the point of care and combined with other sources of information to drive the research and application of new knowledge to clinical practice. In order for this to occur, the information being exchanged must be accurate and complete, or the data’s fitness for use must be at least ascertained.^13,14

We sought to characterize the accuracy of the medication list to allow us to understand the quality of automated data, to inform clinical care, such as previsit planning, and research using medication data from the EHR. Additionally, we investigated if medication information from the EHR medication list replaced manual chart review, since this informs the use of the electronic medication list for decision support activities like previsit planning, as well as analytics and research. Our objective was to describe the agreement between a patient’s medication list and in the ambulatory clinical narrative among patients with inflammatory bowel disease (IBD). The clinical narrative includes all clinical notes generated during the clinical encounter, including nursing notes, physician progress notes, notes generated by other clinical teams (such as social work), and telephone encounters. We are not aware of any work that directly examines medication data in this manner. Previous studies on the quality of data within the medical record have been largely focused on the development of metrics to describe data quality and completeness,^15–20 with specific reviews finding the EHR data quality to often be poor and difficult to assess due to the underlying diversity of healthcare data types and settings.^21,22

Methods

Setting

The study was conducted in 6 outpatient clinical centers that volunteered to participate. All centers were large pediatric gastroenterology clinics affiliated with an academic medical center that participates in the ImproveCareNow quality improvement and research network.^9,23 ImproveCareNow is a 92-center learning network that seeks to improve the care and outcomes of children and adolescents with IBD. It is 1 of 21 Patient-Powered Research Networks participating in the Patient-Centered Clinical Research Network.¹ Four centers participating in our study utilize Epic as their institutional EHR, 1 uses Cerner, and 1 uses Allscripts.

Medical record review

The study team developed, pilot tested, and refined a written data abstraction protocol and data collection form. The participating centers were asked to complete four steps. First, each center selected 30 patients at random, using a random number generator, from their current ImproveCareNow patient population. Second, the data collection form was used to uniformly record the IBD medications on the patients’ outpatient medication list. The data abstraction protocol included a list of commonly used IBD drugs, such as mesalamine, mercaptopurine, methotrexate, and several biologics. In addition, the centers could make additions to the list if they routinely used a medicine for IBD that was not included. Third, the patient’s most recent office visit (visit to the outpatient gastrointestinal clinic at the academic medical center for their IBD) notes were manually reviewed and the medications listed in the narrative from the clinic visit were compared with those abstracted from the medication list. Medications that were part of the notes, but not on the medication list, or vice versa, were flagged on the data collection form. Care was taken to account for generic and brand name differences, as well as for any abbreviations. Fourth, centers repeated the third step for any encounters that occurred after the most recent office visit, such as a “Telephone”, or an “Orders Only” encounter.

Analysis

The primary outcome of interest was the percent of patients at each site with complete agreement between the medication list and the clinical narrative for all IBD medications. To quantify this, we counted the number of patients with complete agreement between the medication list and the clinical narrative (medication name, dose, and frequency agree) and divided by the total number of patients whose medical records were reviewed. The secondary outcome was the percent of medications with complete agreement between the medication list and the clinical narrative. To quantify this, we counted the number of IBD medications that were listed on both the medication list and clinical narrative (i.e., complete agreement) divided by the total number of IBD medications reviewed.

Results

A total of 379 IBD medications were noted in the any part of the medical record of 180 patients (30 patients at each site). Patients took an average of 2 IBD medications each. The number of medications reviewed varied from 51 to 68 medications per center.

There was variation by center with the top center having 90% patients with complete agreement on IBD medications between medication lists and clinical narrative, and the lowest ranked center having 50% agreement (Table 1). The percentage of patients within each center with a medication listed in the clinical narrative but absent from the medication list ranged from 7% to 20%. The percentage of patients with a medication on their medication list that was not in the clinical narrative ranged from 3% to 37%. At one study site, most (91%) medications were listed in the medication list and missing from the clinical narrative (71%). At the other sites, one was not more complete than the other. The top center had 92% of IBD medications included in both the medication lists and the clinical narrative and the lowest ranked center had 65% of IBD medications included in both locations (Table 2). The percentage of medications included in the clinical narrative but absent from the medication list ranged from 4% to 17%. The percentage of medications included on the medication list and not in the clinical narrative ranged from 4% to 28%.

Table 1.

The Number of Patients Who Perfect Agreement Between the Electronic Health Record Medication List and the Clinical Narrative on Which IBD Medications the Patient was taking (each center reviewed medical records of 30 patients)

Measure	Number of patients for each center, N (%)
Centers	1	2	3	4	5	6	Total
Perfect agreement between medication list and clinical narrative	27	22	15	24	22	21	131
	(90)	(73)	(50)	(80)	(73)	(70)	(76)
Patients with medication in clinical narrative not in med list	2	6	5	5	4	5	27
	(7)	(20)	(17)	(17)	(13)	(17)	(15)
Patient medication on med list not in narrative	1	5	11	1	2	5	16
	(3)	(3)	(37)	(3)	(7)	(17)	(9)

Table 2.

The Number of Medications Listed Both in the Electronic Health Record Medication List and the Clinical Narrative, and the Number not Listed in Both

Measure	Number of medications for each center
	N (%)
Centers	1	2	3	4	5	6	Total
# Medications	51	66	68	64	68	62	379
Perfect agreement between medication list and clinical narrative	47	48	44	47	59	53	298
	(92)	(73)	(65)	(73)	(87)	(86)	(79)
Medications in narrative not in list	2	10	5	11	4	5	37
	(4)	(15)	(7)	(17)	(6)	(8)	(10)
Medications on list not in narrative	2	8	19	6	5	4	44
	(4)	(12)	(28)	(10)	(7)	(6)	(11)

Some centers had one medication or class of medications that was missing from the majority of medication lists or clinical narratives, which caused their percent agreement to be relatively low. For one center, the missing medication was prednisone, and for the other center the missing medication was infliximab. For the center that routinely missed prednisone, if it had been included all of the time, their percentage of medication agreement would have been 90%.

Discussion

This is the first study, to our knowledge, to examine the accuracy of the medication list for use in large multisite research studies or clinical decision support activities. In our review of IBD medications at 6 clinical sites, we found a large variation in the agreement between the medication list and progress note across different centers within our network. Two centers had complete agreement for at least 80% of their patients. There were 3 centers with discrepancies in <15% of the medications in their data. These centers may be a resource for the other network centers in terms of identifying and adopting best practices. They may also be best prepared to participate in further network research and care improvement projects that use medication data. Some centers may be able to improve the accuracy of their lists by examining patterns in the missing information, such as having a single medication missing from most patients’ lists. For example, missing inflixamib information showing how a site compares to other sites may also be a motivation for improvement.

The lack of agreement between the medication list and clinical narrative regarding important IBD medications may also increase the risk for ambulatory medication errors. Rates of medication errors in children with chronic conditions are high.^24,25 While the exact level of accuracy needed for productive clinical decision support is not known, it seems reasonable to expect that the data should be accurate and complete for effective use in activities like previsit planning, given the toxicity of many medications particularly those used to treat chronic conditions such as IBD.^26,27 Otherwise, a situation results where care teams chose not to use the previsit reports because they do not trust the content or, in the worst case, patient harm occurs because the inaccurate data is trusted and used to make clinical decisions. The level of accuracy needed for a given research study is unknown, but it is reasonable to expect that centers with a higher percentage of agreement may be more desirable research sites. Percent agreement may then be a useful metric to inform project planning decisions, such as whether to verify electronic data with a manual chart review.

While this is a multisite study, it has some limitations. We did not attempt to understand which medications patients were actually taking at home in this medical record review. This is likely some combination of the medications listed in the progress notes and medication list. We also did not attempt to look at all the medications the patients were taking, but only those IBD medications relevant to ImproveCareNow. These patients took 1.7–2.3 IBD medications; this is less than we found in children with sickle cell disease and cancer in prior studies, but those studies included all medications, not just those to treat a single condition. This may bias the study toward higher rates of accuracy, assuming that the medication list is less accurate for other, less critical medications. While we did not attempt to assess the clinical significance of each missing medication, missing medications were all used to treat IBD, including remicaid, prednisone, inflixamib, and others. Finally, we included a sample of the centers which volunteered to participate. If centers with better agreement are more likely to volunteer, then this may have inflated the percent agreements. However, we still found variation in practice.

Conclusions

Our findings have important implications for deciding whether to solely rely on medication order data as a proxy for medication usage in research or clinical decision support. We found wide site-to-site variations in the accuracy and completeness of the patients’ ambulatory medication list when compared to the narrative portions of the ambulatory EHR. Our study may provide critical information to help sites improve their documentation practices in order to produce a more usable medication list both for quality improvement and research purposes. Information about which medications are missing from the record can be helpful, and information about how the site compares to other sites may provide motivation to change. Finally, this work should give pause to those who hope that the push for interoperability and the adoption of common standards and interfaces will allow external partners to make analytic or care decisions solely on medication order data. While that might eventually be the case, for the time being, this information should not be utilized without additional validation.

Funding

This work was supported through funding from the Agency for Healthcare Research and Quality (R01 HS022974) and by the participating centers of the ImproveCareNow Network (www.improvecarenow.org). Competing Interests Statement: the authors have no competing interests to declare.

Contributors

Dr KEW. led the conception or design of the work and the acquisition, performed the analysis, and led the interpretation of data for the work; and drafted; and gave final approval of the version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Dr KAM co-led the conception or design of the work and the acquisition, participated in the interpretation of data for the work, helped in drafting the work; and gave final approval of the version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Dr CD participated in the acquisition and analysis of the data, interpretation, and revising the work critically; and gave final approval of the version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. CD participated in the acquisition and analysis of the data, interpretation, and revising the work critically, and; and gave final approval of the version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. CD participated in the acquisition and analysis of the data, interpretation, and revising the work critically, and; and gave final approval of the version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. CD participated in the acquisition and analysis of the data, interpretation, and revising the work critically, and; and gave final approval of the version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. CD participated in the acquisition and analysis of the data, interpretation, and revising the work critically, and; and gave final approval of the version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. CD participated in the acquisition and analysis of the data, interpretation, and revising the work critically, and; and gave final approval of the version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. PM participated in the conception or design of the work and the acquisition, in the analysis, and the interpretation of data for the work; and drafting the work or revising it critically for important intellectual content; and gave final approval of the version to be published; and agrees to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.