1. INTRODUCTION
Medication discrepancies, defined as unintended differences between medication lists, 1 occur in up to 60% of patients' electronic health records (EHRs). 2 , 3 , 4 They are associated with adverse drug events and increased healthcare utilization 5 , 6 , 7 , 8 ; thus, the Joint Commission recommends medication reconciliation to resolve discrepancies between patient‐reported medications and those documented within the record. 9
Medication reconciliation tools to identify and correct discrepancies can provide patient safety and cost‐related benefits. 7 Discrepancy types within classification systems differ, but common types include Commission (i.e., medication present in EHR but patient not taking), Omission (i.e., medication absent from EHR), and Drug‐dose (i.e., dose missing or incorrect). 10
The Medication Discrepancy Taxonomy (MedTax) is a universal classification system developed to advance research and evaluations to reduce discrepancies. 1 MedTax was designed and validated for use by pharmacists only. Exclusively relying on pharmacists' or clinicians' expertise to classify medication discrepancies may be limited by costs and time availability.
Using nonclinicians in medication reconciliation processes could provide many benefits, especially in research and quality improvement (QI) initiatives. However, little is known about the role of nonclinical personnel in the classification subprocess. We determined if research assistants (RAs) without formal clinical education can classify identified medication discrepancies with accuracy comparable to pharmacists.
2. METHODS
2.1. Dataset
A dataset containing 1024 discrepancies (e.g., omissions, commissions) was derived from medication lists of 179 patients (5.7 discrepancies/patient) collected during care transitions as part of a larger trial from December 2019 to October 2020. 11 All study procedures were approved by the VA Boston Healthcare System Institutional Review Board.
2.2. Clinical and educational training of coders
Three coders participated in the classification process. The pharmacist had a Doctor of Pharmacy degree and was completing a postgraduate pharmacy residency. There were two RAs; RA‐1 had a Master of Arts in Psychology, and RA‐2 had a Master of Public Health.
2.3. MedTax training procedures
We used a modified version of the validated MedTax 1 to classify potential medication discrepancies. To modify MedTax, two physicians (Steven R. Simon and Amy M. Linsky) used clinical and content expertise to select medication discrepancy types most applicable to the healthcare setting of the study (see Text Box 1).
Text Box 1. Modified medication discrepancy taxonomy.
| Type of medication discrepancy | Description |
|---|---|
|
Patient reports taking a medication that is omitted from the medical record. |
|
The medical record includes a medication that the patient reports not taking. |
|
Medication is recorded multiple times in the medical record. |
|
The medical record lists the strength and/or frequency and/or number of units of dosage form and/or total daily dose different from what the patient reports taking. |
|
Medication listed as “Expired” due to original prescription more than 1‐year‐old, but patient reports still taking. |
|
The medical record lists the dosage, form, and/or route of administration of the medication different from what the patient reports taking. |
|
All discrepancies that do not qualify as Types 1–6 |
|
A 2‐h training session on the application of the modified MedTax was conducted using 79 medication discrepancies. Participants included three coders: pharmacist (Megan Nowak) and RAs (Kate Yeksigian and Julianne E. Brady) – and two physicians (Steven R. Simon and Amy M. Linsky). During Round 1, all five participants reviewed 25 discrepancies sequentially and collaboratively discussed each classification. After Round 1, the pharmacists, RAs, and physicians independently classified the remaining 54 discrepancies in five training rounds (Rounds 2–6; 13.5 discrepancies/round; range 7–14).
Classification between the three coders and the two physicians (i.e., the “gold standard”) was assessed. Discordant classifications were discussed as a group, with the physicians providing additional guidance. The training aimed to achieve acceptable a priori agreement between the RAs and pharmacist (>75%). 12
2.4. Independent coding session
After the training, the pharmacist and each RA independently coded the remaining 945 discrepancies. The pharmacist reviewed all discrepancies (n = 945), and the RAs divided the discrepancies for review (RA‐1: n = 516; RA‐2: n = 429).
2.5. Analysis
2.5.1. Interrater reliability of the training session
To determine the agreement between the pharmacist and two RAs, Krippendorff's α was calculated and interpreted using the following benchmarks of agreement: 0–0.667: poor; 0.668–0.799: possibly acceptable; and 0.800–1: acceptable. 13 To assess individual coder accuracy, Cohen's κ was calculated to compare each of the pharmacist and RA coders to the gold standard and interpreted using the following benchmarks of agreement: 0–0.20: poor; 0.21–0.40: fair; 0.41–0.60: moderate; 0.61–0.80: substantial; and 0.81–1.00: almost perfect. 14 Percent agreement was also calculated.
2.5.2. Interrater reliability of independent coding session
In the primary analysis, we calculated Krippendorff's α to measure interrater reliability (IRR) between the three coders. In secondary analyses, we calculated Cohen's κ and percent agreement to assess the IRR of each RA to the pharmacist. In supplemental analyses, all IRR statistics for each discrepancy type were calculated. All analyses were conducted using R 4.0 15 and Stata 16. 16
3. RESULTS
3.1. Training session IRR
The RAs and pharmacist showed moderate agreement with each other during the training rounds (α = 0.73; 95% confidence interval [CI]: 0.59, 0.83). Across all three coders, the overall percent agreement was 69.8%, just below the a priori goal of 75%.
3.2. Independent coding IRR
In the primary analysis, RAs classified discrepancies using the modified MedTax with acceptable accuracy compared to the pharmacist (945 discrepancies, α = 0.81, 95% CI: 0.78–0.84). In secondary analyses, both RAs individually coded with acceptable performance compared to the pharmacist, with almost perfect concordance between the pharmacist and both RA‐1 (516 discrepancies; ƙ = 0.82, 95% CI: 0.79–0.86; 86.6% agreement) and RA‐2 (429 discrepancies; ƙ = 0.80, 95% CI: 0.75–0.84; 84.6% agreement).
In supplemental analyses, the pharmacist and RAs classified five of eight (63%) MedTax medication discrepancy types with acceptable concordance: Omission (α = 0.86), Commission (α = 0.83), Duplication (α = 0.92), Strength (α = 0.85), and Computer System Expiration (α = 0.92). The pharmacist and RAs had possibly acceptable concordance classifying Drug Dose/Form/Route (α = 0.67) and poor concordance classifying No Discrepancy (α = 0.65) and Other discrepancies (α = 0.31).
A post hoc review of the discordance classifying Other discrepancies found these were all classified as Other by the pharmacist and omission by the RAs. In post hoc blinded physician adjudication of all discordant classifications, the physician agreed with RA classifications for 100% (12/12) of the Other disagreements and 74% (71/96) of the No Discrepancy disagreements.
4. DISCUSSION
RAs without formal clinical education can successfully classify medication discrepancies with accuracy comparable to a clinical pharmacist. These findings that RAs can be trained to reliably use a modified medication discrepancy classification taxonomy (MedTax) support engagement of nonclinical individuals to classify discrepancies within research and QI settings (i.e., nonclinical settings) where frequency and types of medication discrepancies are often used to assess the effectiveness of medication reconciliation interventions. 17 While direct contribution by clinical pharmacists in such efforts is beneficial, competing clinical responsibilities 18 may preclude their involvement and require identification of nonclinical individuals (e.g., RAs) to perform specific aspects of medication reconciliation.
Employing clinical pharmacists in research and QI projects is difficult due to direct and opportunity costs 19 , 20 Cost and time savings, combined with our finding that RA classification is reliable and valid, would yield even greater net benefit when combined with the clinical value of high‐quality medication reconciliation. 21 , 22
The RAs classified five of eight discrepancy types (Omission, Commission, Duplication, Strength, Computer System Expiration) with near perfect concordance. Classifying changes in Dosage Form or Route of Administration may be more complex and require additional training. However, post hoc analyses supported an RA's ability to accurately classify these two discrepancy types, despite lower concordance with the pharmacist.
Findings should be interpreted in the context of the following limitations. There were only a small number of individuals; findings should be replicated including coders with more diverse clinical and nonclinical backgrounds. The pharmacist was completing a pharmacy residency; one with more professional experience may yield different findings. This study evaluated the ability to train RAs on one classification system. Nevertheless, the modified taxonomy was based upon the validated MedTax and included classifications for the most common medication discrepancies.
5. CONCLUSION
Training RAs to classify medication discrepancies reliably has many benefits in both research and QI settings. Greater availability and lower overall cost of individuals without formal clinical education optimizes the use of limited time and financial resources while maintaining confidence that nonclinical personnel can be trained to classify previously identified medication discrepancies.
AUTHOR CONTRIBUTIONS
Julianne E. Brady: Data curation and writing – original draft preparation. Steven R. Simon: Funding acquisition, conceptualization, methodology, and writing – reviewing and editing. Kate Yeksigian: Data curation, and writing – reviewing and editing. Alan J. Zillich: Conceptualization, methodology, and writing – reviewing and editing. Jonathan Moyer: Formal analysis, and writing – reviewing and editing. Amy M. Linsky: Conceptualization, methodology, and writing – original draft preparation. All authors have read and approved the final version of the manuscript.
CONFLICT OF INTEREST
The authors declare no conflict of interest.
TRANSPARENCY STATEMENT
The lead author Julianne E. Brady affirms that this manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.
ACKNOWLEDGMENTS
The authors would like to thank late Megan Nowak, PharmD (1994–2021) for her work on the data coding and analysis. This study was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Services (IIR 14‐059; PI Steven R. Simon).
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions. Julianne E. Brady had full access to all of the data in this study and takes complete responsibility for the integrity of the data and the accuracy of the data analysis.
REFERENCES
- 1. Almanasreh E, Moles R, Chen TF. The medication discrepancy taxonomy (MedTax): the development and validation of a classification system for medication discrepancies identified through medication reconciliation. Res Social Adm Pharm. 2020;16(2):142‐148. [DOI] [PubMed] [Google Scholar]
- 2. Ernst ME, Brown GL, Klepser TB, Kelly MW. Medication discrepancies in an outpatient electronic medical record. Am J Health Syst Pharm. 2001;58(21):2072‐2075. [DOI] [PubMed] [Google Scholar]
- 3. Manley HJ, Drayer DK, McClaran M, Bender W, Muther RS. Drug record discrepancies in an outpatient electronic medical record: frequency, type, and potential impact on patient care at a hemodialysis center. Pharmacotherapy. 2003;23(2):231‐239. [DOI] [PubMed] [Google Scholar]
- 4. Linsky A, Simon SR. Medication discrepancies in integrated electronic health records. BMJ Qual Saf. 2013;22(2):103‐109. [DOI] [PubMed] [Google Scholar]
- 5. Coleman EA, Smith JD, Raha D, Min S‐j. Posthospital medication discrepancies: prevalence and contributing factors. Arch Intern Med. 2005;165(16):1842‐1847. [DOI] [PubMed] [Google Scholar]
- 6. Varkey P, Cunningham J, O'Meara J, Bonacci R, Desai N, Sheeler R. Multidisciplinary approach to inpatient medication reconciliation in an academic setting. Am J Health Syst Pharm. 2007;64(8):850‐854. [DOI] [PubMed] [Google Scholar]
- 7. Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med. 2003;138(3):161‐167. [DOI] [PubMed] [Google Scholar]
- 8. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee‐for‐service program. N Engl J Med. 2009;360(14):1418‐1428. [DOI] [PubMed] [Google Scholar]
- 9. The Joint Commission . National patient safety goals. 2020. Accessed September 9, 2020. https://www.jointcommission.org/standards/national-patient-safety-goals/
- 10. Kostas T, Paquin AM, Zimmerman K, Simone M, Skarf LM, Rudolph JL. Characterizing medication discrepancies among older adults during transitions of care: a systematic review focusing on discrepancy synonyms, data sources and classification terms. Aging Health. 2013;9(5):497‐508. [Google Scholar]
- 11. Simon SR, Russ‐Jara AL, Zillich A, et al. A Randomized Controlled Trial for Medication Reconciliation via Secure Messaging to Reduce Medication Discrepancies After Hospital Discharge. Springer; 2020:S20. [Google Scholar]
- 12. Kottner J, Audigé L, Brorson S, et al. Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J Clin Epidemiol. 2011;64(1):96‐106. 10.1016/j.jclinepi.2010.03.002 [DOI] [PubMed] [Google Scholar]
- 13. Krippendorff K. Content analysis: An introduction to its methodology. Sage Publications; 2018. [Google Scholar]
- 14. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159‐174. [PubMed] [Google Scholar]
- 15. R Foundation for Statistical Computing . R: A language and environment for statistical computing. 2020. https://www.r-project.org/
- 16. StataCorp LLC . Stata Statistical Software: Release 16 ;2019.
- 17. van den Bemt PM, van der Schrieck‐de Loos EM, van der Linden C, Theeuwes AM, Pol AG, Group DCWHsS . Effect of medication reconciliation on unintentional medication discrepancies in acute hospital admissions of elderly adults: a multicenter study. J Am Geriatr Soc. 2013;61(8):1262‐1268. [DOI] [PubMed] [Google Scholar]
- 18. American Society of Health‐System Pharmacists . Recommendations of the summit. Am J Health Syst Pharm. 2014;71(16):1390‐1391. 10.2146/ajhp140299 [DOI] [PubMed] [Google Scholar]
- 19. Armour C, Brillant M, Krass I. Pharmacists' views on involvement in pharmacy practice research: strategies for facilitating participation. Pharm Pract. 2007;5(2):59‐66. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Cvijovic K, Boon H, Jaeger W, Vohra S. Pharmacists' participation in research: a case of trying to find the time. Int J Pharm Pract. 2010;18(6):377‐383. [DOI] [PubMed] [Google Scholar]
- 21. Mueller SK, Sponsler KC, Kripalani S, Schnipper JL. Hospital‐based medication reconciliation practices: a systematic review. Arch Intern Med. 2012;172(14):1057‐1069. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Mekonnen AB, McLachlan AJ, Jo‐anne EB. Effectiveness of pharmacist‐led medication reconciliation programmes on clinical outcomes at hospital transitions: a systematic review and meta‐analysis. BMJ Open. 2016;6(2):e010003. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions. Julianne E. Brady had full access to all of the data in this study and takes complete responsibility for the integrity of the data and the accuracy of the data analysis.