Abstract
Background:
The advent of hospital electronic medical records (EMRs) with electronic prescribing provides considerable opportunity for pharmacoepidemiological research. However, validity of EMR prescribing data for research purposes is not well established. Validity concerns the percentage of cases in which medications and characteristics (name, type, formulation, dose) are true when verified with an independent data source. This study evaluated the validity of EMR discharge prescription data within the Eastern Health hospital network in Melbourne, Australia.
Methods:
A random sample of patients were selected who had a diagnosis of atrial fibrillation (AF) and were prescribed at least five medications. Prescription records from 2012 to 2015 were compared with pharmacy dispensing and hospital medical records (reference standards). Medication name, dose, directions and route of administration were compared. Discrepancies between data sources were categorized as omissions, additions, discrepancies in dose, medication form or route of administration or discrepancies in reordering. Sensitivities and 95% confidence intervals (CIs) for intended medication exposure were estimated for therapeutic classes.
Results:
A total of 5724 prescription orders for 479 patients for whom reference standards were available were included. There were 163 discrepancies (2.8%) between prescription records and reference standards. Additions were the most common data discrepancy (n = 65; ~1.1% of total prescriptions evaluated), followed by discrepancies in reordering (n = 34; 0.59%). Sensitivities for intended patient exposure to a medication for each therapeutic class at the first level of the Anatomical Therapeutic Chemical (ATC) classification system were between 97% and 100%. The genitourinary system and sex hormone level of the ATC system demonstrated the lowest sensitivity, (97.3%; 95% CI 92.0%–100%) and the cardiovascular system level demonstrated the highest sensitivity (99.9%; 95% CI 99.7%–100%).
Conclusion:
EMR discharge prescription records for patients with AF are a valid information source for conducting pharmacoepidemiological research within Eastern Health in Melbourne, Australia. Further studies in different regions, countries and patient cohorts are required to establish validity of hospital EMR prescription records for pharmacoepidemiological research.
Keywords: electronic medical record, hospital discharge prescription records, pharmacoepidemiology, validation studies
Introduction
Pharmacoepidemiological studies play an important role in providing information about the risks and benefits of medications across diverse care settings.1,2 The increasing availability of electronic health data supports the capacity for researchers and clinicians to investigate targeted clinical questions central to everyday clinical care.2,3 For example, investigation of rates and reasons for rehospitalization is a high priority for healthcare organizations.4 Unplanned readmissions are costly, potentially avoidable and pose a risk to patient safety.5–7 Adverse drug events (ADEs) contribute substantially to postdischarge complications and readmission.8–10 However, a lack of readily accessible medication data has precluded pharmacoepidemiological analyses to identify high-risk patients at the time of hospital discharge. The near-real-time availability of electronic medical record (EMR) prescribing data could assist clinicians and policy makers to identify patients at high risk of ADEs and of readmissions, and to target clinical interventions accordingly.4,11
Atrial fibrillation (AF) is a common age-related cardiac arrhythmia encountered in medical practice.12,13 Patients with AF commonly have concomitant cardiovascular and metabolic disorders, take multiple medications, and experience frequent hospital admissions.14,15 Oral anticoagulation (OAC) can mitigate stroke risk in patients with AF,16–20 but must be appropriately prescribed and managed to reduce bleeding risk.21 This patient group could potentially benefit from the use and analysis of EMR discharge prescription records to improve the understanding of pharmacotherapy-related predictors for early unplanned hospitalization.
Eastern Health (EH), a large metropolitan public hospital network in Melbourne, Australia, implemented an EMR with electronic prescribing (e-prescribing) in 2011. Discharge prescribing, one aspect of the e-prescribing functionality, is a routine component of discharge procedures undertaken by hospital physicians. The process within the hospital network involves creating an electronic discharge prescription record that details all medications intended for use by a patient once discharged from hospital. The prescription is printed on paper before being reviewed by a hospital pharmacist and forwarded to a pharmacy for dispensing. The EMR discharge prescription records, which are generated for the purposes of clinical care, are consolidated in a database and potentially represent a comprehensive source of medication information at the time of hospital discharge. These data potentially provide the opportunity to study prescribing patterns, and the corresponding impact on rehospitalization and health outcomes.
EMR discharge prescription records should be of equivalent quality to pharmacy claims and dispensing data, but there are no readily available authoritative reports about the quality of EMR discharge prescription records. The extent of data inaccuracies is not known. Inaccuracies may occur, for example, when amendments are made on the paper copies of prescriptions after they have been printed, without these changes being made in the electronic system. The objective of this study was to evaluate the validity of EMR discharge prescription records as a source of medication data for use in pharmacoepidemiological research within the EH hospital network in Melbourne, Australia.
Methods
Setting
This study was undertaken at EH in Melbourne, Australia. EH is a large metropolitan public hospital network of three acute and four subacute hospitals (1423 beds). The EH network services a catchment area of 750,000 people of diverse ethnic and socioeconomic backgrounds. Between July 2015 and June 2016, EH provided care for 157,532 emergency department presentations and 1,175,249 patient episodes.22 Hospital services are located across the largest geographical area of any metropolitan health service in the state of Victoria, Australia.22
Overview of data sources
EMR discharge prescribing data
Hospital doctors prepare discharge prescriptions within the EMR prior to patient discharge. This most often occurs within 24 h prior to the planned time of discharge. The discharge prescription details all medications intended for use by the patient after discharge. Discharge prescriptions are printed to hard copy and reviewed by hospital pharmacists. Medication supply occurs via either hospital or community pharmacies, using the printed hard copy prescription. Discharge prescriptions are prepared irrespective of the patients’ need for medication supply or dispensing at the time of discharge, as the prescription records form part of the medical discharge summary. Discharge prescriptions therefore provide a record of current medications at the time of discharge. An exception to this is surgical patients with a length of stay less than 24 h; not all current medications are prescribed for this patient group, only medications that relate to the episode of care are prescribed. The workflow for discharge prescribing is depicted in Figure 1. EMR discharge prescribing data include the information outlined in Table 1.
Figure 1.
Workflow for physician prescribing and pharmacist review of electronic medical record (EMR) discharge prescriptions for patients being discharged from hospital.
Table 1.
List of variables contained in the electronic medical record (EMR) discharge prescribing data.
| Patient details | Medication details | Hospital details | Other |
|---|---|---|---|
| Hospital record number Dates of hospitalization |
Medication name Strength Dose form Route of administration Frequency and patient instructions Quantity prescribed Repeats/refills prescribed Order status (ordered or discontinued) |
Hospital site Name of prescriber Ward location Treating medical unit |
Pharmaceutical Benefits Scheme (PBS) item approval code
(the PBS is Australia’s universal pharmaceutical
reimbursement system) Date and time of last modification of prescription |
Hospital pharmacy dispensing records
Patient prescriptions are dispensed using hospital pharmacy dispensing software. Medication details from the printed copy of the EMR discharge prescription are manually entered into the software by pharmacy technicians and pharmacists. Currently there is no electronic transfer of discharge prescription orders from the EMR to dispensing software. Dispensing records contain full details of the dispensing, including date of supply, the date the medication orders were prescribed, prescriber information, medication, strength, form, instructions for use, quantity and repeats/refills and Pharmaceutical Benefits Scheme (PBS) approval and reimbursement details. The PBS is Australia’s universal pharmaceutical reimbursement system.23
Hospital medical records
The hospital network is in the process of transitioning fully to EMRs, however some workflows remain paper based, and printed copies of the EMR discharge prescriptions will continue to be used as this is a legal requirement in Australia. After discharge, all paper documents generated during inpatient episodes are scanned and are available electronically within the hospital medical record. This includes printed copies of discharge prescriptions. Scanned prescriptions contain clinical pharmacist endorsement and review notes as well as details of dispensing, if medications were dispensed by hospital pharmacy services. Clinical pharmacist review also occurs for discharge prescriptions not dispensed by hospital pharmacy services. However, there are instances when this does not occur, for discharges that occur outside of pharmacy service hours or in clinical areas that do not routinely have pharmacy services.
Study design
This study evaluated the validity of the EMR discharge prescription data for a sample of medications prescribed during 2012–2015. Validity has been previously defined as the ‘percentage of cases in a registry with a given characteristic (age, gender, disease) which “truly” has this attribute. In practice, it is the percentage of agreement between registry data and an independent source objectively measuring the same variable’.24 For this study, validity refers to data accuracy for predicting a patient’s intended exposure to a medication and concerns the percentage of cases in which medications and relevant characteristics (name, type, formulation or dose) are true when verified with an independent data source (reference standards). Validity was measured by calculating sensitivity values for therapeutic classes of medications. EMR discharge prescribing functionality was implemented in 2011. Data from 2012 and onwards will be used for pharmacoepidemiological research as users were gaining familiarity with the new system during 2011. Hospital medical records and pharmacy dispensing records were used as the reference standards. Two reference standards were used as it was predicted that not all records would be available for review within only one of the reference standards. This is because not all patients have their prescriptions dispensed at the hospital and therefore not all patients have hospital pharmacy dispensing records that could be used for validation. Patients who have their prescriptions dispensed externally to hospital pharmacy services have no pharmacy dispensing data available for validation, but as it is current practice for pharmacists to sign off all orders on paper copies of the EMR discharge prescriptions, the pharmacist-endorsed records within the hospital medical record were available for review. The availability of either one of the reference standards for comparison resulted in fewer exclusions than could have occurred if only one reference standard for data validation was used. In the first instance hospital pharmacy dispensing records were used to validate the EMR discharge prescribing records. If hospital pharmacy dispensing records were not available, then hospital medical records were used.
Selection of study sample
Six hundred patients with a discharge diagnosis of AF and prescribed five or more medications at the time of discharge from one of the seven EH hospitals were randomly selected using a computer-generated number sequence. The 600 patients were selected from an established patient cohort of 4734 patients (5941 patient care episodes). The sample size of 600 was selected to represent over 10% of the AF cohort and to capture the range of medications, formulations, dosing and dosing intervals commonly prescribed to people with AF. Previous validation studies have included similar sample sizes.25,26 The validity of prescriptions in the AF population was evaluated for several reasons. This patient group may have important data discrepancies given that AF is a condition of older age, and patients were likely to be prescribed several medications at hospital discharge, have multiple comorbidities, require complex discharge planning and demonstrate considerable hospitalization rates.13,14
For each patient in the study sample, a final year pharmacy student reviewed all EMR discharge prescription records and compared the orders with the reference standards. The following details were verified: medication, dose, directions and route of administration. Any identified discrepancies were classified according to the criteria described below. Separately, all discrepancies were also independently classified by the first author. Each discrepancy was classified only once. EMR prescription data were coded using Anatomical Therapeutic Chemical (ATC) codes of the World Health Organization.27
Routinely collected hospital data that include demographic and discharge diagnoses were linked to discharge prescribing data. Demographic information and up to 40 discharge diagnoses per patient were extracted by the health service’s Decision Support Unit. Discharge diagnoses are coded using the International Statistical Classification of Diseases and Related Health Problems, 10th revision, Australian Modification (ICD-10-AM) system.28 Approval for the study was obtained from both the health service and university Human Research Ethics Committees (study numbers LR2015-64 and CF16/1211).
Classification of data discrepancies
Discrepancies between EMR discharge prescription records and the reference standards were categorized according to the criteria in Table 2. The criteria were adapted from the medication error criteria used by Barker and colleagues to classify medication errors detected in a prospective cohort that investigated the prevalence of medication errors conducted across 36 healthcare facilities in the USA.29
Table 2.
Criteria for classifying data discrepancies.
| Discrepancy type | Description |
|---|---|
| Omission | An order not present in EMR discharge prescribing data but
present within reference standards, such as an order
handwritten onto the printed electronic prescription but not
prescribed within the EMR Example: simvastatin 40 mg tablets dispensed, but not prescribed in the EMR system |
| Addition | An order present in EMR discharge prescribing data but not
present within reference standards, such as an order ceased
by hand on the paper prescription but not discontinued in
the EMR Example: aspirin clopidogrel 100–75 mg prescribed in EMR system, but not dispensed or apparent in either reference standard |
| Discrepancy in dose | A discrepancy of any dose, frequency or instructions in the
EMR discharge prescribing data compared with reference
standards, including differences in the volume, number or
strength of a preformed dosage unit and differences in the
frequencies and instructions. Orders where the total dose
was equal but made up of different numbers or strengths of
preformed dosage units were not considered
discrepancies Example: an EMR order for sertraline 50 mg daily prescribed, but sertraline 100 mg daily dispensed |
| Discrepancy in medication form | A discrepancy of medication form between EMR discharge
prescribing data and reference standards. Example: substitution of tablets for liquid |
| Discrepancy in route of administration | A discrepancy in route of administration between EMR
discharge prescribing data and reference
standards Example: oral route of administration prescribed, sublingual route of administration dispensed |
| Discrepancy in reordering | This classification was applied to duplicate medication
orders in the EMR, where a new order was created but an
older order for the same medication (different dose or dose
form) had not been discontinued Example: metformin 500 mg twice daily ordered as well as metformin 500 mg extended release twice daily and only one product had been dispensed. The first order is incorrect but remains in the system |
Statistical analyses
Routinely collected hospital data that include demographic and discharge diagnoses were linked to discharge prescribing data. Given the broad number of medications prescribed in hospitals, we hypothesized there could be variation in sensitivities between the first level of Anatomical Therapeutic Chemical (ATC) groups.27 For example, there may be differences for ATC classes that include a higher proportion of chronic-use medications compared with as-needed medications. Therefore, discrepancies between EMR discharge prescribing data and reference standards were evaluated at the first level of the ATC classification system. Data discrepancies identified are presented according to the main ATC classes (Table 4). Sensitivity for first-level ATC groups and corresponding 95% confidence intervals (CIs) were calculated using the binomial (Clopper–Pearson) exact method30 (Table 5). Sensitivity is a commonly used statistic to estimate accuracy of data or identifying algorithms in administrative data sources.31 Sensitivities were calculated using the following equation: [true positive/(true positive + false negative)]; where true positives are prescription orders which match the reference standards and false negatives are prescription orders in reference standards that were not present in EMR discharge prescribing data (omission). False positives (orders categorized as addition, discrepancy in dose, discrepancy in medication form and discrepancy in reordering) were also calculated and included in Table 5. True negative results could not be calculated because the validation study sample did not include patients who were not prescribed medications at the time of hospital discharge. χ2 tests for categorical variables and t tests for continuous variables were used to test for differences in baseline demographics between included and excluded patient groups (Table 3). All analyses were undertaken in SAS 9.4 (SAS Institute Inc., Cary, NC, USA).32
Table 4.
Types of discrepancies between data sources.
| Discrepancy categorization | n (% total discrepancies) | n (% of total prescription orders) |
|---|---|---|
| Addition | 65 (39.8) | 65 (1.1%) |
| Discrepancy in reordering | 34 (20.9) | 34 (0.6%) |
| Discrepancy in dose | 27 (16.6) | 27 (0.5%) |
| Discrepancy in medication form | 20 (12.3) | 20 (0.3%) |
| Omission | 17 (10.4) | 17 (0.3%) |
| Total discrepancies (n = 163) | Total prescription orders (n = 5724) | |
| Discrepancy rate | 163/5724 (2.8%) |
Table 5.
Sensitivities of electronic medical record (EMR) discharge prescribing data according to first level of the Anatomical Therapeutic Chemical (ATC) classification system.
| ATC main system level | n | Number of discrepancies, (% total) | True positive | False positive | False negative | Sensitivity, % (95% CI)* |
|---|---|---|---|---|---|---|
| Alimentary tract and metabolism (A) | 1248 | 46 (3.7) | 1202 | 42 | 4 | 99.7 (99.3–100) |
| Antineoplastic and immunomodulating agents (L) | 24 | 1 (4.2) | 23 | 1 | 0 | 100 |
| Antiparasitic products, insecticides and repellents (P) | 0 | 0 | 0 | 0 | 0 | N/A |
| Blood and blood-forming organs (B) | 853 | 16 (1.9) | 837 | 13 | 3 | 99.6 (92.2–100) |
| Cardiovascular system (C) | 1448 | 40 (2.8) | 1408 | 38 | 2 | 99.9 (99.7–100) |
| Dermatologicals (D) | 51 | 1 (2.0) | 50 | 1 | 0 | 100 |
| Genitourinary system and sex hormones (G) | 38 | 2 (5.3) | 36 | 1 | 1 | 97.3 (92.0–100) |
| Anti-infectives for systemic use (J) | 357 | 12 (3.4) | 345 | 11 | 1 | 99.7 (99.1–100) |
| Musculoskeletal system (M) | 107 | 5 (4.7) | 102 | 4 | 1 | 99.0 (97.1–100) |
| Nervous system (N) | 817 | 21 (2.6) | 796 | 18 | 3 | 99.6 (99.2–100) |
| Respiratory system (R) | 422 | 14 (3.3) | 408 | 13 | 1 | 99.8 (99.3–100) |
| Sensory organs (S) | 115 | 3 (2.6) | 112 | 2 | 1 | 99.1 (97.3–100) |
| Systemic hormonal preparations, excluding sex hormones and insulins (H) | 215 | 2 (0.9) | 213 | 2 | 0 | 100 |
| Various (V) | 29 | 0 (0.0) | 29 | 0 | 0 | 100 |
| Total | 5724 | 163 (2.8) | 5561 | 152 | 17 | 99.7 (99.6–99.8) |
True positive = prescriptions orders correctly identified in reference standards; false positive = EMR prescription orders with discrepancies in references standards. These include orders categorized as addition, discrepancy in dose, discrepancy in medication form and discrepancy in reordering. False negative = medication orders found in reference standards that were not present in EMR discharge prescribing data and include orders categorized as an omission. True negative results could not be calculated because the validation study sample did not include patients who were not prescribed medications at the time of hospital discharge.
Sensitivity = [true positive / (true positive + false negative)]; 95% confidence intervals (CIs) were calculated using the binomial (Clopper–Pearson) exact method based on the β distribution.
N/A, not applicable.
Table 3.
Characteristics of study sample and patients excluded from review.
| Included patients (study
sample) (n = 479) |
Excluded patients (n = 121) |
p value‡ | |
|---|---|---|---|
| Age, mean (SD) | 79.1 (10.6) | 79.4 (10.5) | 0.796 |
| Women, n (%) | 259 (54) | 69 (57) | 0.622 |
| Length of stay (days), median (IQR) | 6 (4–10) | 7 (4–12) | 0.834 |
| Admission type | 0.045 | ||
| Acute admission, n (%) | 456 (95.2) | 120 (99.2) | |
| Subacute admission, n (%) | 23 (4.8) | 1 (0.8) | |
| Year of hospital discharge, n (%) | 0.112 | ||
| 2012 | 103 (21.5) | 17 (14) | |
| 2013 | 99 (20.7) | 24 (19.8) | |
| 2014 | 129 (26.9) | 30 (24.8) | |
| 2015 | 148 (30.9) | 50 (41.3) | |
| Discharge destination, n (%) | 0.027 | ||
| Private residence/accommodation | 410 (86) | 75 (62) | |
| Aged care residential facility or another health care facility | 69 (14) | 46 (38) | |
| Comorbidities*, n (%) | |||
| Diabetes | 101 (21.1) | 27 (22.3) | 0.801 |
| CCF | 121 (25.3) | 29 (24.0) | 0.564 |
| Stroke/TIA/systemic embolism | 41 (8.6) | 11 (9.0) | 0.956 |
| Dementia | 51 (10.7) | 14 (11.6) | 0.793 |
| Hypertension | 128 (26.7) | 37 (30.6) | 0.426 |
| COPD | 68 (14.2) | 16 (13.2) | 0.758 |
| Medications | |||
| Median number of medications (IQR)$ | 10 (6–12) | 10 (7–12) | 0.098 |
| Reference standards utilized for validation, n (%) | |||
| Pharmacy dispensing records | 390 (81.6) | Not applicable | |
| Hospital medical records | 89 (18.4) |
Based on International Classification of Disease and Health Related Problems, 10th revision, Australian Modified (ICD-10-AM).
Selection for validation required patients to have at least five medication orders on the EMR.
χ2 tests for categorical variables and t tests for continuous variables were used to test for differences in baseline demographics between included and excluded patient groups.
CCF, congestive cardiac failure; COPD, chronic obstructive pulmonary disease; IQR, interquartile range; SD, standard deviation; TIA, transient ischemic attack.
Results
In total, EMR discharge prescription data of 479 patients (5724 prescription orders) were compared with reference standards. For the remaining 121 patients (1546 prescription orders), comparisons could not be undertaken as reference data were not available. The absence of reference data is most likely due to paper prescriptions not being scanned to medical records when prescriptions were provided directly to patients to obtain medications external to hospital services. Table 3 provides demographic and clinical characteristics for patients included and excluded from review, and the distribution of references standards used to validate the data. The mean age of the study sample was 79.1 years, 54% were women and 86% were residing in their own home at the time of the study. The included and excluded patient groups were similar in terms of age, sex, length of hospital stay, comorbidities and median number of prescribed medications. In the group of patients who were excluded, a higher percentage of patients were discharged to aged care facilities or another acute facility. A total 81.6% of records were validated using pharmacy dispensing data as the reference standard and 18.4% of records were validated using hospital medical records.
Discrepancies categorized as additions were most frequent (Table 4). According to ATC groups, the highest percentage of discrepancies per number of prescription orders for each ATC group were found for genitourinary system and sex hormones (5.7%), musculoskeletal system (4.7%), antineoplastic and immunomodulating agents (4.2%) and alimentary tract and metabolism (3.7%) ATC groups (Table 5). The sensitivities for intended patient exposure to a medication were estimated to be between 97% and 100% for all 14 first-level ATC groups (Table 5). Examples of discrepancies between data sources and the potential impact of these discrepancies for pharmacoepidemiological research are outlined in Table 6.
Table 6.
Examples of discrepancies between data sources and the potential impact of these discrepancies on pharmacoepidemiology research.
| Details of discrepancy | Type of discrepancy | Medication involved in discrepancy | Potential impact of discrepancy on validity of data for use in pharmacoepidemiology research |
|---|---|---|---|
| Medication was present in the reference standard but not in EMR discharge prescribing data. The medication was hand written onto the scanned copy of the electronic prescription | Omission | Candesartan 16 mg oral tablet | The patient would be misclassified as ‘unexposed’ to this medication |
| Medication was present in EMR discharge prescribing data, but not within reference standards. The medication was hand crossed off the scanned copy of the prescription with a note to confirm cessation of the medication | Addition | Verapamil 120 mg immediate release oral tablet | The patient would be misclassified as being ‘exposed’ to this medication |
| Pregabalin 25 mg twice daily ordered in EMR, but pregabalin 75 mg twice daily dispensed in pharmacy dispensing records. The 25 mg dose was changed by the prescriber to 75 mg by hand on the scanned copy of the electronic prescription | Discrepancy in dose | Pregabalin 25 mg oral capsule | The patient would be correctly classified as ‘exposed’ to this medication, but the incorrect dose could be assumed for this patient |
| The medication dosage form prescribed in the EMR was a tablet; though a liquid supplied for an equivalent dose | Discrepancy in medication form | Amoxycillin + clavulanic acid tablet/liquid | Same medication supplied just in a different dosage form. This would not have an impact on classifying patients as exposed or unexposed to a medication |
| Two orders exist in the EMR discharge prescribing data for metoprolol for different doses (75 mg twice daily and 50 mg twice daily). One prescription dispensed in the pharmacy records (50 mg twice daily). It appears there had been a dose change, however the old order was not discontinued in the EMR | Discrepancy in reordering | Metoprolol | The patient would be correctly classified as ‘exposed’ to the medication. However, the dose the patient was prescribed at the time of hospital discharge is not clear |
EMR, electronic medical record.
Discussion
This study demonstrated that EMR discharge prescription data are highly sensitive (97–100%) for predicting a patient’s intended exposure to a medication at the time of hospital discharge and that there is a low rate of false positives (0–4.2%). The genitourinary system and sex hormone (G) level of the ATC system demonstrated the lowest sensitivity (97.3%; 95% CI 92.0%–100%) and the cardiovascular system (C) level demonstrated the highest sensitivity (99.9%; 95% CI 99.7%–100%) (Table 5). The overall discrepancy rate between EMR discharge prescribing data and the reference standards was 2.8% (Table 4). The most common type of discrepancy was an addition (n = 65; 1.1% of all prescription orders evaluated), followed by discrepancies in reordering (n = 34; 0.6% of all prescription orders evaluated).
To our knowledge, this is the first study to evaluate the validity of EMR discharge prescribing records for use in pharmacoepidemiological research. The findings presented here compare favourably with other sources of medication data used in pharmacoepidemiology. For example, a validation study of the Finnish Prescription Register for measuring psychotropic drug use among an elderly Finnish cohort demonstrated ‘almost perfect’ [κ (Cohen’s κ coefficient) = 0.81–1.00] or ‘substantial’ (κ = 0.61–0.80) agreement between self-reported sources and the prescription register for all categories and subcategories of psychotropic drugs. In terms of sensitivity, there was some variability for time periods. Sensitivity was estimated to be 0.79–0.93 for a 12-month time frame, but was greatest during the 4-month time period (0.94–0.99).26 Further, a study investigating self-reported drug use and drug use modelled using the PRE2DUP method (a method that evaluates personal drug purchasing patterns to generate a drug use period) found very good agreement (κ = 0.81–1.00) for 75% of studied drugs and classes; and very good or good agreement (κ = 0.61–1.00) for 93% of studied drugs and classes. Agreement was greater for regular medications than for ‘as-needed’ medications. Importantly, this study evaluated all classes of commonly used medications, including ‘as-needed’ medications.25
In this study, the overall rate of data discrepancies was less than 3%. While the discrepancy rate is small, the sensitivity of predicting a patient’s intended exposure to a medication was high (97%–100%) and the rate of false positives was low (0–4.2%); the impact of different types of data discrepancies on study validity and findings should be considered for pharmacoepidemiological research. A discrepancy of addition indicates that the medication was present in EMR discharge prescribing data, but not within reference standards. This could mean that the medication was hand crossed off the paper copy of the prescription, but the amendment was not made on the electronic system. This could result in misclassification of the exposure. A patient could be considered ‘exposed’ to a medication when they are actually ‘unexposed’. For discrepancies of omission, the opposite could occur. A patient could be misclassified as ‘unexposed’ when they are truly ‘exposed.’ For discrepancies of reordering, exposure classification would not be impacted. Rather, the medication dose of exposed individuals could be impacted. For discrepancies of reordering, two orders remain in the system; however one order is no longer valid. Without further verification, the medication dose remains unclear. For studies investigating exposure only, discrepancies in reordering would not threaten study findings. However, if the research involves the specifics of medication dose, then the impact of any discrepancies in reordering would need to be considered. Similarly, discrepancies in dose would need comparable consideration to discrepancies in reordering. Finally, a discrepancy in medication form would have no impact for studies of either medication exposure or medication dose.
The use of EMR discharge prescribing records for research is cost effective and enables research to be conducted for local populations in which results can be directly translated. However, there are potential limitations of the use of EMR discharge prescribing records for research. The limitations could result from system design, input of patient data, evolving systems and the original purpose for which the data were collected.33 We have identified some specific limitations of the EMR discharge prescribing records validated in this study. First, it is recognized that even though medications have been prescribed, they may not be taken by patients.2 The impact of nonadherence on pharmacoepidemiological research outcomes is unknown and should be taken into consideration when interpreting research findings using discharge prescribing data. Second, the EMR discharge prescribing records validated in this study are essentially cross sectional and provide information for current medications for patients at the time of hospital discharge only. However, hospital discharge is a point at which treatments may be initiated, switched or discontinued, and therefore provide the opportunity to use a treatment decision design in pharmacoepidemiological studies.34 Further, EMR discharge prescribing data may provide comprehensive medication information because current medications at the time of hospital discharge are prescribed regardless of the reimbursement, supply or dispensing arrangements, as the process is essential for the purposes of completing a comprehensive discharge summary.
For this study we evaluated the validity of EMR discharge prescribing records for a sample of patients with AF. Patients with AF are older, commonly experience multimorbidity, take multiple medications, and experience frequent hospitalizations.14,15 This study was undertaken in this population as discrepancies could be easily identified in this patient group, most likely due to the complexity of medication regimens. Further, patients with AF may be representative of other patients who are commonly hospitalized with complex medication lists. The EH hospital network includes an ageing population, with approximately 20% more people aged 80 and older accessing health services compared with other hospital networks of metropolitan Melbourne. Selecting patients from an ageing cohort who are most likely inordinate users of medications reflects the ageing demographic of the catchment area for the hospital network. Further studies in different regions, countries and patient cohorts are required to establish the generalizability of the validity results presented here.
Strengths and limitations
The study results presented here should be interpreted in the context of certain study strengths and limitations. First, we were unable to validate 1546 EMR discharge prescription orders for 121 patients, as there were no reference standards available for comparison. The patient characteristics of those included did not differ markedly from those not included in the study, hence there is no major reason to suspect the EMR discharge prescription records for patients not included in the study were lower in quality to those reviewed in this study. However, it may be possible that EMR discharge prescription records compared with the reference standards in this study were of a higher quality, as there was direct pharmacist involvement in clinical review or prescription dispensing. Second, the study sample presented here is only a small subset of the overall discharge prescription dataset. It is possible that more discrepancies could be detected in samples involving particular medications, medical units, hospitals or particular medical conditions. However, the nature of the included sample (patients with a chronic condition often accompanied by multiple comorbidities, with multiple medication changes made in hospital) suggests that the study sample could serve as a good model for other medical conditions. Third, completeness of medication information cannot be entirely assured from this study. The EMR is an isolated hospital system that is not linked to a central healthcare information repository. While thorough efforts are made by hospital staff to obtain complete medication information on hospital admission and communicate medication information at the time of hospital discharge, completeness of information may only be achieved within a central health care data repository.
Conclusion
This study demonstrated that EMR discharge prescription records for patients with AF at EH in Melbourne, Australia is a valid and accurate source of medication information and the data accurately predict patients’ intended exposure to medications at the time of discharge from hospital. The most common data discrepancy was addition, which may incorrectly classify patients as exposed to a medication. Overall, the percentage of data discrepancies was small. However, a small sample was included in this study and there is scope to undertake future studies to understand data validity of both EMR discharge prescription data and other sources of EMR data. Further studies are required to confirm the generalizability of our results to other sources of EMR medication data at other Australian and international healthcare organizations. Additionally, the broader impact of increased availability of EMR data for use in pharmacoepidemiological research could be explored. Areas for future research may include adverse drug effects and risk management (including detection of rare events) and the impact of prescribing on rehospitalization and health outcomes.
Acknowledgments
The authors of this study would like to thank and acknowledge the Eastern Health Electronic Medical Record team and the Victorian Department of Health and Human Services for assistance with data access. All authors contributed to the concept and design. LV performed comparisons between data sources. LF performed data analysis and drafted the manuscript. All authors contributed to the interpretation of the data, manuscript revisions, and read and approved the final manuscript.
Footnotes
Funding: This work was supported by funding from the Victorian Department of Health and Human Services, Clinical Leadership Group for the Care of Older People in Hospital and a Joint Medicine-Pharmacy (JMP) Monash Strategic Grant. LF is supported by a Research Training Scheme PhD Scholarship from the Australian Government: Department of Education and Training. JI is supported by the National Health and Medical Research Council’s Early Career Fellowship.
Conflict of interest statement: The authors declare that there is no conflict of interest.
ORCID iD: Laura Fanning 
https://orcid.org/0000-0002-2685-3875
Contributor Information
Laura Fanning, Eastern Health Clinical School, Level 2, 5 Arnold Street, Box Hill, 3128, Victoria, Australia.
Lilian Vo, Centre for Medicine Use and Safety, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Melbourne, Australia.
Jenni Ilomäki, Centre for Medicine Use and Safety, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Melbourne, Australia; School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia.
J. Simon Bell, Centre for Medicine Use and Safety, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Melbourne, Australia; School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia; Sansom Institute, School of Pharmacy and Medical Sciences, University of South Australia, Adelaide, Australia.
Rohan A. Elliott, Centre for Medicine Use and Safety, Faculty of Pharmacy and Pharmaceutical Sciences, Monash University, Melbourne, Australia Pharmacy Department, Austin Health, Melbourne, Australia.
Pēteris Dārziņš, Eastern Health Clinical School, Faculty of Medicine Nursing and Health Sciences, Monash University, Melbourne, Australia; Geriatric Medicine, Eastern Health, Melbourne, Australia.
References
- 1. Hilmer SN, Gnjidic D, Abernethy DR. Pharmacoepidemiology in the postmarketing assessment of the safety and efficacy of drugs in older adults. J Gerontol A Biol Sci Med Sci 2012; 67: 181–188. [DOI] [PubMed] [Google Scholar]
- 2. Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol 2005; 58: 323–337. [DOI] [PubMed] [Google Scholar]
- 3. Staa TP, Goldacre B, Gulliford M, et al. Pragmatic randomised trials using routine electronic health records: putting them to the test. BMJ 2012; 344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Nguyen OK, Makam AN, Clark C, et al. Predicting all-cause readmissions using electronic health record data from the entire hospitalization: model development and comparison. J Hosp Med 2016; 11: 473–480. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med 2009; 360: 1418–1428. [DOI] [PubMed] [Google Scholar]
- 6. van Walraven C, Bennett C, Jennings A, et al. Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ 2011; 183: E391–E402. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Caughey GE, Pratt NL, Barratt JD, et al. Understanding 30-day re-admission after hospitalisation of older patients for diabetes: identifying those at greatest risk. Med J Aust 2017; 206: 170–175. [DOI] [PubMed] [Google Scholar]
- 8. Forster AJ, Murff HJ, Peterson JF, et al. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med 2003; 138: 161–167. [DOI] [PubMed] [Google Scholar]
- 9. Dalleur O, Beeler PE, Schnipper JL, et al. 30-day potentially avoidable readmissions due to adverse drug events. J Patient Saf. Epub ahead of print 17 March 2017. doi: 10.1097/pts.0000000000000346. [DOI] [PubMed] [Google Scholar]
- 10. Davies EC, Green CF, Mottram DR, et al. Emergency re-admissions to hospital due to adverse drug reactions within 1 year of the index admission. Br J Clin Pharmacol 2010; 70: 749–755. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Kalisch LM, Caughey GE, Barratt JD, et al. Prevalence of preventable medication-related hospitalizations in Australia: an opportunity to reduce harm. Int J Qual Health Care 2012; 24: 239–249. [DOI] [PubMed] [Google Scholar]
- 12. Ball J, Thompson DR, Ski CF, et al. Estimating the current and future prevalence of atrial fibrillation in the Australian adult population. Med J Aust 2015; 202: 32–35. [DOI] [PubMed] [Google Scholar]
- 13. Lip GY, Fauchier L, Freedman SB, et al. Atrial fibrillation. Nat Rev Dis Primers 2016; 2: 1–26. [DOI] [PubMed] [Google Scholar]
- 14. Steinberg BA, Kim S, Fonarow GC, et al. Drivers of hospitalization for patients with atrial fibrillation: Results from the Outcomes Registry for Better Informed Treatment of Atrial Fibrillation (ORBIT-AF). Am Heart J 2014; 167: 735–742. e732. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Blomstrom Lundqvist C, Lip GY, Kirchhof P. What are the costs of atrial fibrillation? Europace 2011; 13(Suppl. 2): ii9–ii12. [DOI] [PubMed] [Google Scholar]
- 16. Patel MR, Mahaffey KW, Garg J, et al. Rivaroxaban versus warfarin in nonvalvular atrial fibrillation. N Eng J Med 2011; 365: 883–891. [DOI] [PubMed] [Google Scholar]
- 17. Granger CB, Alexander JH, McMurray JJ, et al. Apixaban versus warfarin in patients with atrial fibrillation. N Eng J Med 2011; 365: 981–992. [DOI] [PubMed] [Google Scholar]
- 18. Connolly SJ, Ezekowitz MD, Yusuf S, et al. Dabigatran versus warfarin in patients with atrial fibrillation. N Eng J Med 2009; 361: 1139–1151. [DOI] [PubMed] [Google Scholar]
- 19. Giugliano RP, Ruff CT, Braunwald E, et al. edoxaban versus warfarin in patients with atrial fibrillation. N Eng J Med 2013; 369: 2093–2104. [DOI] [PubMed] [Google Scholar]
- 20. Hart RG, Pearce LA, Aguilar MI. Meta-analysis: antithrombotic therapy to prevent stroke in patients who have nonvalvular atrial fibrillation. Ann Intern Med 2007; 146: 857–867. [DOI] [PubMed] [Google Scholar]
- 21. Yao X, Abraham NS, Alexander GC, et al. Effect of adherence to oral anticoagulants on risk of stroke and major bleeding among patients with atrial fibrillation. J Am Heart Assoc 2016; 5: pii: e003074. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Lilly A, Flynn J. Eastern health 2015–2016 annual report 2016. Melbourne, Australia: Eastern Health, 2016. [Google Scholar]
- 23. Australian Government: Department of Health. The Pharmaceutical Benefits Scheme. http://www.pbs.gov.au/.
- 24. Goldberg J, Gelfand HM, Levy PS. Registry evaluation methods: a review and case study. Epidemiol Rev 1980; 2: 210–220. [DOI] [PubMed] [Google Scholar]
- 25. Taipale H, Tanskanen A, Koponen M, et al. Agreement between PRE2DUP register data modeling method and comprehensive drug use interview among older persons. Clin Epidemiol 2016; 8: 363–371. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Rikala M, Hartikainen S, Sulkava R, et al. Validity of the Finnish Prescription Register for measuring psychotropic drug exposures among elderly Finns: a population-based intervention study. Drugs Aging 2010; 27: 337–349. [DOI] [PubMed] [Google Scholar]
- 27. World Health Organisation Collaborating Centre for Drug Statistics Methodology: Norwegian Institute of Public Health. Anatomical Therapeutic Chemical (ATC) classification system, http://www.whocc.no/atc_ddd_index/ (2015, accessed 1 October 2016).
- 28. Bossuyt PM, Reitsma JB, Bruns DE, et al. STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies. BMJ 2015; 351: h5527. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Barker KN, Flynn EA, Pepper GA, et al. Medication errors observed in 36 health care facilities. Arch Intern Med 2002; 162: 1897–1903. [DOI] [PubMed] [Google Scholar]
- 30. Deeks JJ, Altman DG. Sensitivity and specificity and their confidence intervals cannot exceed 100%. BMJ 1999; 318: 193–193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Benchimol EI, Manuel DG, To T, et al. Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data. J Clin Epidemiol 2011; 64: 821–829. [DOI] [PubMed] [Google Scholar]
- 32. Manuel DG, Rosella LC, Stukel TA. Importance of accurately identifying disease in studies using electronic health records. BMJ 2010; 341. [DOI] [PubMed] [Google Scholar]
- 33. Kanas G, Morimoto L, Mowat F, et al. Use of electronic medical records in oncology outcomes research. CEOR 2010; 2: 1–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Brookhart MA. Counterpoint: the treatment decision design. Am J Epidemiol 2015; 182: 840–845. [DOI] [PMC free article] [PubMed] [Google Scholar]