Effects of an online personal health record on medication accuracy and safety: a cluster-randomized trial

Abstract

Objective

To determine the effects of a personal health record (PHR)-linked medications module on medication accuracy and safety.

Design

From September 2005 to March 2007, we conducted an on-treatment sub-study within a cluster-randomized trial involving 11 primary care practices that used the same PHR. Intervention practices received access to a medications module prompting patients to review their documented medications and identify discrepancies, generating ‘eJournals’ that enabled rapid updating of medication lists during subsequent clinical visits.

Measurements

A sample of 267 patients who submitted medications eJournals was contacted by phone 3 weeks after an eligible visit and compared with a matched sample of 274 patients in control practices that received a different PHR-linked intervention. Two blinded physician adjudicators determined unexplained discrepancies between documented and patient-reported medication regimens. The primary outcome was proportion of medications per patient with unexplained discrepancies.

Results

Among 121 046 patients in eligible practices, 3979 participated in the main trial and 541 participated in the sub-study. The proportion of medications per patient with unexplained discrepancies was 42% in the intervention arm and 51% in the control arm (adjusted OR 0.71, 95% CI 0.54 to 0.94, p=0.01). The number of unexplained discrepancies per patient with potential for severe harm was 0.03 in the intervention arm and 0.08 in the control arm (adjusted RR 0.31, 95% CI 0.10 to 0.92, p=0.04).

Conclusions

When used, concordance between documented and patient-reported medication regimens and reduction in potentially harmful medication discrepancies can be improved with a PHR medication review tool linked to the provider's medical record.

Trial registration number

This study was registered at ClinicalTrials.gov (NCT00251875).

Background and significance

Medication-related morbidity and mortality is estimated to result in $76 billion dollars in total costs annually.¹ One drug-related problem, adverse drug events (ADEs), broadly defined as injuries due to medications,² is estimated to occur in 25% of ambulatory patients.³ Of these, approximately 11% are considered preventable and an additional 28% ameliorable.

One important cause of ambulatory ADEs is medication discrepancies, including unexplained differences between medication regimens patients think they should be taking and regimens collectively prescribed by their physicians.⁴ Discrepancies can have serious consequences, including prolonged periods of over- or under-treatment.^5–8 By definition, medication discrepancies are not reported by patients and can only be detected by active surveillance.

Communication regarding medication-related problems may be completely absent between patient visits and is often inadequate even during visits because of competing demands, patient concerns about bothering their physicians, or limited patient involvement in their own care. Consequently, drug-related problems remain undetected³ and the opportunity to mitigate these problems is lost.

By empowering patients to become active participants in their own care, a personal health record (PHR) linked to an ambulatory electronic health record (EHR) has the potential to address many medication safety and quality issues.⁹ A PHR module focused on medications could allow patients online access to update medication data from their EHR, identify discrepancies, and report medication concerns. This information could then be conveyed to the patient's physician, who can discuss it with the patient, update the EHR, and take action as needed.

At Partners HealthCare System, we developed and deployed such a tool within a PHR (Patient Gateway, PG) to address medication issues.^{10 11} The design of the PG Medications Module, preliminary usage data, and patient and provider impressions of the application have been described previously.¹² This manuscript reports how the module affected medication safety outcomes and patient–provider communication. We hypothesized that the module would reduce discrepancies between patient-reported and EHR-documented medication regimens and reduce potential and preventable/ameliorable ADEs.

Methods

Setting

Partners HealthCare System is an integrated regional healthcare delivery network in eastern Massachusetts with more than 20 affiliated primary care clinics. The main EHR used in Partners ambulatory clinics is the Longitudinal Medical Record (LMR), an internally developed, certified EHR.¹³ Informed consent was obtained from eligible patients prior to notification of practice randomization status. This study was approved by the Partners HealthCare institutional review board and registered at ClinicalTrials.gov (NCT00251875).

Patient Gateway

PG is a secure online PHR developed by Partners to improve patient–provider communication. At the time of the study, PG allowed patients limited access to their LMR data (including read-only access to their medication lists) and gave them the ability to request appointments and referrals, communicate with their physician via secure email, request prescription renewals, and access a health information library.

Prepare for Care study

The Prepare for Care study was a cluster-randomized trial with active controls conducted from September 2005 to March 2007 in 11 Partners primary care practices that used PG and agreed to participate in the study. Practices were randomized to one of the two arms after matching for setting (urban vs suburban), services (women's health vs general), and size (large vs small). Randomization of matched pairs of practices was carried out using random number generation in Excel (Microsoft, Redmond, Washington, USA) by the study statistician.

To be eligible for the study, patients had to have an active PG account (ie, had logged in at least once) and at least one visit with their designated primary care provider (PCP) in a study practice in the prior year. As part of this study, the pre-visit eJournal, a new feature of PG, was developed.^{10 11}

In the present study, patients in the intervention arm were invited to complete medications eJournals prior to an upcoming PCP visit that allowed them to review and indicate updates to their medication lists, allergies, and if applicable, diabetes management information. Patients in the active control arm were invited to complete eJournals that let them review and update family history and provided views of health maintenance reminders (hereafter referred to as health maintenance eJournals).

Given the nature of the interventions, practices and providers could not be blinded, but outcome adjudicators remained blinded (see below). Data collectors were initially blinded to study arm, but some patients revealed information during follow-up phone calls that may have led to unblinding in some cases.

Medications Module

The design of the PG Medications Module has been described previously.¹² Upon invocation of the module, patients saw the current active LMR medication list and were asked about any discrepancies between this list and what patients thought they should be taking (eg, differences in dose, missing medications). The module asked patients about any problems they might be having with adherence, any possible side effects, and if they needed a prescription refill.

Once a medications eJournal had been submitted, the patient's practice could view the information in a modified LMR medication screen that displayed automatically instead of the usual medication screen. A PCP could easily verify and move medications eJournal information (eg, changes to the medication list) into the LMR. Patients were informed that their PCPs would have access to their eJournals during the consent process, but they were not automatically notified of whether their providers had viewed the information.

Medication sub-study outcomes

We conducted a sub-study of 541 patients from the main Prepare for Care study to assess the effect of the intervention when used, that is, an on-treatment analysis. First, we selected 267 patients in the intervention arm who had an eligible visit and submitted a medications eJournal. We then matched those patients to 274 patients in a matched practice in the active control arm who had a similar visit (annual or follow-up) during the same month. Prepare for Care study subjects were invited to participate in the sub-study by email and were contacted if they did not opt out within 1 week; patients could also refuse to participate when contacted. Because only annual visits triggered health maintenance eJournal invitations (at the request of the practices) while both annual and follow-up visits triggered medications eJournal invitations, not all patients in the control arm of the sub-study were invited to or completed a health maintenance eJournal.

All medication safety outcomes were assessed using a process similar to that employed in previous studies.^{4 14 15} Intervention and control patients were contacted by phone by a trained research assistant (RA) beginning 3 weeks after the PCP visit following a predefined protocol. The survey was developed based on previous studies regarding medication discrepancies and ADEs after discharge,^2–4 but the survey was not otherwise validated. RAs first asked patients to name all the medications they thought they were supposed to be taking, including dose, route, and frequency. If any discrepancies between that list and the LMR medication list were found, reasons for the discrepancy were explored.

Patients were then asked about possible ADEs. RAs conducted a thorough review of symptoms patients might have had within the previous 3 months. If symptoms were reported, RAs elicited further details and asked directed questions to determine the possible relationship of the symptom to medication use.

Results of each survey were presented to two blinded physician adjudicators (adjudicators included JLS, TKG, RWG, and one non-author physician). Adjudicators first decided whether a discrepancy between documented and reported medication regimens was readily explained (eg, a change made by one of their physicians since the visit). For the remaining (unexplained) medication discrepancies, adjudicators decided whether the discrepancy had potential for patient harm. Severity of potential harm (significant, serious, or life-threatening) was also assessed.⁴

For patient-reported symptoms, based on the phone survey and medical records within a month of the patient visit, adjudicators decided whether the symptom was due to a medication using a six-point confidence scale, using the Naranjo algorithm as a guide.¹⁶ In the event of a likely ADE, adjudicators decided on its severity (significant, serious, life-threatening, or fatal), whether the ADE could have been prevented, and if not, whether the ADE could have been ameliorated (ie, lessened in severity or duration).⁴ All differences between adjudicators were resolved by consensus.

Survey outcomes

As part of a follow-up survey administered in October through December 2006 to all patients in the main study who submitted an eJournal of any type, we assessed how frequently patients self-reported communicating with their physicians regarding medication issues.

Analysis

The primary outcome was discordance between documented and reported medication regimens, that is, the proportion of each patient's medication regimen with unexplained medication discrepancies. The denominator was calculated as all medications on either the LMR medication list or reported by the patient, while the numerator was the number of unexplained discrepancies assessed by outcome adjudicators. Secondary outcomes included the number of discrepancies with potential for harm per patient and number of discrepancies with potential for severe harm. We also assessed the number of preventable or ameliorable ADEs per patient, and the duration of ameliorable ADEs.

Adjusted analyses were conducted using PROC GENMOD in SAS using a binomial logistic model in which the outcome was in the form of X/N, where X was the number of unexplained discrepancies and N was the total number of medications. To adjust for possible confounding (ie, due to imperfect randomization at the practice level or imbalance between eJournal submitters and non-submitters), we used propensity scores¹⁷: we first derived a model to predict being in each of the two study arms, adjusting for patient age, sex, race, number of medications, number of prior visits, and median income by zip code. The score derived from this model was then used in all subsequent models of study outcomes. We used general estimating equations to adjust for clustering by provider.

For the number of discrepancies and ADEs per patient, we used propensity-score adjusted Poisson regression. For duration of ameliorable ADEs, we used a multinomial model since the outcome was an eight-level ordinal response.

To address issues of possible unadjusted confounding caused by differences between patients who submitted and did not submit eJournals, we conducted a secondary analysis of all outcomes limiting the study population to just those who submitted eJournals of any type. We also conducted an analysis in which we adjusted for clustering at the practice level instead of the provider level.

Power and sample size

With an anticipated sample size of 300 patients in each arm of the medications sub-study, we had 80% power to detect a decrease in unexplained medication discrepancies from 50% to 38% with a two-sided α of 0.05, estimated cluster size of three patients per PCP, and intra-class correlation coefficient of 0.10. Two-sided p values <0.05 were considered significant. All analyses were conducted using SAS V.9.2.

Results

During the study period, among 121 046 patients in primary care practices using PG, 21 533 (18%) had active accounts and were invited to participate (figure 1). Of these, 3979 (18%) consented to be in the study. Among the 1761 patients in the intervention arm, 1053 (60%) had an eligible primary care visit and were invited to submit a medications eJournal, of whom 743 (71%) submitted one. Of those, 267 were selected to be in the medications sub-study. These patients were matched to 274 patients in the control arm, of whom 170 had been invited to complete a health maintenance eJournal during the study period and 135 (79%) submitted one. Eligible patients who did not participate in the sub-study for various reasons (see figure 1) were on fewer medications and had fewer prior visits compared with those who did participate; these differences were similar in both arms of the study (see table A1 in the supplementary online appendix).

Figure 1.

Study flow diagram. *‘No match’ indicates patients in the active control arm who did not match any patient in the intervention arm of the sub-study by practice and by date and type of visit. †‘Opted out’ includes those patients who opted out by email after receiving the invitation to participate in the medications sub-study and those who declined participation once contacted by phone. ‡‘Never called’ indicates those patients in the active control arm who did not need to be called because a sufficient number of eligible patients had already been enrolled. PCP, primary care provider.

Table 1 compares patients in the two arms of the main study and in the medications sub-study. Among patients in the medications sub-study, those in the intervention arm were older and less often white, with higher median income by zip code and on more medications than control patients. The difference in number of medications was exaggerated when intervention patients were compared with control patients who submitted a health maintenance eJournal (5.6 vs 4.7) as opposed to all control patients (5.6 vs 4.9). Among patients in the intervention arm of the main study, those who submitted medications eJournals were prescribed more medications than non-submitters (5.2 vs 4.8, p=0.046). Among patients in the active control arm, there were no differences in the number of documented medications between those who submitted health maintenance eJournals and those who did not (4.6 vs 4.4, p=0.41; data not shown).

Table 1.

Comparison of study participants in the active control and intervention arms

Characteristics	All study participants (N=3979)		Medication sub-study participants (N=541)
	Intervention arm	Active control arm	Intervention arm§	Control arm—all patients	Control arm—submitted an eJournal
Patients, N	1761	2218	267	274	135
Age in years, mean (SD)	51.2 (12.8)	47.0 (12.7)*	55 (11)	50 (14)†	51 (13)‡
Women, N (%)	965 (54.8)	1432 (64.5)*	147 (55)	155 (57)	82 (61)
Median income by zip code, mean (SD)	55385 (9748)	54024 (10406)*	55899 (9959)	53886 (10357)†	54270 (10289)‡
Race/ethnicity, N (%)¶		*		†
Black	37 (2.1)	69 (3.1)	4 (1.5)	7 (2.6)	4 (3.0)
Hispanic	17 (1.0)	33 (1.5)	1 (0.4)	4 (1.5)	1 (0.7)
White	1543 (87.7)	1923 (86.6)	218 (82)	239 (87)	118 (87)
Other	163 (9.3)	194 (8.7)	44 (17)	24 (9)	12 (9)
Insurance, N (%)		*
Private	1421 (80.7)	1948 (87.8)	230 (86.1)	233 (85.0)	115 (85.2)
Medicare	321 (18.2)	241 (10.9)	36 (13.5)	37 (13.5)	16 (11.9)
Medicaid/free care	3 (0.2)	11 (0.5)	1 (0.4)	2 (0.7)	2 (1.5)
Self-pay/none	14 (0.8)	17 (0.8)	0	1 (0.4)	1 (.07)
Unknown	1 (0.1)	2 (0.1)	0	1 (0.4)	1 (0.7)
Medications, mean (SD)	5.0 (4.0)	4.5 (3.8)*	5.6 (4.1)	4.9 (4.0)†	4.7 (3.6)‡
Prior visits, mean (SD)	6.0 (9.7)	5.7 (6.9)	7.7 (18.3)	7.0 (8.1)	6.5 (7.3)
Charlson Score, mean (SD)			1.8 (1.1)	1.7 (0.8)	1.7 (1.5)

^*p<0.05 for comparison of arm 1 and arm 2 patients among all study participants.

^†p<0.05 for comparison of arm 1 and arm 2 patients among medication sub-study participants.

^‡p<0.05 for comparison of arm 1 and arm 2 patients among medication sub-study participants who submitted an eJournal of any type.

^§All patients in the intervention arm submitted medications eJournals.

^¶As recorded in hospital administrative data and based on patient self-report. Categories of race and ethnicity are as specified by NIH and AHRQ.

In the intervention arm, 78% of patients who were invited to complete a medications eJournal opened it, and 72% completed and submitted it. The primary care practices opened 77% of submitted medications eJournals.

Three weeks after the index primary care visit, the average medication regimen was 51% discordant among control patients. In other words, on average, slightly less than half the medications reported by patients and documented in the LMR completely agreed with each other. Unexplained discrepancies between documented and reported medication regimens included missing medications (ie, reported to be taken by the patient but not in the LMR, in 36% of patients), differences in dose and frequency (46%), and additional medications (ie, documented in the LMR but not being taken by patients (68%)). In the intervention arm, the discordance of the medication list 3 weeks after the visit was significantly lower at 42% (adjusted OR 0.71, 95% CI 0.54 to 0.94, p=0.01; table 2). This difference was mainly driven by fewer additional medications (57% vs 68% of patients) and less so by fewer missing medications (29% vs 36% of patients).

Table 2.

Results of medication safety evaluation

	Intervention	Control	Unadjusted p value	Propensity score adjusted and clustered effect size (95% CI)*	Adjusted and clustered p value
Patients, N	267	274
% Discordant medications, mean (SD)	42% (29%)	51% (30%)	<0.001	OR 0.71 (0.54 to 0.94)	0.01
Number of discrepancies per patient with potential for harm, mean (SD)	0.24 (0.62)	0.34 (0.85)	0.23	RR 0.65 (0.42 to 1.02)	0.059
Number of discrepancies per patient with potential for severe harm, mean (SD)	0.03 (0.22)	0.08 (0.35)	0.03	RR 0.31 (0.10 to 0.92)	0.04
Number of preventable or ameliorable ADEs per patient	0.16 (0.44)	0.14 (0.47)	0.45	RR 1.21 (0.74 to 1.98)	0.45
Duration of ameliorable ADEs in days, median (IQR)	5 (1–7)	6 (1–8)	0.07	Conditional OR 0.64 (0.25 to 1.66)	0.35

^*Propensity score adjusted for patient age, sex, race, number of medications, number of prior physician visits, and median income by zip code.

ADE, adverse drug event.

There was a borderline but non-significant difference in the number of unexplained discrepancies per patient with potential for harm in the two study groups (0.24 per patient in the intervention arm vs 0.34 in the control arm, adjusted p=0.059). However, there was a significant difference in the number of discrepancies per patient with potential for severe harm (0.03 vs 0.08 per patient, adjusted RR 0.31, 95% CI 0.10 to 0.92, p=0.04). There was no significant difference in the number of preventable or ameliorable ADEs per patient between the two study arms or in the duration of ameliorable ADEs (table 2).

In secondary analyses restricted to patients who submitted eJournals, the direction and magnitude of effect sizes for all study outcomes were similar (see table A2 in the supplementary online appendix). Differences between study and control arms in the discordance of medication regimens was virtually identical to the primary analysis (42% vs 50%) and remained statistically significant in unadjusted and adjusted analyses, but the difference in the number of discrepancies with potential for severe harm was no longer statistically significant. In another analysis in which clustering was performed at the practice level instead of the provider level, the results were again similar except that the effect of the intervention on discrepancies with potential for any (not just severe) harm was statistically significant (see table A3 in the supplementary online appendix).

Of 1423 patients who submitted an eJournal in either arm of the main study, 1278 (90%) completed a follow-up survey. Compared with patients in the active control group, patients in the intervention arm more often reported that they always told their PCP about all prescribed medications they took, even those prescribed by other doctors (table 3). In addition, more patients in the intervention arm reported that they told their doctor about any medication-related problems or new symptoms.

Table 3.

Survey results among patients who submitted eJournals

Survey question and responses	Number who provided that response/number who answered that question (%)		p Value
	Intervention (N=699)	Active control (N=579)
Do you tell your doctor about all prescribed medications, even those prescribed by other doctors?			0.048
Always	515/689 (74.8)	399/572 (69.8)
Do you tell your doctor about any medication-related problems or new symptoms?			<0.001
Yes	236/241 (97.9)	149/171 (87.1)
How long do you usually wait before telling your doctor about any medication-related problems or symptoms?			0.10
Tell doctor soon after they occur	88/241 (36.5)	58/176 (33.0)
Wait until next doctor's visit	64/241 (26.6)	40/176 (22.7)
Wait until think have not improved or have gotten worse	60/241 (24.9)	60/176 (34.1)

Discussion

In this study, patients using an electronic application focused on medications within a PHR had greater concordance between documented and patient-reported medication regimens and fewer unexplained discrepancies with potential for severe harm. There was no difference in the total number of preventable or ameliorable ADEs or in the duration of ameliorable ADEs.

We believe that the PG Medications Module encouraged patients to review their medication regimen as displayed in the ambulatory EHR and note discrepancies, and it activated PCPs to update the medication list. The results of the survey suggest that it may also have led to discussions about medication adherence and potential side effects. The intervention seemed particularly effective at decreasing unexplained discrepancies with potential for severe harm, perhaps because these caught the attention of PCPs and led to action.

On the other hand, some intervention patients still experienced potentially harmful medication discrepancies (0.24 per patient in the intervention arm). That 71% of eligible patients submitted a medications eJournal suggests that it was generally well accepted, but further improvements in usability might have increased that rate. We know that PCPs and practice staff did not open medications eJournals in 23% of cases, and they may not have accurately documented all they discussed with patients. Practices were expected to use the module when they signed up for the Prepare for Care study, but this decision was made by practice leaders with variable buy-in from individual providers. PCPs were trained in the use of the module and were encouraged to use it, but in the end, use was left to provider discretion. The module was designed to integrate with provider workflow (eg, automatically appearing instead of the usual medications screen when an eJournal had been submitted), but apparently some PCPs never opened a medications screen at all during some visits (eg, only opening the note-writing screen and importing the medication list into their note). Also, to our knowledge, the module was rarely used by other practice staff.

We were not surprised that the total number of preventable ADEs did not differ in the two study arms. Only a small number of ADEs were judged to be completely preventable (1.5% of all ADEs in this study), and causes are often multifactorial, requiring multi-faceted interventions. More feasible is decreasing the duration of ameliorable ADEs, but because only a proportion of ADEs in this study were ameliorable (58%), the study was likely under-powered to detect a difference.

Most EHR-connected PHRs facilitate patient–provider communication, completion of administrative tasks, and sharing of portions of the EHR with patients. Studies of such PHRs have shown effects on patient–physician collaboration¹⁸ and patient access to care.¹⁹ Studies of other interventions of ambulatory medication safety have focused on pharmacist interventions and mostly provider-centric health information technology (HIT) tools such as e-prescribing.^{20 21} To our knowledge, this study is one of the few examples of a patient-centric HIT tool focused on improving medication safety and one of the only successful ones.

This study has several limitations. First, it should be emphasized that this was an on-treatment analysis that evaluated a small proportion of patients compared with the over 120 000 patients theoretically eligible to participate in this study. To estimate the effect in all patients, the 9% absolute difference in the discordance rate should at least be multiplied by the 71% adherence rate (those who actually submitted a medications eJournal), that is, 6.4%. Because adherence might be less among the 82% of patients who had a PG account but chose not to participate in the study, the actual effect size would likely be smaller than that. Finally, this intervention was only effective among active PG users. At the time of the study, the rate of PG use was 18%. As of March 2011, with marketing and time, the rate was 35% and continues to increase.

This study has several other limitations. As suggested above, the low enrollment rate in PG in general and with this study in particular limits the generalizability of the findings, perhaps to particularly motivated early adopters of technology. The large number of people who did not participate in the medications sub-study also limits generalizability, although at least differences between participants and non-participants were similar in the two study arms. Second, this study was conducted in one medical system using internally developed software. However, the 11 primary care practices in this study had considerable heterogeneity, and such technology could be adopted by other EHRs that have associated PHRs. Third, as noted above, the sample size of the medications sub-study was small and had limited power to detect clinically important differences in some secondary outcomes. Fourth, we did not compare patient-reported medication regimens or documented regimens to a gold standard, that is, what a patient's providers collectively think the patient should be taking. This regimen does not usually exist anywhere and would require interviews with all providers (ideally, simultaneously) at the time of data collection.

Finally, there is no ideal comparison group to the patients in the intervention arm. On the one hand, a comparison of patients who submitted eJournals in both arms adjusts for potential confounding due to factors that lead to eJournal submission in general, such as comfort with technology and willingness to communicate asynchronously with providers. However, the reasons why a patient may submit a medications eJournal are likely different than the reasons why a patient may submit an eJournal about family history or health maintenance. Specifically, patients who submit a medications eJournal are prescribed more medications than those who do not, and they may be more likely to have potentially harmful medication discrepancies, an effect not seen among those offered a health maintenance eJournal. This is why we instead chose to match patients based on visit date, visit type, and comparable practice. In the secondary analysis of eJournal submitters only, the effects of the intervention on medication discordance were virtually identical to the main effects; the loss of statistical significance in discrepancies with potential for severe harm may reflect the difference described above between eJournal submitters and non-submitters in the different study arms, a loss of statistical power, unmeasured confounding, or a combination of these factors.

Future work is needed to increase patient enrollment in PHRs and to close the well-documented ‘digital divide’ between users and non-users of HIT.²² Improvements to the design of the patient side of the application might further increase submission of eJournals. Further work is also needed to better integrate the Medications Module with provider workflow, perhaps with further improvements to the PCP side of the application (eg, prompting review of the module from the EMR notes screen) and through changes in practice design (eg, team-based care in which other clinical staff members review medications eJournal information with patients prior to final vetting of the information by PCPs). Issues of provider expectations, training, and buy-in would also need to be addressed to maximize the effectiveness of the intervention. From a research standpoint, larger studies conducted in various healthcare systems and using multiple EHR/PHR platforms would establish the role of this kind of technology with greater generalizability.

In conclusion, when used, an interactive tool within a PHR focused on medications was able to decrease the discordance of documented and patient-reported medication regimens and reduce discrepancies with potential for severe harm. This technology shows promise, and further studies should be conducted integrating this kind of tool into other EHR platforms and evaluating its effects on medication safety.