Abstract
Background
Electronic patient portals are online applications that allow patients access to their own health information, a form of asynchronous virtual care. The long-term impact of portals on the use of traditional primary care services is unclear, but it is an important question at this juncture, when portals are being incorporated into many primary care practices.
Objective
We sought to investigate how an electronic patient portal affected the use of traditional, synchronous primary care services over a much longer time period than any existing studies and to assess the impact of portal messaging on clinicians’ workload.
Methods
We conducted a propensity-score–matched, open-cohort, interrupted time-series evaluation of a primary care portal from its implementation in 2010. We extracted information from the electronic medical record regarding age, sex, education, income, family health team enrollment, diagnoses at index date, and number of medications prescribed in the previous year. We also extracted the annual number of encounters for up to 8 years before and after the index date and provider time spent on secure messaging through the portal.
Results
A total of 7247 eligible portal patients and 7647 eligible potential controls were identified, with 3696 patients matched one to one. We found that portal registration was associated with an increase in the number of certain traditional encounters over the time period surrounding portal registration. Following the index year, there was a significant jump in annual number of visits to physicians in the portal arm (0.42 more visits/year vs control, P<.001) but not for visits to nurse practitioners and physician assistants. The annual number of calls to the practice triage nurses also showed a greater increase in the portal arm compared to the control arm after the index year (an additional 0.10 calls, P=.006). The average provider time spent on portal-related work was 5.7 minutes per patient per year.
Conclusions
We found that portal registration was associated with a subsequent increase in the number of some traditional encounters and an increase in clerical workload for providers. Portals have enormous potential to truly engage patients as partners in their own health care, but their impact on use of traditional health care services and clerical burden must also be considered when they are incorporated into primary care.
Keywords: electronic health records, health care utilization, patient portals, primary care, medical informatics, office visits, electronic, patient, online applications, virtual care, messaging, clinical, age, sex, education
Introduction
Electronic patient portals are online applications that allow patients access to their own health information, a form of asynchronous virtual care. There has been a great deal of recent interest in patient portals, accompanied by increasing technology adoption by both clinicians and patients [1-3]. The COVID-19 pandemic has also highlighted the importance of virtual care, an area already identified as a national health care priority [4]. Although portal features vary, the safe communication channels in portals may provide alternative ways for patients to obtain services traditionally provided in person, such as renewing prescriptions, sending and receiving secure messages, obtaining test results, and booking appointments [5]. A recent survey indicated that approximately 20% of Canadians had accessed some of their own medical information electronically, and that almost 80% were interested in doing so [6]. However, that survey did not specifically address portals or patient access to their medical information in primary care practice settings, and we are not aware of any studies examining Canadian portal adoption in primary care. Our understanding of the potential value of patient portals is nascent, with portals expected to contribute to more authentic collaboration between clinicians and patients.
The long-term impact of portals on traditional primary care services is unclear, but it is an important question at this juncture, when portals are being incorporated into primary care practices. Many studies reporting on the impact of portals on the use of traditional services evaluated systems that only provided options for web messaging or booking appointments [7-13]. All existing studies that investigated portals with more diverse features were conducted in medical networks, such as health maintenance organizations, where the portals provided access across sectors, including primary care, specialty care, and hospital care; these studies may not be relevant to portals incorporated into exclusively primary care practices. Past studies also reported inconsistent findings regarding the impact of portals on traditional health care use. Some studies demonstrated an increase in visits [14-16] or telephone calls [17]. Others demonstrated no change in visits [18], a reduction in visits [19], or a reduction in hospital readmissions [20]. All these studies also had limited time frames, examining only the period 12 to 30 months after portal registration.
To our knowledge, no long-term evaluation of the impact of a primary care patient portal on traditional health care use has been conducted to date. Providers have expressed interest in patient portals but also concerns regarding medicolegal risk and clerical workload [21]. Some have described an increased clerical burden associated with portals as part of the electronic health environment [12,22]. For instance, a qualitative study examining online patient access to their own health records found that providers felt that their workload had increased as a result [23], while another found that some providers anticipated fewer administrative requests for information when patients had access to their own health records [24]. One study found that online patient access to encounter notes did not significantly affect physician workload [25], although others have described high volumes of portal messages sent by patients [26]. However, no studies have actually tracked the provider time spent specifically on portal-related work. There have also not been any large studies of the impact of electronic patient portals in a Canadian setting. We sought to investigate how an electronic patient portal affects traditional, synchronous, primary care health care use over a much longer time period than any existing studies, and to assess the impact of portal messaging on clinicians’ workload.
Methods
We conducted a propensity-score–matched, open-cohort, interrupted time-series (ITS) evaluation of a primary care portal from its implementation in 2010.
Setting and Study Participants
The practice was a semirural interprofessional clinic in southeastern Ontario, Canada, where 12 family physicians and other allied health providers provide comprehensive primary care under a single-payer model. Under this publicly funded model, physician compensation is primarily through capitation payments for rostered patients. The primary care patient portal initially offered access to laboratory results, the ability to enter vital signs such as blood pressure measurements, and the ability to view when certain screening maneuvers were due. Additional features were introduced over time, including the ability to receive secure messages (in 2012), send secure messages (in 2015), book appointments (in 2016), and renew prescriptions (in 2018.) All practice patients were invited to join the portal via email, posters, and telephone reminders and at in-person encounters.
We collected data for all practice patients except those seen exclusively for focused care (eg, obstetrical care). We retained data from all patients only for the period they were aged 18 years or older. Among patients who adopted the portal, we excluded those for whom we did not have at least one year of data prior to and following their portal registration (ie, index) date. For non–portal patients, we excluded those who did not have at least two consecutive years of data between 2009 and 2019.
Matching
We calculated propensity scores to estimate the probability of individuals registering for the portal using logistic regression [27,28]. Propensity scores were derived based on sex, age, whether the patient was rostered to the family health team, the presence of specific diagnoses on the index date, and the number of in-person and telephone encounters, as well as the number of unique medications prescribed in the 12 months prior to the index date. During the study time period, all appointments with medical doctors (MDs), nurse practitioners (NPs), and physician assistants (PAs) were in person. Since education and income level were recorded for approximately a third of patients, these measures were not included in the propensity score matching. Control patients were entered into the equation for each year they were eligible (ie, for each year they had at least one year of data prior to and after the index date), with their corresponding profile for that year. July 1 of that year was considered the index date.
Variables, Data Sources, and Measurement
The study period was January 2002 through December 2019. We extracted electronic medical record information on patient age, sex, education, income, enrollment with the practice, and presence or absence of specific diagnoses on the index date. We also extracted the dates of in-person encounters with MDs, NPs, and PAs; dates of triage calls (TCs) to the practice triage nurses; and prescription dates and prescribed medications. Prescribed medications included only those that were identified as distinct medications using Anatomical Therapeutic Chemical codes. Diagnoses were defined based on diagnostic codes, using the earliest date when the diagnostic code was applied.
In order to study the clinician workload associated with the portal, two providers (KF and MF) time stamped their portal messages between February 20, 2020, and February 25, 2021. This allowed us to estimate the average provider time spent per message. We also collected the total number of portal messages sent by all providers to all portal patients between January 1, 2019, and December 31, 2019, in order to determine the average amount of time spent per patient on portal-related work.
Analyses
We described the profile of eligible patients prior to matching on their index date for portal patients on July 1 of the median year for which they were eligible to be matched for non–portal patients and again for the matched patients on their index date in both arms. The main study outcome was the frequency of in-person encounters with MDs, NPs, or PAs, as well as frequency of TCs. We used an ITS design to evaluate the impact of portal registration on use of these traditional health care services over time and compared use by portal users to their matched controls. We present the results in the usual ITS format, defining time relative to the index date with year 0 representing the 12 months preceding and including the index date and each time unit representing a 12-month interval. The ITS model includes time as a linear variable to model for an underlying linear time trend and the portal enrollment (ie, the intervention) as a dummy variable. Intervention and time interaction is also included in the model to identify the effect of the intervention on both arms (ie, portal and non–portal) over time.
We plotted the annual number of in-person encounters with MDs, in-person encounters with NPs or PAs, and TCs across time and overlaid the estimates derived from the ITS equations. During the study time period, all appointments with MDs, NPs, and PAs were in person. Because the year-0 results showed a spike in service use in both study groups, likely related to the attribution of the index date, we excluded that year from the ITS model. Also, although the spike in service use at year 1 in the portal arm may represent a transient change in behavior associated with the initial adoption, we also excluded this from the ITS to obtain a more reliable estimate of the impact of portal adoption over time, recognizing that this approach omits significant use; this should be considered in result interpretation.
We also depicted the number of visits per calendar year for patients who adopted the portal grouped by year of portal registration to demonstrate the pattern of changes in these visits over time for the intervention arm.
Ethics Approval
Ethics approval was received from the Bruyère Research Ethics Board (M16-20-012).
Results
Matching
Of the 14,894 patients who met the study criteria, 7247 (48.7%) were portal participants. Of these, 3696 were matched one to one with a control patient (Figure 1). The profile of all eligible patients before and after propensity matching is shown in Table 1. Before matching, portal users differed from non–portal users, but after matching, the mean propensity scores of the 2 groups and their index years, the prevalence of chronic conditions, sex, rostering status, and total visits and medications in the previous years showed good agreement. Income and education levels, which could not be included in the propensity score derivation because of poor data completeness, remained higher in the portal group.
Figure 1.
Study patient selection.
Table 1.
Portal and control patients before and after matching.
| Variable | Portal, index date (n=7247) | Non–portal, median index date (n=7647) | Total (n=14,894) |
P value |
Standard difference | Portal, index date (n=3696) | Control, index date (n=3696) | Total (n=7392) |
P value |
Standard difference | |||||||||||
| On index date |
|
|
|
|
|
||||||||||||||||
|
|
Propensity score, mean (SD) | 0.21 (0.10) | 0.10 (0.08) | 0.15 (0.11) | <.001 | 1.28 | 0.20 (0.10) | 0.20 (0.10) | 0.20 (0.10) | .82 | 0.01 | ||||||||||
|
|
Propensity score groups (participants), n (%) | <.001 |
|
|
.10 |
|
|||||||||||||||
|
|
|
0 | 291 (4) | 2687 (35.1) | 2978 (20) |
|
0.85 | 738 (20) | 740 (20) | 1478 (20) |
|
0 | |||||||||
|
|
|
1 | 947 (13.1) | 2032 (26.6) | 2979 (20) | 0.34 | 735 (19.9) | 743 (20.1) | 1478 (20) |
|
0.01 | ||||||||||
|
|
|
2 | 1349 (18.6) | 1631 (21.3) | 2980 (20) | 0.07 | 743 (20.1) | 736 (19.9) | 1479 (20) |
|
0 | ||||||||||
|
|
|
3 | 2010 (27.7) | 968 (12.7) | 2978 (20) | 0.38 | 740 (20) | 742 (20.1) | 1482 (20) |
|
0 | ||||||||||
|
|
|
4 | 2650 (36.6) | 329 (4.3) | 2979 (20) | 0.87 | 740 (20) | 735 (19.9) | 1475 (20) |
|
0 | ||||||||||
|
|
Index yeara (participants), n (%) | <.001 |
|
|
.99 |
|
|||||||||||||||
|
|
|
2010 | 1-5b | 148-152b | 153 (1) |
|
0.2 | 1-5b | 1-5b | 1-5b |
|
0 | |||||||||
|
|
|
2011 | 1335 (18.4) | 522 (6.8) | 1857 (12.5) | 0.35 | 654 (17.7) | 654 (17.7) | 1308 (17.7) |
|
0 | ||||||||||
|
|
|
2012 | 1240 (17.1) | 390 (5.1) | 1630 (10.9) | 0.39 | 579 (15.7) | 596 (16.1) | 1175 (15.9) |
|
0.01 | ||||||||||
|
|
|
2013 | 948 (13.1) | 349 (4.6) | 1297 (8.7) | 0.3 | 461 (12.5) | 452 (12.2) | 913 (12.4) |
|
0.01 | ||||||||||
|
|
|
2014 | 531 (7.3) | 3717 (48.6) | 4248 (28.5) | 1.04 | 296-300b | 275-279b | 576-580b |
|
0.02 | ||||||||||
|
|
|
2015 | 664 (9.2) | 838 (11) | 1502 (10.1) | 0.06 | 344 (9.3) | 353 (9.6) | 697 (9.4) |
|
0.01 | ||||||||||
|
|
|
2016 | 970 (13.4) | 670 (8.8) | 1640 (11) | 0.15 | 515 (13.9) | 518 (14) | 1033 (14) |
|
0 | ||||||||||
|
|
|
2017 | 793 (10.9) | 681 (8.9) | 1474 (9.9) | 0.07 | 464 (12.6) | 455 (12.3) | 919 (12.4) |
|
0.01 | ||||||||||
|
|
|
2018 | 761-765b | 328-332b | 1093 (7.3) | 0.24 | 378 (10.2) | 388 (10.5) | 766 (10.4) |
|
0.01 | ||||||||||
|
|
Age at index date (years), mean (SD) | 48.9 (14.9) | 45.2 (19.3) | 47.0 (17.4) | <.001 | 0.22 | 46.56 (15.17) | 46.18 (15.93) | 46.37 (15.56) | .29 | 0.02 | ||||||||||
|
|
Sex (participants), n (%) | <.001 | 0.22 |
|
.74 | 0.01 | |||||||||||||||
|
|
|
Female | 4334 (59.8) | 3748 (49) | 8082 (54.3) |
|
|
2104 (56.9) | 2090 (56.5) | 4194 (56.7) |
|
|
|||||||||
|
|
|
Male | 2913 (40.2) | 3899 (50.1) | 6812 (45.7) |
|
|
1592 (43.1) | 1606 (43.5) | 3198 (43.3) |
|
|
|||||||||
|
|
Rostered, n (%) | 7162 (98.8) | 7224 (94.5) | 14,386 (96.6) | <.001 | 0.24 | 3665 (99.2) | 3652 (98.8) | 7317 (99) | .13 | 00.04 | ||||||||||
|
|
Coronary artery disease, n (%) | 243 (3.4) | 268 (3.5) | 511 (3.4) | .61 | 0.01 | 68 (1.8) | 79 (2.1) | 147 (2) | .36 | 0.02 | ||||||||||
|
|
Congestive heart failure, n (%) | 79 (1.1) | 112 (1.5) | 191 (1.3) | .04 | 0.03 | 24 (0.6) | 31 (0.8) | 55 (0.7) | .34 | 0.02 | ||||||||||
|
|
Chronic obstructive pulmonary disease, n (%) | 128 (1.8) | 235 (3.1) | 363 (2.4) | <.001 | 0.09 | 43 (1.2) | 54 (1.5) | 97 (1.3) | .26 | 0.03 | ||||||||||
|
|
Diabetes mellitus, n (%) | 440 (6.1) | 482 (6.3) | 922 (6.2) | .56 | 0.01 | 149 (4) | 154 (4.2) | 303 (4.1) | .77 | 0.01 | ||||||||||
|
|
Hypertension, n (%) | 1343 (18.5) | 1190 (15.6) | 2533 (17) | <.001 | 0.08 | 495 (13.4) | 474 (12.8) | 969 (13.1) | .47 | 0.02 | ||||||||||
|
|
Income level (CAD $)c,d (participants), n (%) | <.001 |
|
|
<.001 |
|
|||||||||||||||
|
|
|
<40,000 | 233 (8.7) | 432 (15.8) | 665 (12.3) |
|
0.22 | 121 (8.7) | 150 (12.2) | 271 (10.4) |
|
0.11 | |||||||||
|
|
|
40,000-60,000 | 357 (13.3) | 484 (17.7) | 841 (15.5) | 0.12 | 173 (12.5) | 218 (17.7) | 391 (14.9) |
|
0.15 | ||||||||||
|
|
|
60,000-100,000 | 846 (31.6) | 836 (30.5) | 1682 (31) | 0.02 | 437 (31.5) | 397 (32.3) | 834 (31.9) |
|
0.02 | ||||||||||
|
|
|
>100,000 | 1242 (46.4) | 989 (36.1) | 2231 (41.2) | 0.21 | 657 (47.3) | 465 (37.8) | 1122 (42.9) |
|
0.19 | ||||||||||
|
|
Education levelc (participants), n (%) | <.001 |
|
|
<.001 |
|
|||||||||||||||
|
|
|
High school or less | 526 (21) | 1037 (39.7) | 1563 (30.5) |
|
0.41 | 284 (21.7) | 380 (32) | 664 (26.6) |
|
0.23 | |||||||||
|
|
|
College | 812 (32.4) | 745 (28.5) | 1557 (30.4) | 0.08 | 397 (30.4) | 381 (32.1) | 778 (31.2) |
|
0.04 | ||||||||||
|
|
|
University or more | 1363 (50.5) | 979 (35.5) | 2342 (42.9) | 0.31 | 626 (47.9) | 427 (35.9) | 1053 (42.2) |
|
0.24 | ||||||||||
| In the 12 months prior to index date | |||||||||||||||||||||
|
|
Medical doctor visitse, n (%) | <.001 |
|
|
.07 |
|
|||||||||||||||
|
|
|
0 | 1068 (14.7) | 3328 (43.5) | 4396 (29.5) |
|
0.67 | 732 (19.8) | 648 (17.5) | 1380 (18.7) |
|
0.06 | |||||||||
|
|
|
1-2 | 3758 (51.9) | 2548 (33.3) | 6306 (42.3) | 0.38 | 1969 (53.3) | 2079 (56.3) | 4048 (54.8) |
|
0.06 | ||||||||||
|
|
|
3-5 | 1809 (25) | 1251 (16.4) | 3060 (20.5) | 0.21 | 776 (21) | 750 (20.3) | 1526 (20.6) |
|
0.02 | ||||||||||
|
|
|
6-10 | 532 (7.3) | 437 (5.7) | 969 (6.5) | 0.07 | 189 (5.1) | 187 (5.1) | 376 (5.1) |
|
0 | ||||||||||
|
|
|
>11 | 80 (1.1) | 83 (1.1) | 163 (1.1) | 0 | 30 (0.8) | 32 (0.9) | 62 (0.8) |
|
0.01 | ||||||||||
|
|
Nurse practitioner or physician assistant visitse, n (%) | <.001 |
|
|
.34 |
|
|||||||||||||||
|
|
|
0 | 3959 (54.6) | 5213 (68.2) | 9172 (61.6) |
|
0.28 | 2272 (61.5) | 2297 (62.1) | 4569 (61.8) |
|
0.01 | |||||||||
|
|
|
1-2 | 2655 (36.6) | 2009 (26.3) | 4664 (31.3) | 0.22 | 1232 (33.3) | 1234 (33.4) | 2466 (33.4) |
|
0 | ||||||||||
|
|
|
>3 | 633 (8.7) | 425 (5.6) | 1058 (7.1) | 0.12 | 192 (5.2) | 165 (4.5) | 357 (4.8) |
|
0.03 | ||||||||||
|
|
Calls to triage nursese, n (%) | <.001 |
|
|
.40 |
|
|||||||||||||||
|
|
|
0 | 4990 (68.9) | 5886 (77) | 10,876 (73) |
|
0.18 | 2827 (76.5) | 2871 (77.7) | 5698 (77.1) |
|
0.03 | |||||||||
|
|
|
1-2 | 1898 (26.2) | 1465 (19.2) | 3363 (22.6) | 0.17 | 759 (20.5) | 728 (19.7) | 1487 (20.1) |
|
0.02 | ||||||||||
|
|
|
>3 | 359 (5) | 296 (3.9) | 655 (4.4) | 0.05 | 110 (3) | 97 (2.6) | 207 (2.8) |
|
0.02 | ||||||||||
|
|
Medications prescribede, n (%) | <.001 |
|
|
.78 |
|
|||||||||||||||
|
|
|
0 | 1705 (23.5) | 3429 (44.8) | 5134 (34.5) |
|
0.46 | 1131 (30.6) | 1132 (30.6) | 2263 (30.6) |
|
0 | |||||||||
|
|
|
1-2 | 2522 (34.8) | 2042 (26.7) | 4564 (30.6) | 0.18 | 1399 (37.9) | 1439 (38.9) | 2838 (38.4) |
|
0.02 | ||||||||||
|
|
|
3-5 | 1928 (26.6) | 1272 (16.6) | 3200 (21.5) | 0.24 | 817 (22.1) | 792 (21.4) | 1609 (21.8) |
|
0.02 | ||||||||||
|
|
|
6-10 | 906 (12.5) | 701 (9.2) | 1607 (10.8) | 0.11 | 288 (7.8) | 281 (7.6) | 569 (7.7) |
|
0.01 | ||||||||||
|
|
|
>11 | 186 (2.6) | 203 (2.7) | 389 (2.6) | 0.01 | 61 (1.7) | 52 (1.4) | 113 (1.5) | .82 | 0.02 | ||||||||||
aIndex date for unmatched control patients: July 1 of the median year of their eligible time period.
bThe value of n for 2010 was smaller than 6. The cells for 2010, 2014, and 2018 therefore do not have precise values due to ethics agreements.
cIncome and education level were not available for all patients.
dCAD $1.00=US $0.75 on January 12, 2023.
eNumber of medications prescribed and number of encounters refer to the 12-month period prior to the index date. The medication records were mapped to the Drug Product Database to assign Anatomical Therapeutic Chemical codes and schedules based on drug identification numbers. We excluded 12.5% of medication records, including records for medications classified as “over-the-counter” or “ethical” in the schedules from the Drug Product Database (4.9%) and medications reclassified manually as “over-the-counter,” “other,” or “N/A” (4.7%). Medications with no drug identification number were classified manually. Those which could not be attributed a drug identification number were also excluded (2.9%). Variable categorization for number of visits, telephone calls and medications was based on clinical judgement and the number of participants in each category.
We used a caliper of 0.2 for matching and limited the potential matching pool for each portal patient to non–portal patients with an index date that was within 1 year of the portal patient’s index date. We identified all potential controls for each portal patient and assigned matches prioritizing first portal patients who had a unique match, then non–portal patients who had a unique portal match. We repeated this after each match to minimize loss of controls. When more than one match was possible, we attributed the control patient whose propensity score was closest to the portal patient’s score. The balance of baseline covariates between the matched portal users and non–portal users was assessed using standardized differences, with values <0.1 representing negligible differences.
Analyses
We plotted the number of visits in relation to the index year for portal and control patients, the estimated slopes for the years before and after the transition, and the shift in visits at the index date derived from the ITS equation (Figures 2-4). The information for the years prior to the index date for portal patients demonstrates their annual visits for the years prior to their registration on the portal. The index date for the control patients was assigned to be within 1 year of the portal’s patient index date in order to control for temporal factors such as health care use trends. The outputs of the ITS analyses are provided in Table 2. The intercepts and slopes prior to the index year were similar in the control and portal arms for MDs, NPs/PAs, and TCs (P>.05). After the index year, there was a significant jump in MD visits in the portal arm (0.42 more visits/year vs control, P<.001) but not for NP or PA visits. The TCs also showed a greater increase in visits in the portal arm compared to the control arm after the index year (0.102 more visits/year vs control, P=.006).
Figure 2.
Interrupted time series for MD face-to-face visits for portal patients versus controls. The intercepts (P=.86) and slopes (P=.15) prior to the index year were similar in the control and portal arms. After the index year, there was no significant change in the number of MD visits in the control arm. However, in the portal arm, there was a significant jump in number of visits and a new intercept (0.42 more visits/year vs control, P<.001). The slope for MD visits increased after the index date in the control arm but became negative in the portal arm, representing an annual reduction of 0.054 visits per year for the portal arm compared to the control arm (P=.001). The two slopes would be expected to cross after 10 years. CLI: confidence limit interval; MD: medical doctor.
Figure 4.
Interrupted time series for triage calls for portal patients versus controls. The intercepts (P=.10) and slopes (P=.26) prior to the index year were similar in the control and portal arms. The number of triage calls following the index year showed a higher value than anticipated based on the preindex slope in the control arm (0.062 more calls annually, P=.02), but a significantly greater jump after the index year in the portal arm (0.10 more calls annually, P=.006). The slopes for annual triage calls were similar in the pre- and postindex periods for both the control arm and portal arm. CLI: confidence limit interval.
Table 2.
Outputs of the interrupted time series. “Annual visits” indicates slope; “period” indicates the pre- or postindex period.
| Variable | Medical doctor visits | Nurse practitioner or physician assistant visits | Triage calls | |||||
|
|
Estimate | P value | Estimate | P value | Estimate | P value | ||
| Intercepta | 1.572 | <.001 | 0.618 | <.001 | 0.335 | <.001 | ||
| Annual visits (slope)b | 0.009 | .19 | 0.049 | <.001 | 0.045 | <.001 | ||
| Period (before or after index)c | 0.002 | .97 | -0.011 | .77 | 0.062 | .02 | ||
| Annual visits × periodd | 0.036 | .002 | -0.042 | <.001 | 0.010 | .05 | ||
| Portale | 0.008 | .86 | -0.018 | .59 | –0.033 | .10 | ||
| Portal × annual visitsf | 0.014 | .15 | -0.010 | .12 | –0.005 | .26 | ||
| Portal × periodg | 0.417 | <.001 | 0.069 | .21 | 0.102 | .006 | ||
| Portal × annual visits × periodh | –0.054 | .001 | 0.028 | .01 | –0.005 | .46 | ||
aControl arm intercept.
bPre–index date slope of annual visits for the control arm.
cChange in number of visits in year 2 post–index date relative to that anticipated from preindex slope for the control arm.
dChange in the slope of annual visits in the postindex period relative to the preindex period for the control arm.
eDifference between the portal arm and control arm in the intercept.
fDifference between the portal arm and control arm in the pre–index date slope of annual visits.
gDifference between the portal arm and control arm in the change in number of visits in year 2 post–index date relative to that anticipated from preindex slope.
hDifference between the portal arm and control arm in the change of the slope of annual visits in the postindex period relative to the preindex period.
Figure 3.
Interrupted time series for nurse practitioner or physician assistant face-to-face visits for portal patients versus controls. The intercepts (P=.59) and slopes (P=.12) prior to the index year were similar in the control and portal arms. After the index year, there was not a significant change in the number of nurse practitioner or physician assistant visits in the portal arm compared to the control arm (P=.21). The slope flattened after the index date in the control arm, but it was relatively unchanged in the portal arm, demonstrating an annual increase of 0.028 visits per year in the portal arm compared to the control arm (P=.01). CLI: confidence limit interval; PA: physician assistant; NP; nurse practitioner.
We also plotted the visit rates for each year for patients having enrolled in the portal, grouped by year of portal registration (Figures 5-7).
Figure 5.
Number of visits to medical doctors per calendar year for each patient group (registered on the portal in 2011-2012, 2013-2014, 2015-2016, and 2017-2018). To reduce noise, the number of visits represents the running average of that year, the previous year, and the following year. The MD visits showed a slight increase in the number of visits in the years immediately following portal registration, followed by an apparent drop in annual rate of visits.
Figure 7.
Number of triage calls per calendar year for each patient group (registered on the portal in 2011-2012, 2013-2014, 2015-2016, and 2017-2018). To reduce noise, the number of visits represents the running average of that year, the previous year, and the following year. The nurse triage calls were introduced in 2010 and show a consistent rise in frequency over time with a small increase in calls associated with the year of portal registration.
The 2 physicians who time stamped 2061 portal messages spent an average of 3.83 minutes on each message. We also extracted the total number of portal messages sent by all providers between January 1 and December 31, 2019, and found that an average of 1.49 messages were sent to each portal patient in the practice. Thus, the average amount of provider time devoted to portal messages was estimated to be 5.7 minutes per portal patient per year.
Figure 6.
Number of visits to nurse practitioners or physician assistants for each patient group (registered on the portal in 2011-2012, 2013-2014, 2015-2016, and 2017-2018). To reduce noise, the number of visits represents the running average of that year, the previous year, and the following year. The NP and PA visits began in 2006 and show a rapid rise in the number of visits until 2010, then a considerable flattening of that slope afterwards with a potential small spike following the year of registration.
Discussion
Main Findings
Our findings suggest that portal registration is associated with an increase in service use, but that some reductions may be expected over subsequent years. Compared to matched controls, portal registration was associated with a significant initial increase in the number of in-person MD encounters and telephone calls, but a subsequent drop in the rate of MD visits and increase in NP visits over time. MDs spent an estimated 5.7 minutes per patient annually to respond to portal messages.
Limitations and Comparison With Prior Work
We believe that ours is the first study to examine the trend in encounters after portal registration over an extended time span and the first study to examine the impact of an exclusively primary care portal on traditional health care usage. It is possible that the observed increase in encounters was due to differences between the two groups that were not captured in the propensity matching. For instance, patients might have registered on the portal when they developed a new health concern, anticipating an increased requirement for health services. The reason for the gradual decrease in MD visits but increase in NP visits that took place after the initial jump in MD visits associated with portal registration is difficult to determine without further study. It is possible that patients initially presented to their own physician after sending them a portal message or viewing results, but the physician shared follow-up care with the nurse practitioner or physician assistant.
There may have been differences in areas such as electronic literacy or internet access that were not identified. It is also possible that the higher frequency of in-person encounters after portal registration was due to increased engagement by patients in their health. For instance, access to laboratory results may have generated questions from patients [29]. Increased awareness of being due for cancer screening or diabetes or blood pressure monitoring may have resulted in a higher number of encounters but improved quality of care or patient satisfaction. We did not examine these areas as they were beyond the scope of this study, but they would benefit from future research. While some past studies demonstrated improvements in certain health outcomes associated with electronic patient portals [30-33], only a few were based in primary care [31,34]. Several systematic reviews that evaluated a variety of portals in different practice settings suggested that portals or similar digital health services may result in improved patient satisfaction, but they did not demonstrate a meaningful impact on health outcomes, cost, or use [35-40].
We found that providers spent less than 6 minutes per year on clerical work for each patient registered on the portal. This is a small amount of time per patient but is significant when considering the context of an entire primary care practice. We note that the time-stamping of messages was performed during the COVID-19 pandemic, while the number of messages sent by all providers was collected prior to 2020. We consider that even if the COVID-19 pandemic resulted in an increased number of messages, the provider time per message would not have changed significantly. Therefore, our estimate of portal-related clerical work reflects prepandemic time requirements, and these may have increased since 2020 due to increased patient interest in asynchronous virtual care. This would also be an area for further study. Portals that do not allow incoming messages or any secure messaging would reduce or eliminate this time requirement but might also limit patient engagement and other potential benefits of the portal. Since the clerical burden associated with electronic environments in health care has been associated with professional burnout, [22,26] it is important to consider the provider time requirement associated with patient portals. The time and cost associated with incorporating a patient portal are currently not specifically addressed in either fee-for-service or capitated Canadian primary care funding models.
There are other limitations to this study. We examined the long-term impact of an electronic patient portal in a single primary care practice, which may not be reflective of the impact in other primary care practices. However, portal adoption has not been widespread for long enough to allow study of the long-term impact of portals across multiple sites. Additionally, the impact of patient portals in other settings, such as hospitals, laboratories, or specialist practices, may be quite different. Further research is needed into electronic patient portals in different settings to determine their impact on various health outcomes.
Conclusions
Electronic patient portals are increasingly being adopted by providers and sought after by patients. We found that portal registration was associated with a subsequent increase in the number of some traditional encounters and a small increase in clerical workload for providers. Portals have enormous potential to truly engage patients as partners in their own health care, but their impact on use of traditional health care services and clerical burden must also be considered when they are incorporated into primary care.
Acknowledgments
This study was funded by a CAD $20,000 (US $15,000) research grant from the University of Ottawa Department of Family Medicine Program for Research and Innovation in Primary Care and Medical Education.
Abbreviations
- ITS
interrupted time-series
- MD
medical doctor
- NP
nurse practitioner
- PA
physician assistant
- TC
triage call
Footnotes
Conflicts of Interest: None declared.
References
- 1.Ryan B, Brown J, Terry A, Cejic S, Stewart M, Thind A. Implementing and using a patient portal: A qualitative exploration of patient and provider perspectives on engaging patients. J Innov Health Inform. 2016 Jul 04;23(2):848. doi: 10.14236/jhi.v23i2.848. https://informatics.bmj.com/lookup/pmidlookup?view=long&pmid=27869582 . [DOI] [PubMed] [Google Scholar]
- 2.Stone JH. Communication between physicians and patients in the era of E-medicine. N Engl J Med. 2007 Jun 14;356(24):2451–4. doi: 10.1056/NEJMp068198.356/24/2451 [DOI] [PubMed] [Google Scholar]
- 3.Gorfinkel Iris, Lexchin J. Enabling patient portals to access primary care medical records: maximizing collaboration in care between patients and providers. Healthc Policy. 2019 May;14(4):21–27. doi: 10.12927/hcpol.2019.25859. https://europepmc.org/abstract/MED/31322111 .hcpol.2019.25859 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Virtual Care: Recommendations for Scaling Up Virtual Medical Services. Canadian Medical Association. [2023-01-12]. https://www.cma.ca/sites/default/files/pdf/virtual-care/ReportoftheVirtualCareTaskForce.pdf .
- 5.Archer N, Fevrier-Thomas U, Lokker C, McKibbon KA, Straus SE. Personal health records: a scoping review. J Am Med Inform Assoc. 2011;18(4):515–22. doi: 10.1136/amiajnl-2011-000105. https://europepmc.org/abstract/MED/21672914 .amiajnl-2011-000105 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.2019 Canadian Digital Health Survey. Canada Health Infoway. [2023-01-12]. https://dataverse.scholarsportal.info/dataset.xhtml?persistentId=doi:10.5683/SP2/SPOAWK .
- 7.Stamenova V, Agarwal P, Kelley L, Fujioka J, Nguyen M, Phung M, Wong I, Onabajo N, Bhatia RS, Bhattacharyya O. Uptake and patient and provider communication modality preferences of virtual visits in primary care: a retrospective cohort study in Canada. BMJ Open. 2020 Jul 06;10(7):e037064. doi: 10.1136/bmjopen-2020-037064. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=32636284 .bmjopen-2020-037064 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Bergmo TS, Kummervold PE, Gammon D, Dahl LB. Electronic patient-provider communication: will it offset office visits and telephone consultations in primary care? Int J Med Inform. 2005 Sep;74(9):705–10. doi: 10.1016/j.ijmedinf.2005.06.002.S1386-5056(05)00066-3 [DOI] [PubMed] [Google Scholar]
- 9.Liederman EM, Lee JC, Baquero VH, Seites PG. Patient-physician web messaging. The impact on message volume and satisfaction. J Gen Intern Med. 2005 Jan;20(1):52–7. doi: 10.1111/j.1525-1497.2005.40009.x. https://europepmc.org/abstract/MED/15693928 .JGI40009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Zhou YY, Garrido T, Chin HL, Wiesenthal AM, Liang LL. Patient access to an electronic health record with secure messaging: impact on primary care utilization. Am J Manag Care. 2007 Jul;13(7):418–24. https://www.ajmc.com/pubMed.php?pii=3340 .3340 [PubMed] [Google Scholar]
- 11.Shimada S, Hogan T, Rao S, Allison J, Quill A, Feng H, Phillips Barrett D, Nazi Kim M, Haidary Susan T, Houston Thomas K. Patient-provider secure messaging in VA: variations in adoption and association with urgent care utilization. Med Care. 2013 Mar;51(3 Suppl 1):S21–8. doi: 10.1097/MLR.0b013e3182780917.00005650-201303001-00007 [DOI] [PubMed] [Google Scholar]
- 12.North Frederick, Luhman Kristine E, Mallmann Eric A, Mallmann Toby J, Tulledge-Scheitel Sidna M, North Emily J, Pecina Jennifer L. A retrospective analysis of provider-to-patient secure messages: how much are they increasing, who is doing the work, and is the work happening after hours? JMIR Med Inform. 2020 Jul 08;8(7):e16521. doi: 10.2196/16521. https://medinform.jmir.org/2020/7/e16521/ v8i7e16521 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Meng D, Palen TE, Tsai J, McLeod M, Garrido T, Qian H. Association between secure patient-clinician email and clinical services utilisation in a US integrated health system: a retrospective cohort study. BMJ Open. 2015 Nov 09;5(11):e009557. doi: 10.1136/bmjopen-2015-009557. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=26553841 .bmjopen-2015-009557 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Palen TE, Ross C, Powers JD, Xu S. Association of online patient access to clinicians and medical records with use of clinical services. JAMA. 2012 Nov 21;308(19):2012–9. doi: 10.1001/jama.2012.14126.1392562 [DOI] [PubMed] [Google Scholar]
- 15.Zhou Y, Leith W, Li H, Tom J. Personal health record use for children and health care utilization: propensity score-matched cohort analysis. J Am Med Inform Assoc. 2015 Jul;22(4):748–54. doi: 10.1093/jamia/ocu018.ocu018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Blok AC, Amante DJ, Hogan TP, Sadasivam RS, Shimada SL, Woods S, Nazi KM, Houston TK. Impact of patient access to online VA notes on healthcare utilization and clinician documentation: a retrospective cohort study. J Gen Intern Med. 2021 Mar;36(3):592–599. doi: 10.1007/s11606-020-06304-0. https://europepmc.org/abstract/MED/33443693 .10.1007/s11606-020-06304-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Dexter EN, Fields S, Rdesinski RE, Sachdeva B, Yamashita D, Marino M. Patient-provider communication: does electronic messaging reduce incoming telephone calls? J Am Board Fam Med. 2016;29(5):613–9. doi: 10.3122/jabfm.2016.05.150371. http://www.jabfm.org/cgi/pmidlookup?view=long&pmid=27613794 .29/5/613 [DOI] [PubMed] [Google Scholar]
- 18.Leveille S, Mejilla R, Ngo L, Fossa A, Elmore J, Darer J, Ralston James D, Delbanco Tom, Walker Jan. Do patients who access clinical information on patient internet portals have more primary care visits? Med Care. 2016 Jan;54(1):17–23. doi: 10.1097/MLR.0000000000000442. [DOI] [PubMed] [Google Scholar]
- 19.Zhong X, Liang M, Sanchez R, Yu M, Budd PR, Sprague JL, Dewar MA. On the effect of electronic patient portal on primary care utilization and appointment adherence. BMC Med Inform Decis Mak. 2018 Oct 16;18(1):84. doi: 10.1186/s12911-018-0669-8. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-018-0669-8 .10.1186/s12911-018-0669-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Martínez Nicolás Ismael, Lê Cook Benjamin, Flores M, Del Olmo Rodriguez Marta, Hernández Rodríguez Corazón, Llamas Sillero Pilar, Baca-Garcia Enrique. The impact of a comprehensive electronic patient portal on the health service use: an interrupted time-series analysis. Eur J Public Health. 2019 Jun 01;29(3):413–418. doi: 10.1093/eurpub/cky257.5240936 [DOI] [PubMed] [Google Scholar]
- 21.Mehta S, Jamieson T, Ackery A. Helping clinicians and patients navigate electronic patient portals: ethical and legal principles. CMAJ. 2019 Oct 07;191(40):E1100–E1104. doi: 10.1503/cmaj.190413. http://www.cmaj.ca/cgi/pmidlookup?view=long&pmid=31591096 .191/40/E1100 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Shanafelt TD, Dyrbye LN, Sinsky C, Hasan O, Satele D, Sloan J, West CP. Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc. 2016 Jul;91(7):836–48. doi: 10.1016/j.mayocp.2016.05.007.S0025-6196(16)30215-4 [DOI] [PubMed] [Google Scholar]
- 23.Turner A, Morris R, McDonagh L, Hamilton F, Blake S, Farr M, Stevenson F, Banks J, Atherton H, Rakhra D, Lasseter G, Feder G, Ziebland S, Hyde E, Powell J, Horwood J. Unintended consequences of patient online access to health records: a qualitative study in UK primary care. Br J Gen Pract. 2023 Jan;73(726):e67–e74. doi: 10.3399/BJGP.2021.0720. https://europepmc.org/abstract/MED/36316163 .BJGP.2021.0720 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Louch G, Albutt A, Smyth K, O'Hara Jane K. What do primary care staff think about patients accessing electronic health records? A focus group study. BMC Health Serv Res. 2022 Apr 29;22(1):581. doi: 10.1186/s12913-022-07954-y. https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-022-07954-y .10.1186/s12913-022-07954-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Delbanco T, Walker J, Bell SK, Darer JD, Elmore JG, Farag N, Feldman HJ, Mejilla R, Ngo L, Ralston JD, Ross SE, Trivedi N, Vodicka E, Leveille SG. Inviting patients to read their doctors' notes: a quasi-experimental study and a look ahead. Ann Intern Med. 2012 Oct 02;157(7):461–70. doi: 10.7326/0003-4819-157-7-201210020-00002. https://www.acpjournals.org/doi/abs/10.7326/0003-4819-157-7-201210020-00002?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub0pubmed .1363511 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Chavez A, Bracamonte J, Kresin M, Yardley M, Grover M. High volume portal usage impacts practice resources. J Am Board Fam Med. 2020;33(3):452–455. doi: 10.3122/jabfm.2020.03.190401. http://www.jabfm.org/cgi/pmidlookup?view=long&pmid=32430378 .33/3/452 [DOI] [PubMed] [Google Scholar]
- 27.Austin PC. Comparing paired vs non-paired statistical methods of analyses when making inferences about absolute risk reductions in propensity-score matched samples. Stat Med. 2011 May 20;30(11):1292–301. doi: 10.1002/sim.4200. https://europepmc.org/abstract/MED/21337595 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. 2011 May;46(3):399–424. doi: 10.1080/00273171.2011.568786. https://europepmc.org/abstract/MED/21818162 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Pillemer F, Price RA, Paone S, Martich GD, Albert S, Haidari L, Updike G, Rudin R, Liu D, Mehrotra A. Direct release of test results to patients increases patient engagement and utilization of care. PLoS One. 2016;11(6):e0154743. doi: 10.1371/journal.pone.0154743. https://dx.plos.org/10.1371/journal.pone.0154743 .PONE-D-14-55360 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Green BB, Cook Andrea J, Ralston James D, Fishman Paul A, Catz Sheryl L, Carlson James, Carrell David, Tyll Lynda, Larson Eric B, Thompson Robert S. Effectiveness of home blood pressure monitoring, Web communication, and pharmacist care on hypertension control: a randomized controlled trial. JAMA. 2008 Jun 25;299(24):2857–67. doi: 10.1001/jama.299.24.2857. https://europepmc.org/abstract/MED/18577730 .299/24/2857 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Nagykaldi Z, Aspy CB, Chou A, Mold JW. Impact of a Wellness Portal on the delivery of patient-centered preventive care. J Am Board Fam Med. 2012;25(2):158–67. doi: 10.3122/jabfm.2012.02.110130. http://www.jabfm.org/cgi/pmidlookup?view=long&pmid=22403196 .25/2/158 [DOI] [PubMed] [Google Scholar]
- 32.Ralston J, Hirsch I, Hoath J, Mullen M, Cheadle A, Goldberg H. Web-based collaborative care for type 2 diabetes: a pilot randomized trial. Diabetes Care. 2009 Feb;32(2):234–9. doi: 10.2337/dc08-1220. https://europepmc.org/abstract/MED/19017773 .dc08-1220 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Wright A, Poon EG, Wald J, Feblowitz J, Pang JE, Schnipper JL, Grant RW, Gandhi TK, Volk LA, Bloom A, Williams DH, Gardner K, Epstein M, Nelson L, Businger A, Li Q, Bates DW, Middleton B. Randomized controlled trial of health maintenance reminders provided directly to patients through an electronic PHR. J Gen Intern Med. 2012 Jan;27(1):85–92. doi: 10.1007/s11606-011-1859-6. https://europepmc.org/abstract/MED/21904945 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Tom JO, Chen C, Zhou YY. Personal health record use and association with immunizations and well-child care visits recommendations. J Pediatr. 2014 Jan;164(1):112–7. doi: 10.1016/j.jpeds.2013.08.046.S0022-3476(13)01074-3 [DOI] [PubMed] [Google Scholar]
- 35.Ammenwerth E, Schnell-Inderst P, Hoerbst A. The impact of electronic patient portals on patient care: a systematic review of controlled trials. J Med Internet Res. 2012 Nov 26;14(6):e162. doi: 10.2196/jmir.2238. https://www.jmir.org/2012/6/e162/ v14i6e162 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Goldzweig CL, Orshansky G, Paige NM, Towfigh AA, Haggstrom DA, Miake-Lye I, Beroes JM, Shekelle PG. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med. 2013 Nov 19;159(10):677–87. doi: 10.7326/0003-4819-159-10-201311190-00006.1770672 [DOI] [PubMed] [Google Scholar]
- 37.Kruse CS, Bolton K, Freriks G. The effect of patient portals on quality outcomes and its implications to meaningful use: a systematic review. J Med Internet Res. 2015 Feb 10;17(2):e44. doi: 10.2196/jmir.3171. https://www.jmir.org/2015/2/e44/ v17i2e44 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Zanaboni P, Fagerlund AJ. Patients' use and experiences with e-consultation and other digital health services with their general practitioner in Norway: results from an online survey. BMJ Open. 2020 Jun 17;10(6):e034773. doi: 10.1136/bmjopen-2019-034773. https://bmjopen.bmj.com/lookup/pmidlookup?view=long&pmid=32554721 .bmjopen-2019-034773 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Mold F, de Lusignan S, Sheikh A, Majeed A, Wyatt JC, Quinn T, Cavill M, Franco C, Chauhan U, Blakey H, Kataria N, Arvanitis TN, Ellis B. Patients’ online access to their electronic health records and linked online services: a systematic review in primary care. Br J Gen Pract. 2015 Mar 02;65(632):e141–e151. doi: 10.3399/bjgp15x683941. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Neves AL, Freise L, Laranjo L, Carter AW, Darzi A, Mayer E. Impact of providing patients access to electronic health records on quality and safety of care: a systematic review and meta-analysis. BMJ Qual Saf. 2020 Dec;29(12):1019–1032. doi: 10.1136/bmjqs-2019-010581. http://qualitysafety.bmj.com/lookup/pmidlookup?view=long&pmid=32532814 .bmjqs-2019-010581 [DOI] [PMC free article] [PubMed] [Google Scholar]
