Abstract
Background The patient portal allows patients to engage with their health care team beyond the clinical encounter. While portals can improve patient outcomes, there may be disparities in which patients access the portal by sociodemographic factors. Understanding the characteristics of patients who use the portal could help design future interventions to expand portal adoption.
Objectives This study aimed to (1) examine the socioeconomic factors, comorbid conditions, and health care utilization among patients of a large academic primary care network who are users and non-users of the patient portal; and (2) describe the portal functions most frequently utilized.
Methods We included all adult patients at Atrium Health Wake Forest Baptist who had at least two primary care visits between 2018 and 2019. Patients' demographics, comorbidities, health care utilization, and portal function usage were extracted from the electronic health record and merged with census data (income, education, and unemployment) from the American Community Survey. A myWakeHealth portal user was defined as a patient who used a bidirectional portal function at least once during the study period. We used multivariable logistic regression to determine which patient characteristics were independently associated with being a portal user.
Results Of the 178,720 patients who met inclusion criteria, 32% ( N = 57,122) were users of myWakeHealth. Compared to non-users, users were more likely to be 18 to 64 years of age, female, non-Hispanic White, married, commercially insured, have higher disease burden, and have lower health care utilization. Patients residing in areas with the highest educational attainment had 51% higher odds of being a portal user than the lowest ( p <0.001). Among portal users, the most commonly used function was messaging clinic providers.
Conclusion We found that patient demographics and area socioeconomic factors were associated with patient portal adoption. These findings suggest that efforts to improve portal adoption should be targeted at vulnerable patients.
Keywords: patient portal, academic medical center, ambulatory practice, electronic health record
Background and Significance
The electronic health record (EHR) is a source of substantial patient clinical and demographic data, 1 and patient portals tethered to the EHR provide clinicians with a tool to communicate and share information with patients as well as to collect patient-reported outcomes. 2 The Medicare and Medicaid EHR Incentive Programs were created by the 2009 Health Information Technology for Economic and Clinical Health (HITECH) Act, which presented financial incentives to an eligible professional or eligible hospital who used certified EHR technology (CEHRT). 3 4 The incentive programs have since transformed into the Promoting Interoperability (PI) Programs. 5 6 The PI Programs' reporting measures are intended to capture improvements in patient care quality and health outcomes. To receive incentives specific to CEHRT, the PI Programs required eligible professionals and hospitals to report on secure messaging with patients and allow patients to view their laboratory test results, assessments, and treatment plans, among other elements. 3 5 6 Accordingly, most EHRs have tethered patient portals that securely provide patients access to their personal health information and messaging. Hence, health systems have leveraged patient portals in the hopes of achieving better health outcomes and to be eligible for CEHRT incentives.
The advent of COVID-19 led to an increased need and demand for telehealth services and in online scheduling for vaccinations and other health maintenance visits. At large health systems, such virtual services were often made available through patient portals. 7 Portals can serve as important tools for informing clinical decisions 8 and encouraging preventive health behaviors. 9 The portal may improve patient care by allowing patients to provide health care providers with up-to-date information. Patients who reported higher portal visit frequency were more likely to identify and request error corrections to their medical records. 10 Patient barriers to portal adoption, including differences in who is offered portal activation codes, 11 limit the realization of these benefits. The availability of clinical notes may also influence portal usage. 12 Prior studies suggest that patient portal adoption may vary by sociodemographic characteristics. 9 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 For example, users of myPennMedicine, at the University of Pennsylvania Health System, were found to be younger, White, with higher incomes, and were commercially insured. 9 Among adult patients of University of Florida Health, male sex, Hispanic or African American race, not being married, and having public health insurance or self-pay were negatively associated with adopting MyUFHealth. 13 Prior studies have found that patient portal adoption is highest among patients who are White, 17 20 22 25 27 29 commercially insured, 17 and have higher incomes. 17 24 27 28 Office visits have also been reported to be either higher 14 or lower 25 among patients who are portal adopters, whereas comorbidity scores are often higher. 20 24 29 Additionally, Arcury et al conducted a survey to evaluate factors associated with patient portal utilization in low-income older adults from two rural and one urban clinic in North Carolina. 24 The study found that patients in the urban clinic were more likely to report using the patient portal than patients of the rural clinics. Further, participants who reported to have ever used the portal reported higher incomes and educational attainment. 24 Despite the growing literature, there is still limited data on the clinical and socioeconomic factors associated with patients becoming portal users, particularly being more active users and the specific portal functions they use. Also, prior studies evaluating patient portal use have primarily used data available in the EHR, that is often missing socioeconomic factors such as income and education, and few studies have linked area-level socioeconomic estimates to EHR data.
We conducted a cross-sectional analysis of EHR data merged with area socioeconomic characteristics from the U.S. census bureau to: (1) examine the socioeconomic factors, disease burden, and health care utilization rates among patients of a large academic primary care network who are users and non-users of the patient portal; and (2) describe the bidirectional functions that adopters most frequently use.
Methods
Setting
This study conformed to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist 30 and was approved by the Wake Forest School of Medicine Institutional Review Board.
The study was conducted at Atrium Health Wake Forest Baptist (AHWFB), an integrated academic health system serving the communities of central and western North Carolina. The system is comprised of a tertiary care hospital in Winston-Salem, North Carolina, four community hospitals, and over 300 ambulatory practice sites, including 84 general internal medicine or family medicine primary care clinics. AHWFB operates a tethered patient portal called myWakeHealth, which is an adaption of EpicCare's MyChart.
Study Design
The study used a cross-sectional design to compare differences in sociodemographic and clinical characteristics between myWakeHealth users and non-users.
Study Sample
We included all patients 18 years of age or older who had two or more in-person primary care visits to a general internal medicine or family medicine AHWFB clinic between January 1, 2018, and December 31, 2019. Only patients whose residential address contained a North Carolina five-digit zip code were included ( N = 188,113). We excluded patient records with missing values for sex ( N = 9), whose primary health insurance was listed as “other government programs,” “worker's comp,” “liability,” or were missing data on insurance ( N = 1,024), or who had American Community Survey estimates (see below) missing at both the census block and zip code level ( N = 8,360). Our final study population was 178,720 (95.0%).
Data Sources and Data Elements
Patient-level data were extracted from Epic's Clarity database. The following demographic variables were included: patient age, sex, race/ethnicity (non-Hispanic White, non-Hispanic Black, Hispanic, or other), marital status (married, separated/widowed/divorced, or single), and primary health insurance (commercial, Medicare, Medicaid, self-pay, or Tricare). Race and ethnicity categories included: non-Hispanic White, non-Hispanic Black, Hispanic, and other (where ethnicity was “Patient Refused,” “Unknown,” or missing, and race was “American Indian or Alaskan,” “Asian,” “Native Hawaiian or Other,” “Patient Refused,” “Other,” “Unknown,” or missing).
Chronic conditions were quantified by the Charlson Comorbidity Index (CCI), which was generated by screening all diagnosis codes documented in encounter diagnoses, billing account diagnoses, admission diagnoses, medical histories and problem lists before or on the date of patients' most recent encounters with the health system. Diagnosis codes for 17 potential comorbidities were assigned weights and the CCI scores for respective patients were calculated from the sum of the weights, with scores ranging from (lowest to highest) 0 to 29. 31 Based on prior studies, we categorized CCI as 0, 1 to 2, 3 to 4, or ≥5. 32
For describing health care utilization between myWakeHealth users and non-users, we calculated the number of primary care visits, emergency room (ER) visits, inpatient visits, inpatient days, and appointment no-shows for each group during the study period. Since not all patients had the same follow-up time, we standardized the measurements for health care utilization by calculating counts per patient per year (PPPY). The PPPY for each health care utilization measure was calculated by dividing the total count by the patient-years.
Individual patients' residential addresses were geocoded to the census block level and merged with data from the 2015 to 2019 American Community Survey (ACS) 5-year estimates. The ACS is a national survey administered by the U.S. Census Bureau that collects population estimates of geographical area demographic and socioeconomic factors. 33 We imputed zip-code level data (which were derived from the ACS zip code tabulations) for patients whose address could not be geocoded to the census block level due to either missing or invalid addresses. We included three estimates from the ACS data: (1) median household income; (2) educational attainment defined as the percent of the population aged 25 years of age and older with a bachelor's degree or higher; and (3) the unemployment rate defined as the percent of the population aged 16 years of age and older who are in the civilian labor force and are unemployed.
Active Portal Users
Our primary outcome was whether a patient was a portal user or a non-user. The portal functions available to patients in myWakeHealth include “active” or “bidirectional” (e.g., sending messages) and “passive” or “unidirectional” (e.g., viewing test results) portal actions. As noted in prior studies, 34 there is significant variability in how prior studies define a “portal user.” We used a more strict definition of a “portal user” as our goal was to understand the characteristics of patients who are more highly engaged in utilizing the patient portal because health systems are increasingly utilizing the portal to communicate with patients and provide telehealth services. Similar to Chan et al, 35 we defined a myWakeHealth portal “user” as a patient who used any one of four “bidirectional” portal functions (yes or no) at least once during the study period: (1) sent an appointment request message; (2) sent a medication renewal request message; (3) sent a medical advice message; and (4) directly scheduled an appointment. Each patient action has a timestamp for the date and time the message was created, or when the action was performed. For this study, only functions that occurred during the study period were included and functions performed by a patient's proxy were counted as if performed by the patient themselves.
Types of Portal Use
Among users, we assessed usage patterns of bidirectional and unidirectional portal functions. We evaluated for the percentage of portal users who sent messages and the types of messages sent: appointment requests, medication renewal requests, and medical advice requests. Medical advice request messages have subject categories for free text such as requests for non-urgent medical advice and prescription questions. For portal actions, we considered the completion of the direct scheduling action by a patient to be a bidirectional function. Direct scheduling allows patients to schedule an appointment in myWakeHealth based on available providers, locations, and times, and is an alternative to sending an appointment request message.
We also described the percentage of users who performed unidirectional portal actions, including viewing their visit summaries and test results; accessed their messages; and viewing their billing summary (billing summaries became available in myWakeHealth starting in September 2019).
Statistical Analysis
We reported the descriptive characteristics of patients overall and of users and non-users of myWakeHealth using frequencies and percentages for categorical variables, and frequencies and means for continuous variables. Bivariable comparisons of users and non-users were made using Chi-square tests for categorical variables and Wilcoxon rank-sum tests for continuous variables.
To assess for multicollinearity, we assessed bivariable correlations among predictors and assessed the variance inflation factors (VIF) for all predictors from a fitted regression model. We did not include any variables that had a high correlation coefficient (>0.8) or VIF (>4). 36
We used a generalized linear mixed model with logit link function (PROC GLIMMIX of SAS) to model patient portal users as the primary outcome variable, and controlled for clustering by the primary care clinic the patient was primarily seen at using random intercepts. Adjusted odds ratios (aOR) included adjustments for the random effect and fixed effects. We included predictors based on previous studies 9 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 and from significant bivariable statistical tests. To simplify the evaluation of results we categorized ACS estimates into quartiles. As such we included the following variables as fixed effects into our mixed-effects logistic regression model: age, sex, race/ethnicity, marital status, primary health insurance, CCI, the three ACS estimates, and patients' numbers of primary care visits during the study period. The intraclass correlation coefficient (ICC) was calculated to indicate how much of the total variation in portal use was accounted for by clinics. All statistical tests are two-sided, and we considered an α <0.05 to be statistically significant. All analyses were completed using SAS 9.4 (SAS Institute, Inc, Cary, North Carolina, United States).
Sensitivity Analyses
We conducted sensitivity analyses to evaluate the robustness of the regression model results. The sensitivity analyses included (1) modifying the inclusion criteria to only adult patients with three or more visits during the study period; (2) restricting to patients who had ACS estimates at the census block group level; (3) defining portal use as having used any one of four “unidirectional” portal functions (accessed messaging, or viewed the visit summary, test results, or billing summary) at least once during the study period; (4) modifying the inclusion criteria to only adult patients with one or more visits during the study period; and (5) defining portal use as having used three or more bidirectional portal functions at least once during the study period.
Results
Patient Characteristics
Of the 178,720 patients included in the study, 114,263 (63.93%) patients had ACS estimates at the census block level and 64,457 (36.07%) patients had ACS estimates at the zip code level. The majority of the population was female (58.5%), non-Hispanic White (77.1%), and married (58.4%) ( Table 1 ). Approximately half of patients had commercial health insurance (49.9%), and 50.9% had a CCI of 0. The median household income was $51,250 and the median for the percent of persons with a bachelor's degree or higher was 21.6%. A quarter of patients had at least seven primary care visits during the study period.
Table 1. Baseline characteristics of portal users and non-users from 2018 to 2019.
| Total ( N = 178,720) | Portal user ( N = 57,122) | Portal non-user ( N = 121,598) | p -Value | |
|---|---|---|---|---|
| Age, n (%) | <0.001 a | |||
| 18–44 | 48,043 (26.9%) | 18,167 (37.8%) | 29,876 (62.2%) | |
| 45–64 | 72,114 (40.4%) | 24,420 (33.9%) | 47,694 (66.1%) | |
| ≥65 | 58,563 (32.8%) | 14,535 (24.8%) | 44,028 (75.2%) | |
| Sex, n (%) | <0.001 a | |||
| Female | 104,604 (58.5%) | 36,565 (35.0%) | 68,039 (65.0%) | |
| Male | 74,116 (41.5%) | 20,557 (27.7%) | 53,559 (72.3%) | |
| Ethnicity and race, n (%) | <0.001 a | |||
| Non-Hispanic White | 137,768 (77.1%) | 47,189 (34.3%) | 90,579 (65.7%) | |
| Non-Hispanic Black | 26,704 (14.9%) | 6,383 (23.9%) | 20,321 (76.1%) | |
| Hispanic | 6,577 (3.7%) | 1,401 (21.3%) | 5,176 (78.7%) | |
| Other | 7,671 (4.3%) | 2,149 (28.0%) | 5,522 (72.0%) | |
| Marital status, n (%) | <0.001 a | |||
| Married | 104,424 (58.4%) | 37,452 (35.9%) | 66,972 (64.1%) | |
| Separated, widowed, divorced | 32,661 (18.3%) | 8,406 (25.7%) | 24,255 (74.3%) | |
| Single | 41,635 (23.3%) | 11,264 (27.1%) | 30,371 (72.9%) | |
| Primary health insurance, n (%) | <0.001 a | |||
| Commercial | 89,135 (49.9%) | 34,486 (38.7%) | 54,649 (61.3%) | |
| Medicare | 70,068 (39.2%) | 18,199 (26.0%) | 51,869 (74.0%) | |
| Public (Medicaid, Tricare) | 10,063 (5.6%) | 2,438 (24.2%) | 7,625 (75.8%) | |
| Self-pay | 9,454 (5.3%) | 1,999 (21.1%) | 7,455 (78.9%) | |
| Charlson Comorbidity Index, n (%) | <0.001 a | |||
| 0 | 91,031 (50.9%) | 28,002 (30.8%) | 63,029 (69.2%) | |
| 1–2 | 54,090 (30.3%) | 17,948 (33.2%) | 36,142 (66.8%) | |
| 3–4 | 16,077 (9.0%) | 5,062 (31.5%) | 11,015 (68.5%) | |
| ≥5 | 17,522 (9.8%) | 6,110 (34.9%) | 11,412 (65.1%) | |
| Median household income b , n (%) | <0.001 a | |||
| < $42,721 | 45,882 (25.7%) | 10,666 (23.2%) | 35,216 (76.8%) | |
| $42,721–$51,249 | 43,475 (24.3%) | 13,178 (30.3%) | 30,297 (69.7%) | |
| $51,250–$64,329 | 44,706 (25.0%) | 13,850 (31.0%) | 30,856 (69.0%) | |
| ≥$64,330 | 44,657 (25.0%) | 19,428 (43.5%) | 25,229 (56.5%) | |
| Percentage bachelor's degree or higher 2% , n (%) | <0.001 a | |||
| < 15.1 | 44,407 (24.8%) | 9,691 (21.8%) | 34,716 (78.2%) | |
| 15.1–21.6 | 45,069 (25.2%) | 12,036 (26.7%) | 33,033 (73.3%) | |
| 21.7–37.4 | 44,624 (25.0%) | 15,109 (33.9%) | 29,515 (66.1%) | |
| ≥37.5 | 44,620 (25.0%) | 20,286 (45.5%) | 24,334 (54.5%) | |
| Percentage unemployed 2% , n (%) | <0.001 a | |||
| < 2.4 | 44,201 (24.7%) | 14,853 (33.6%) | 29,348 (66.4%) | |
| 2.4–4.6 | 45,101 (25.2%) | 15,828 (35.1%) | 29,273 (64.9%) | |
| 4.7–6.7 | 45,767 (25.6%) | 14,194 (31.0%) | 31,573 (69.0%) | |
| ≥6.8 | 43,651 (24.4%) | 12,247 (28.1%) | 31,404 (71.9%) | |
| Primary care visits c , n (%) | <0.001 a | |||
| 2 | 32,402 (18.1%) | 8,677 (26.8%) | 23,725 (73.2%) | |
| 3–4 | 52,239 (29.2%) | 16,566 (31.7%) | 35,673 (68.3%) | |
| 5–6 | 49,443 (27.7%) | 16,877 (34.1%) | 32,566 (65.9%) | |
| ≥7 | 44,636 (25.0%) | 15,002 (33.6%) | 29,634 (66.4%) |
Note: A patient was defined as a portal user if they used any one of four bidirectional portal functions during the study period: (1) sent an appointment request message; (2) sent a medication renewal request message; (3) sent a medical advice message; and (4) directly scheduled an appointment.
Chi-square p -value.
Sourced from American Community Survey census block group and 5-digit zip code estimates.
Primary care visits at Atrium Health Wake Forest Baptist from January 1, 2018 to December 31, 2019.
Of the study population, 32% ( N = 57,122) of patients were users of the myWakeHealth patient portal. In bivariable analysis ( Table 1 ), there were significant differences between portal users and non-users across age, sex, race and ethnicity, marital status, primary health insurance, CCI, and primary care visits. There were also significant differences between portal users and non-users across area median household income, educational attainment, and unemployment.
Health Care Utilization
Compared to portal users, non-portal users had fewer primary care visits and more ER visits, although magnitudes of the differences were small. Non-users also had more no-show visits (0.27 vs. 0.17 PPPY, p <0.001) ( Table 2 ).
Table 2. Health care utilization of portal users and non-users between January 1, 2018, and December 31, 2019.
| Total ( N = 178,720) | Portal user ( N = 57,122) | Portal non-user ( N = 121,598) | p -Value | |
|---|---|---|---|---|
| Primary care visits | ||||
| Per patient per year | 3.01 | 3.08 | 2.98 | <0.001 a |
| Emergency room visits | ||||
| Per patient per year | 0.18 | 0.17 | 0.19 | <0.001 a |
| Inpatient visits | ||||
| Per patient per year | 0.09 | 0.10 | 0.09 | <0.001 a |
| Inpatient days | ||||
| Per patient per year | 0.45 | 0.44 | 0.45 | <0.001 a |
| Patient no shows | ||||
| Per patient per year | 0.24 | 0.17 | 0.27 | <0.001 a |
Note: A patient was defined as a portal user if they used any one of four bidirectional portal functions during the study period: (1) sent an appointment request message; (2) sent a medication renewal request message; (3) sent a medical advice message; and (4) directly scheduled an appointment.
Wilcoxon rank sum p -value for per patient per year.
In the mixed-effects logistic regression model, patients most likely to be portal users were of age 18 to 44, female, non-Hispanic White, married, commercially insured, have higher comorbidity scores, have higher incomes and educational attainment, and have more primary care visits ( Fig. 1 ). Clinic level clustering of patients accounted for 22.6% (ICC = 0.226) of the total variation in portal use. The odds of using the portal were 2.46 times higher in patients 18 to 44 years of age versus patients 65 years of age (aOR 2.46, 95% CI 2.35–2.58, p <0.001). Females had 46% higher odds of being portal users compared to males (aOR 1.46, 95% CI 1.43–1.50, p <0.001), while non-Hispanic Black and Hispanic patients had 56% and 54% lower odds of using the portal compared to non-Hispanic White patients, respectively (aOR 0.44, 95% CI 0.43–0.46; and aOR 0.46, 95% CI 0.43–0.49, all p <0.001). Patients who were publicly insured or uninsured had 40% and 43% lower odds of being portal users than patients with commercial health insurance, respectively (aOR 0.60, 95% CI 0.57–0.64; and aOR 0.57, 95% CI 0.54–0.61, all p <0.001). Patients with CCI scores of 5 or more had 33% higher odds of using the portal than patients with scores of 0 (aOR 1.33, 95% CI 1.28–1.39, p <0.001).
Fig. 1.
Association between patient characteristics and the odds of being a portal user. A patient was defined as a portal user if they used any one of four bidirectional portal functions during the study period: (1) sent an appointment request message; (2) sent a medication renewal request message; (3) sent a medical advice message; and (4) directly scheduled an appointment.
Regarding socioeconomic indicators, patients living in areas at the highest income quartile had 13% higher odds of being a user compared to areas at the lowest quartile (aOR 1.13, 95% CI 1.08–1.18, p <0.001). Patients living in areas at the highest quartile for education had 51% higher odds of being a portal user compared to the lowest quartile (aOR 1.51, 95% CI 1.45–1.58, p <0.001).
In sensitivity analyses, the effects observed in odds ratios were similar to the original model ( Supplementary Figs. S1 – S5 , available in the online version).
Portal Usage
Among the 57,122 bidirectional portal users, 57,067 (99.9%) sent a message through the portal ( Table 3 ). The most common type of message sent was a “request for medical advice” ( N = 48,449 [88.4%]). During the study period, there were 5,365,544 portal logins (mean logins per user per month = 3.92). Among portal users, 56,707 (99.3%) accessed a message in the portal and 55,231 (96.7%) viewed a test result ( Table 4 ).
Table 3. Percentage of portal users who sent a message through the portal between January 1, 2018, and December 31, 2019.
| Number of portal users | Proportion | |
|---|---|---|
| Sent an initial message | 57,076 | 99.9% |
| Requested appointment | 27,422 | 48.0% |
| Requested medication refill | 22,817 | 39.9% |
| Requested medical advice | 48,449 | 84.8% |
| Requested medical advice: non-urgent medical question | 35,028 | 61.3% |
| Requested medical advice: prescription question | 29,379 | 51.4% |
| Requested medical advice: test results question | 19,446 | 34.0% |
| requested medical advice: request update to records | 4,841 | 8.5% |
| Requested medical advice: referral request | 10,794 | 18.9% |
| Requested medical advice: other | 1 | 0.002% |
Note: Results are based on the number of patients defined as a portal user ( N = 57,122). We defined a portal user as any patient who used at least any one of four bidirectional portal functions within the study period.
Table 4. Percentage of portal users who utilized a portal action between January 1, 2018, and December 31, 2019.
| Number of portal users | Proportion | |
|---|---|---|
| Logged in | 56,952 | 99.7% |
| Directly scheduled appointment | 14,512 | 25.4% |
| Viewed visit summary | 3,775 | 6.6% |
| Viewed test results | 55,231 | 96.7% |
| Accessed messaging | 56,707 | 99.3% |
| Viewed billing summary a | 23,135 | 40.5% |
Became available starting in September 2019. Results are based on the number of patients defined as a portal user ( N = 57,122). We defined a portal user as any patient who used at least any one of four bidirectional portal functions within the study period.
Discussion
In this study of over 178,000 patients across a large integrated academic medical center, we found that patient demographics, comorbidities, health care utilization, and area socioeconomic factors were associated with patients being a bidirectional patient portal user. Patients living in areas with lower median household incomes and lower educational attainment had lower odds of using the portal. Also, users primarily used the portal to send messages to providers requesting medical advice.
Consistent with previous studies, 9 17 18 19 21 22 26 27 28 37 38 we found that portal users were more likely to be younger, female, non-Hispanic White, be privately insured, have higher CCI scores and more likely to live in areas with a higher median income and higher educational attainment. A study of the 2017 and 2018 Health Information National Trends Survey (HINTS) showed higher odds of portal use in respondents with a bachelor's degree and incomes in the middle tertile. 28 Likewise, a different study of the 2017 HINTS showed that participants with higher incomes and educational attainment, and who were employed were more likely to access their online medical records. 38 Similar to our study, Goel et al explored patient use of the portal by demographics and used census data for measuring educational attainment and income. 18 They found disparities across age and gender in patients requesting advice from providers through the patient portal, both when not adjusting and adjusting for education and income. 18
Although unclear, there are several potential reasons we may have found that median household income and education level were associated with portal use, e.g., differences in who and how the portal is introduced by providers to patients and how comfortable patients are with technology. Another potential reason could be patients' broadband internet access. 39 We observed that patients who were Black or Hispanic were less likely to use the portal than Non-Hispanic Whites, which could be due to the digital divide that exists among historically marginalized groups. Perzynski et al used census block group level estimates for median household income and years of education from the American Community Survey in their study of the association between patients' access to broadband internet and using the patient portal. 17 They used home broadband internet access data from 2013 Federal Communications Commission reports, and found a linear association between the number of internet connections in zip codes and the number of patient portal logins. 17 Further, they found that income and education slightly modified the association between home broadband internet and the odds of sending a message. 17 Although, we did not evaluate internet access, this may be an important area for research in the future. The FCC launched the Emergency Broadband Benefit (EBB) Program in May 2021 for offering affordable internet to low-income households during the COVID-19 pandemic. The EBB is transitioning to a permanent Affordable Connectivity Program because of the 2021 Infrastructure Investment and Jobs Act. 40 As broadband access improves, the internet may become less of a barrier to portal adoption among patients who have been marginalized.
Interestingly, patients in the upper third and fourth quartiles for primary care visits had 98% and 141% greater odds of using the portal than those patients in the lower quartile. Patients with higher acute health care utilization, however, were less likely to be bidirectional portal users. This could be due to differences in patients who seek care from a primary care providers versus those who primarily received acute health care utilization. Patients who feel more engaged with their health care providers may be more likely to have a primary care home and find benefits to utilizing the portal to communicate with their primary care clinic. The differences in health care utilization and portal use could also be due to how and where health care providers educate patients about the patient portal and its potential benefits. As one study showed, introducing and educating patients in the hospital about accessing the portal improved the number of patients registering to be portal users. 41 Implementing programs and interventions to educate patients about the portal or developing workflows that allow patients to sign up for the portal in the ER or hospital could be an important methods for improving portal adoption. However, although the differences in ER visits, inpatient visits, and inpatient days between portal users and non-users were statistically significant, they were small and may not be clinically significant.
Interestingly, there was a large difference in no-shows for visits between portal users and non-users, and these results were consistent with previous studies of patients who had “active” patient portal accounts, 42 or who used the patient portal's messaging feature. 13 In contrast, another study found no difference in no-shows between users and non-users, but it used the top 75th percentile for portal logins to define a portal user. 14 This finding provides an additional important feature to consider in the development of prediction models for missed appointments, as it represents a previously unassessed predictor in such models. 43 44 45 Because of the cross-sectional nature of our data, we are unable to draw temporal relationships between being a portal user and no-shows. An area of future research could be evaluating differences in no shows between appointments made through the portal compared to other means. This does highlight the importance in how a portal user is defined. As noted in prior studies, 34 there is significant variability in the patient portal literature in the criteria used in defining a “user.” During the COVID-19 pandemic, health systems frequently used the portal to provide communication and access to telehealth services. If health systems plan to continue to use the portal to provide services after the pandemic has ended, understanding which patients are meaningfully engaged with the portal and how that impacts the services patients receive will be increasingly important or disparities in health education and health care access may widen.
Limitations
Although our study has many strengths, including the linkage of area-level socioeconomic estimates with EHR data, a large sample size across a large integrated health system, and accounting for clinic variation, there are several limitations that should be acknowledged. First, the cross-sectional design of this study prevents our ability to infer temporal relationships. Second, using data from the ACS subjects our analyses to the “ecological fallacy” whereby local area estimates may not reflect individual patient's income, educational attainment, or employment status. Third, since our analysis is limited to a single health system our results may not be generalizable to other medical centers. Fourth, we were unable to identify and account for patients experiencing homelessness. As health systems are increasingly screening patients for unmet social needs (e.g., housing instability), this could be an important area for future research and clinical care. Fifth, because of the low frequency of messages sent through a proxy account and the challenges with distinguishing messages sent through patient or proxy, 46 we did not distinguish between portal usage performed by a patient account versus a proxy account.
Conclusion
We found that patients' demographics, comorbidities, health care utilization, and area socioeconomic factors were associated with patient portal use. Patients residing in areas with the highest income and educational attainment had higher odds of being a portal user compared to patients in the lowest areas. These findings suggest that efforts to improve portal adoption should be targeted at vulnerable patients. Further investigation is needed to see if the characteristics of patient portal users changed after the onset of the COVID-19 pandemic and the availability of affordable broadband internet to low-income households, and whether portal adoption leads to improved outcomes.
Clinical Relevance Statement
This study adds to the body of literature assessing the patient characteristics associated with using bidirectional portal functions. The results underscore the influence that socioeconomic factors have on the odds of using the patient portal. Other institutions can use this study as a template to examine the gaps in portal use among their patient populations.
Multiple Choice Questions
-
What was the most common message type sent by portal users?
Appointment request.
Referral request.
Request for a medication refill.
Non-urgent medical advice.
Correct Answer: The correct answer is option d. A request for non-urgent medical advice was the most frequently sent message by portal users.
-
Portal users are more likely to have which characteristic:
Reside in areas with high unemployment.
Have commercial health insurance.
Reside in areas with low educational attainment.
Have public health insurance.
Correct Answer: The correct answer is option b. Patients who are commercially insured are more likely to use the patient portal.
Acknowledgments
The authors gratefully acknowledge the use of the services and facilities of the Wake Forest Clinical and Translational Science Institute and the Division of Public Health Sciences.
Funding Statement
Funding The study was funded by U.S. Department of Health and Human Services, National Institutes of Health (TL1TR003136), and National Heart, Lung, and Blood Institute (K23HL146902). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Conflict of Interest None declared.
Protection of Human and Animal Subjects
This study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles of Medical Research involving Human Subjects. The Wake Forest School of Medicine Institutional Review Board reviewed and approved the study and waived informed consent because it did not involve patient contact.
Supplementary Material
References
- 1.Lai Y, Moseley E, Salgueiro F. Cham (CH): Springer; 2016. Integrating non-clinical data with EHRs; pp. 51–60. [PubMed] [Google Scholar]
- 2.Lyles C R, Nelson E C, Frampton S, Dykes P C, Cemballi A G, Sarkar U. Using electronic health record portals to improve patient engagement: research priorities and best practices. Ann Intern Med. 2020;172 11:S123–S129. doi: 10.7326/M19-0876. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Department of Health and Human Services . Centers for medicare and medicaid services. Electronic health record incentive program—stage 2. (42 CFR Parts 412, 413, and 495) Fed Regist. 2012;77(171):53968–54162. [PubMed] [Google Scholar]
- 4.Ahern D K, Woods S S, Lightowler M C, Finley S W, Houston T K.Promise of and potential for patient-facing technologies to enable meaningful use Am J Prev Med 201140(5, suppl 2):S162–S172. [DOI] [PubMed] [Google Scholar]
- 5.Department of health and human services 21st century cures act: interoperability, information blocking, and the ONC Health IT Certification Program. Fed Regist. 2020;85(85):25646. [Google Scholar]
- 6.Office of the National Coordinator for Health Information Technology (ONC), Department of Health and Human Services (HHS) . 2015 Edition Health Information Technology (Health IT) Certification Criteria, 2015 Edition Base Electronic Health Record (EHR) Definition, and ONC Health IT Certification Program Modifications. Final rule. Fed Regist. 2015;80(200):62601–62759. [PubMed] [Google Scholar]
- 7.Turer R W, DesRoches C M, Salmi L, Helmer T, Rosenbloom S T. Patient perceptions of receiving COVID-19 test results via an online patient portal: an open results survey. Appl Clin Inform. 2021;12(04):954–959. doi: 10.1055/s-0041-1736221. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Ginting K, Stolfi A, Wright J, Omoloja A. Patient portal, patient-generated images, and medical decision-making in a pediatric ambulatory setting. Appl Clin Inform. 2020;11(05):764–768. doi: 10.1055/s-0040-1718754. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Huang J, Chen Y, Landis J R, Mahoney K B. Difference between users and nonusers of a patient portal in health behaviors and outcomes: retrospective cohort study. J Med Internet Res. 2019;21(10):e13146. doi: 10.2196/13146. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Nguyen O T, Hong Y R, Alishahi Tabriz A, Hanna K, Turner K. Prevalence and factors associated with patient-requested corrections to the medical record through use of a patient portal: findings from a national survey. Appl Clin Inform. 2022;13(01):242–251. doi: 10.1055/s-0042-1743236. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Bush R A, Vemulakonda V M, Richardson A C, Deakyne Davies S J, Chiang G J. Providing access: differences in pediatric portal activation begin at patient check-in. Appl Clin Inform. 2019;10(04):670–678. doi: 10.1055/s-0039-1695792. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Sulieman L, Steitz B, Rosenbloom S T. Analysis of employee patient portal use and electronic health record access at an academic medical center. Appl Clin Inform. 2020;11(03):433–441. doi: 10.1055/s-0040-1713412. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Zhong X, Park J, Liang M. Characteristics of patients using different patient portal functions and the impact on primary care service utilization and appointment adherence: retrospective observational study. J Med Internet Res. 2020;22(02):e14410. doi: 10.2196/14410. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Tsai R, Bell E J, III, Woo H, Baldwin K, Pfeffer M A. How patients use a patient portal: an institutional case study of demographics and usage patterns. Appl Clin Inform. 2019;10(01):96–102. doi: 10.1055/s-0038-1677528. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Abd-Alrazaq A A, Bewick B M, Farragher T, Gardner P. Factors that affect the use of electronic personal health records among patients: a systematic review. Int J Med Inform. 2019;126:164–175. doi: 10.1016/j.ijmedinf.2019.03.014. [DOI] [PubMed] [Google Scholar]
- 16.Oest S ER, Hightower M, Krasowski M D. Activation and utilization of an electronic health record patient portal at an academic medical center-impact of patient demographics and geographic location. Acad Pathol. 2018;5:2.374289518797573E15. doi: 10.1177/2374289518797573. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Perzynski A T, Roach M J, Shick S. Patient portals and broadband internet inequality. J Am Med Inform Assoc. 2017;24(05):927–932. doi: 10.1093/jamia/ocx020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Goel M S, Brown T L, Williams A, Hasnain-Wynia R, Thompson J A, Baker D W. Disparities in enrollment and use of an electronic patient portal. J Gen Intern Med. 2011;26(10):1112–1116. doi: 10.1007/s11606-011-1728-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Trivedi N, Patel V, Johnson C, Chou W S. Barriers to accessing online medical records in the United States. Am J Manag Care. 2021;27(01):33–40. doi: 10.37765/ajmc.2021.88575. [DOI] [PubMed] [Google Scholar]
- 20.Elston Lafata J, Miller C A, Shires D A, Dyer K, Ratliff S M, Schreiber M. Patients' adoption of and feature access within electronic patient portals. Am J Manag Care. 2018;24(11):e352–e357. [PMC free article] [PubMed] [Google Scholar]
- 21.Furniss S, Rinehart-Thompson L, MacEwan S.Characteristics of patients using a patient portal via mobile technology Perspect Health Inf Manag 2020(Winter):1–10. [Google Scholar]
- 22.Mook P J, Trickey A W, Krakowski K E. Exploration of portal activation by patients in a healthcare system. Comput Inform Nurs. 2018;36(01):18–26. doi: 10.1097/CIN.0000000000000392. [DOI] [PubMed] [Google Scholar]
- 23.Ancker J S, Barrón Y, Rockoff M L. Use of an electronic patient portal among disadvantaged populations. J Gen Intern Med. 2011;26(10):1117–1123. doi: 10.1007/s11606-011-1749-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Arcury T A, Quandt S A, Sandberg J C. Patient portal utilization among ethnically diverse low income older adults: observational study. JMIR Med Inform. 2017;5(04):e47. doi: 10.2196/medinform.8026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Weingart S N, Rind D, Tofias Z, Sands D Z. Who uses the patient internet portal? The PatientSite experience. J Am Med Inform Assoc. 2006;13(01):91–95. doi: 10.1197/jamia.M1833. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Wallace L S, Angier H, Huguet N. Patterns of electronic portal use among vulnerable patients in a nationwide practice-based research network: from the OCHIN practice-based research network (PBRN) J Am Board Fam Med. 2016;29(05):592–603. doi: 10.3122/jabfm.2016.05.160046. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Turner K, Clary A, Hong Y R, Alishahi Tabriz A, Shea C M. Patient portal barriers and group differences: cross-sectional national survey study. J Med Internet Res. 2020;22(09):e18870. doi: 10.2196/18870. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Swoboda C M, DePuccio M J, Fareed N, McAlearney A S, Walker D M. Patient portals: useful for whom and for what? a cross-sectional analysis of national survey data. Appl Clin Inform. 2021;12(03):573–581. doi: 10.1055/s-0041-1731339. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Griffin A, Skinner A, Thornhill J, Weinberger M. Patient portals: who uses them? What features do they use? And do they reduce hospital readmissions? Appl Clin Inform. 2016;7(02):489–501. doi: 10.4338/ACI-2016-01-RA-0003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.STROBE Initiative . von Elm E, Altman D G, Egger M, Pocock S J, Gøtzsche P C, Vandenbroucke J P. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61(04):344–349. doi: 10.1016/j.jclinepi.2007.11.008. [DOI] [PubMed] [Google Scholar]
- 31.Quan H, Li B, Couris C M. Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol. 2011;173(06):676–682. doi: 10.1093/aje/kwq433. [DOI] [PubMed] [Google Scholar]
- 32.Huang Y Q, Gou R, Diao Y S. Charlson comorbidity index helps predict the risk of mortality for patients with type 2 diabetic nephropathy. J Zhejiang Univ Sci B. 2014;15(01):58–66. doi: 10.1631/jzus.B1300109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.American Community Survey 2015–2019 5-Year Data ReleaseDecember 10, 2020. Updated October 8, 2021. Accessed November 1, 2021 at:https://www.census.gov/newsroom/press-kits/2020/acs-5-year.html
- 34.Beal L L, Kolman J M, Jones S L, Khleif A, Menser T. Quantifying patient portal use: systematic review of utilization metrics. J Med Internet Res. 2021;23(02):e23493. doi: 10.2196/23493. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Chan B, Lyles C, Kaplan C, Lam R, Karliner L. A comparison of electronic patient-portal use among patients with resident and attending primary care providers. J Gen Intern Med. 2018;33(12):2085–2091. doi: 10.1007/s11606-018-4637-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Glantz S, Slinker B, Neilands T. 3rd ed. New York: McGraw-Hill Education/Medical; 2016. Primer of Applied Rregression and Analysis of Variance. [Google Scholar]
- 37.Graetz I, Gordon N, Fung V, Hamity C, Reed M E. The digital divide and patient portals: internet access explained differences in patient portal use for secure messaging by age, race, and income. Med Care. 2016;54(08):772–779. doi: 10.1097/MLR.0000000000000560. [DOI] [PubMed] [Google Scholar]
- 38.Park J, Liang M, Alpert J M, Brown R F, Zhong X. The causal relationship between portal usage and self-efficacious health information-seeking behaviors: secondary analysis of the health information national trends survey data. J Med Internet Res. 2021;23(01):e17782. doi: 10.2196/17782. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Greenberg-Worisek A J, Kurani S, Finney Rutten L J, Blake K D, Moser R P, Hesse B W. Tracking healthy people 2020 internet, broadband, and mobile device access goals: an update using data from the health information national trends survey. J Med Internet Res. 2019;21(06):e13300. doi: 10.2196/13300. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Benefit E B.Federal Communications CommissionUpdated November 15, 2021. Accessed November 30, 2021 at:https://www.fcc.gov/broadbandbenefit
- 41.Stein J N, Klein J W, Payne T H. Communicating with vulnerable patient populations: a randomized intervention to teach inpatients to use the electronic patient portal. Appl Clin Inform. 2018;9(04):875–883. doi: 10.1055/s-0038-1676333. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Zhong X, Liang M, Sanchez R. On the effect of electronic patient portal on primary care utilization and appointment adherence. BMC Med Inform Decis Mak. 2018;18(01):84. doi: 10.1186/s12911-018-0669-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Alaeddini A, Yang K, Reddy C, Yu S. A probabilistic model for predicting the probability of no-show in hospital appointments. Health Care Manage Sci. 2011;14(02):146–157. doi: 10.1007/s10729-011-9148-9. [DOI] [PubMed] [Google Scholar]
- 44.Harvey H B, Liu C, Ai J. Predicting no-shows in radiology using regression modeling of data available in the electronic medical record. J Am Coll Radiol. 2017;14(10):1303–1309. doi: 10.1016/j.jacr.2017.05.007. [DOI] [PubMed] [Google Scholar]
- 45.Daghistani T, AlGhamdi H, Alshammari R, AlHazme R H. Predictors of outpatients' no-show: big data analytics using Apache spark. Journal of Big Data. 2020;7(01):108. [Google Scholar]
- 46.Pecina J, Duvall M J, North F. Frequency of and factors associated with care partner proxy interaction with health care teams using patient portal accounts. Telemed J E Health. 2020;26(11):1368–1372. doi: 10.1089/tmj.2019.0208. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.