Interventions to increase patient portal use in vulnerable populations: a systematic review

Abstract

Background

More than 100 studies document disparities in patient portal use among vulnerable populations. Developing and testing strategies to reduce disparities in use is essential to ensure portals benefit all populations.

Objective

To systematically review the impact of interventions designed to: (1) increase portal use or predictors of use in vulnerable patient populations, or (2) reduce disparities in use.

Materials and Methods

A librarian searched Ovid MEDLINE, EMBASE, CINAHL, and Cochrane Reviews for studies published before September 1, 2018. Two reviewers independently selected English-language research articles that evaluated any interventions designed to impact an eligible outcome. One reviewer extracted data and categorized interventions, then another assessed accuracy. Two reviewers independently assessed risk of bias.

Results

Out of 18 included studies, 15 (83%) assessed an intervention's impact on portal use, 7 (39%) on predictors of use, and 1 (6%) on disparities in use. Most interventions studied focused on the individual (13 out of 26, 50%), as opposed to facilitating conditions, such as the tool, task, environment, or organization (SEIPS model). Twelve studies (67%) reported a statistically significant increase in portal use or predictors of use, or reduced disparities. Five studies (28%) had high or unclear risk of bias.

Conclusion

Individually focused interventions have the most evidence for increasing portal use in vulnerable populations. Interventions affecting other system elements (tool, task, environment, organization) have not been sufficiently studied to draw conclusions. Given the well-established evidence for disparities in use and the limited research on effective interventions, research should move beyond identifying disparities to systematically addressing them at multiple levels.

INTRODUCTION

Last year, millions of Americans accessed their own health records online, more than ever before.^1–4 Secure websites called patient portals offer convenient, 24-hour access to records, as well as appointment scheduling, medication monitoring, and other health management features.⁵ Portals provide patients with unprecedented transparency into health information, which evidence suggests can prevent medical errors,^6–11 increase shared decision-making,^12–17 and improve health outcomes.^18,19 As such, transparency has been hailed as the next “blockbuster drug” and “healthcare revolution” by prominent media outlets.^20–22

Patient portals have only recently gained popularity. The percentage of healthcare organizations offering portals rose from 43% in 2013 to 92% in 2015.^4,23,24 As availability has increased, more patients have used portals.^25–28 In the United States (US), self-reported use rose from 17% in 2014 to 28% in 2017.^29,30 Multiple factors have contributed to the increase in portal availability, including the perceived impact on outcomes,³¹ consumers' desire for transparency,³² and the federal Meaningful Use program, which requires that organizations allow patients to view, download, and transmit their health records.^33,34

Some researchers initially hoped that portals could reduce health inequities,^35,36 a highly significant and refractory problem in the US.³⁷ Health inequities lead to poor health management and outcomes, which contribute to rising healthcare costs.³⁸ Vulnerable populations often demonstrate lower health literacy and experience significant barriers to care, such as inflexible job hours, cost, and insurance status.³⁹ Portal features such as messaging, online education, and automatic medication refills might increase convenience, improve health literacy, and overcome at least some barriers to care, thereby reducing health inequities.

Unfortunately, more than 100 studies now show substantial health-equity–relevant disparities in portal use (additional citations available upon request).^28,40–56 Vulnerable populations use portals less often, including elderly persons,^{44,46–48,56} racial minorities,^43,46–50 as well as persons with low socioeconomic status,^28,43,54 low health literacy,^{44,49,51–53} chronic illness,^41,46,50,56 or disabilities.^44,49,55 Relatively low portal use in vulnerable populations may lead to intervention-generated inequity, a phenomena where well-intentioned solutions worsen existing health inequities rather than reduce them.^57–59 Developing, implementing, and evaluating strategies to reduce disparities in portal use is critical to ensure portals benefit all populations as originally intended.

In this systematic review, we explore how researchers have confronted differential use of patient portals. Our review focuses on two critical questions: (1) what interventions impact portal use or predictors of portal use in vulnerable populations? (2) what interventions impact disparities in portal use?

MATERIALS AND METHODS

We conducted and reported this systematic review in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).⁶⁰ A technical protocol that details our eligibility criteria, includes the complete search strategies, and contains additional results tables is available as Supplementary Material.

Eligibility criteria

We developed eligibility criteria with respect to publication characteristics (type, language, year, and status) and study characteristics (participants, interventions, comparisons, outcomes, study design [PICOS], and technology), as described in Supplementary Table 1. Publication Characteristics: We included English-language research articles published or in press. Participants: We required that the interventions occur in 1 or more vulnerable populations. To define vulnerable populations, we used the PROGRESS-Plus framework developed by Campbell and Cochrane Equity Methods Group.^61–63 The PROGRESS-Plus framework identifies characteristics that stratify health opportunities and outcomes, including Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, and Social capital. “Plus” considers additional characteristics associated with social disadvantage, including age, disability, and illness status. Additionally, we included characteristics known to disadvantage portal users: (1) chronic, critical, or psychiatric illness;^40,64–66 (2) low functional, health, or technology literacy;^40,66,67 (3) low numeracy or graph literacy;^40,68,69 (4) low patient engagement, activation, or participation.^40,66Interventions: We included any intervention designed to impact an eligible outcome. Comparisons: Studies had to include a comparison to evaluate the effect of the intervention. Comparisons could involve measurements before and after implementation, or the intervention could be compared with some concurrent control condition or group. Outcomes: Studies had to include at least 1 outcome measure that captured portal use (such as rate of portal registration or number of logins), a predictor of portal use (such as usability or intended use), or a health-equity–relevant disparity in portal use (such as the difference between enrollment rates among white and non-white patients). We included studies regardless of whether this outcome measure was the primary outcome or a secondary outcome. Study Design: We included any study design as long as an eligible comparison occurred. Technology: We excluded consumer health technologies other than patient portals, such as telehealth, mobile health (mHealth), or electronic visit (eVisit) platforms.

Data sources and searches

We searched Ovid MEDLINE, EMBASE, CINAHL, and Cochrane Reviews for English-language studies published before September 1, 2018. The Supplementary Materials include the full electronic database names, search dates, and search strategies. First, 3 authors (LVG, RMC, JSA) identified relevant Medical Subject Headings (MeSH) and free-text search terms based on the eligibility criteria, potentially relevant studies, and personal expertise. Then, an experienced librarian (DW) developed and conducted all searches. A second librarian reviewed the searches for completeness and accuracy. Additionally, we manually searched our personal reference libraries, reference lists of included studies, and pertinent reviews to identify potentially relevant citations our search might have missed. Finally, we searched tables of contents of pertinent scientific journals between May 1, 2018 and December 1, 2018 to identify recently published citations. When necessary, we directly communicated with study authors to ensure we had included all relevant citations and to obtain any manuscripts in press.

Study selection

We used Covidence systematic review software (Veritas Health Innovation, Melbourne, Australia; available at www.covidence.org) for citation screening, as recommended by Cochrane.⁷⁰ Initially, 2 researchers independently evaluated each citation for eligibility based on the title and abstract. For all potentially eligible studies identified in the initial screening, at least 2 researchers reviewed the full text to determine final eligibility. Conflicts were resolved by discussion with the study team.

Data extraction

The study team developed the data extraction form based on an initial review of included studies. Information extracted included the study objective, setting, population, design, eligibility criteria, intervention category, and findings. One team member extracted relevant data from each article, and a second team member reviewed all data extractions for completeness and accuracy.

Risk of bias assessment

To assess risk of bias (internal validity), we used predefined criteria from the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews to rate studies as low, medium, high, or unclear risk of bias.⁷¹ The criteria evaluate common sources of selection, performance, attrition, detection, and reporting bias. The guide specifies which criteria apply to different study designs, which was important because we included multiple study designs in this review. Two reviewers independently assessed risk of bias for each study, and differences were resolved by discussion with the study team.

Data analysis and synthesis

Descriptive analysis of study characteristics was conducted in Microsoft Excel. When relevant estimates could not be extracted directly from the article, we computed or estimated them based on published data (see footnotes to Tables 1, 2, and 4 for details). We assessed intensity of intervention as per the Cochrane Handbook for Systematic Reviews of Interventions.⁷² Given the heterogeneity of included interventions, we could not apply 1 single measure of intensity to all interventions. In general, we defined low intensity as 1 mode of delivery or episode of patient contact, medium as 2 or 3, and high as more than 3.

Table 1.

Studies included in the systematic review

Study	Portal-Relevant Objective	Study Design	Intervention (Category)a	Main Finding(s)
Ali et al. 201877	Identify usability challenges in a portal, and evaluate whether recommended solutions improved its usability	Pre-post (quasi-experimental)	Improve usability (B7)	System usability score (81.9 after vs 69.2 before, p=.049) and task completion (87% after vs 55% before) improved after recommended solutions were appliedb
Ancker et al. 201778	Estimate the effect of a universal access policy on socioeconomic disparities in use of the portal	Time series (quasi-experimental)	Universal access policy (E11) Spanish translation (B6) Mobile portal system (B8)	Significant disparities in portal use by age, race, and ethnicity vanished after replacing an opt-in policy with a universal access policy (among other interventions), but disparities on the basis of income did not disappear
Casey 201679	Evaluate the effectiveness of a hands-on technology education intervention in improving portal use	Pre-post with controls (quasi-experimental)	Technology education (A1)	The intervention group sent significantly more messages (54 vs 12 control, p<.001) than the matched control group in the month post-intervention
Graetz et al. 201880	Assess if mobile access increases the frequency and timeliness of portal use by diabetes patients	Time series (quasi-experimental)	Mobile access (B8)	Mobile access increases frequency in all patients (0.78 days more per month [0.74-0.83]) and timeliness in non-White patients (64% after vs 59% before, p<.001)
Greysen et al. 201881	Evaluate the efficacy of a bedside education intervention to increase portal use by inpatients	RCT (experimental)	Bedside education (A1) Hospital-provided iPads (D10)	The intervention was feasible, however, a significant increase in mean number of logins (3.48 vs 2.94 control, p=.60) and use of key portal functions was not observed
Kim et al. 200582	Determine the impact of technical help from nurses on portal information updates by patients	Time series (quasi-experimental)	Technical assistance (A1) Public computers (D10)	Information update events occurred primarily on days when technical help was available (58%) or the day afterward (23%)b
Leisy et al. 201783	Assess the effect of an iBooks-based tutorial on comfort with portal features	Pre-post (quasi-experimental)	iBooks-based tutorial (A1)	The tutorial increased comfort levels with portal features by 20%-80%, and most patients (86%) agreed the tutorial would increase their future portal use
Leveille et al. 201684	Investigate the impact of OpenNotes on use of the portal and its functions*	Time series (quasi-experimental)	OpenNotes (B5)	Overall frequency of portal use did not change, but the proportion of login days dedicated to record viewing increased from 24% to 35%b
Lyles et al. 201885	Evaluate an in-person vs online self-paced training program on portal use	RCT (experimental)	Portal training (A1)	Training of either type increased portal use compared to usual care (21% vs 9% logins, p<.001), but no differences existed between in-person and online training
Mafi et al. 201686	Assess the impact of email alerts on whether patients viewed their doctor's notes through portals	Time series (quasi-experimental)	Email alerts (E12)	Note viewing declined substantially and immediately beginning when email alerts ceased (RR 0.29 [0.26-0.32]) and persisting until the study's end (RR 0.20 [0.17-0.23])
McInnes et al. 201387	Evaluate group training to increase portal skills in vulnerable populations with limited computer experience	Pre-post (quasi-experimental)	Group training (A1)	Portal use increased directly after training (use score of 2.00 vs 0.36 baseline, p<.001), and remained elevated 3 months later (1.36 vs 0.36 baseline, p=.01)
Navaneethan et al. 201788	Assess the effect of an enhanced portal and navigator program on portal use in CKD patients*	Pragmatic RCT (experimental)	Enhanced portal for CKD (B5) CKD navigator program (A1)	The patient navigator group reported more logins than other groups (estimated median 49 vs 37 usual care, 36 portal only, 41 navigator and portal, p=.04)b
Phelps et al. 201489	Investigate characteristics that impact persistence of portal use over time	Post only (quasi-experimental)	Assistance with first login (A1)	Provision of assistance with first login is associated with higher odds of completing the initial login (OR 3.22[2.17-4.76])b
Ramsey et al. 201790	Determine effectiveness of dedicating staff (MyChart Geniuses) to assist adolescents with portal sign-up	Post only (quasi-experimental)	MyChart Geniuses (A1)	MyChart Geniuses sign up more patients (86% vs 59% general population, p<.001), but those patients were less likely to activate their accounts (20% vs 77%, p<.001)b,c
Shaw et al. 201791	Increase portal utilization through nurse navigators and assignment of health education videos to patients	Pre-post (quasi-experimental)	Nurse navigators (A1) Assignment of videos (C9)	2 of 19 participants reported portal use in the 6 months prior to intervention, whereas 4 of 19 participants reported portal use within 30 days post-intervention
Stein et al. 201892	Assess an intervention to teach vulnerable inpatients to access their discharge summaries using a portal	RCT (experimental)	Portal training (A1) Reminder emails (E12)	Hospitalized patients who received training and email reminders were more likely to register for the portal (48% vs 11% control, p<.01)
Turvey et al. 201693	Investigate the impact of training veterans to use the Blue Button feature in the VA portal	Pilot RCT (experimental)	Blue Button training (A1) Reminder phone call (E12)	Training increased health record sharing with outside providers (90% vs 17% control, p<.001)
Weisner et al. 201694	Assess effect of a patient engagement intervention (LINKAGE) on portal use*	Nonrandomized CT (quasi-experimental)	Portal training (A1)	LINKAGE significantly increased mean number of portal login-days (IRR 1.53, p=.001) and mean number of messages sent by a provider (IRR 1.45, p=.02)

Abbreviations: RCT, randomized controlled trial; CT, clinical trial; CKD, chronic kidney disease; VA, veterans affairs; RR, relative risk; OR, odds ratio; IRR, incidence rate ratio.

^*Denotes an objective that is secondary to the study's primary objective.

^aSee Table 3 for descriptions of intervention categories.

^bEstimates calculated from published data by systematic review authors.

^cChi-squared test performed by systematic review authors.

We categorized interventions according to the components described in the System Engineering Initiative for Patient Safety (SEIPS) model.^73–75 The SEIPS model segments work systems into 5 tightly coupled components. Per the model, a person (component 1) performs a range of tasks (component 2) using various tools and technologies (component 3). Performance of tasks occurs within a physical environment (component 4) under specific organization conditions (component 5). Interventions may be made on work system processes to impact outcomes, which may target 1 or more of the 5 components. We categorized interventions based on which component(s) were addressed. One team member categorized the interventions, and a second team member with experience applying the SEIPS model (NCB) reviewed the categorizations. Using the SEIPS model allowed us to determine gaps in the targets of current interventions and shortcomings related to considering the interaction among components of the work system.

In our protocol, we initially planned to conduct a meta-analysis and grade strength of evidence as per the Evidence-Based Practice Center program guidelines.⁷⁶ Unfortunately, the paucity of literature and lack of directly comparable outcomes limited us to the systematic review component only.

RESULTS

Literature searches identified 719 potentially relevant citations. Of those, 91 studies were deemed eligible for full text review, and 18 studies fulfilled the inclusion criteria for this systematic review (Figure 1).^77–94

Figure 1.

Flow diagram for study selection

Study characteristics

Table 1 summarizes the objective, design, intervention, and main finding(s) of included studies. Most included studies were published between 2016 and 2018, with 1 study published in 2014, 1 in 2013, and 1 in 2005. Designs included 5 randomized controlled trials (28%), 1 non-randomized clinical trial (6%), 5 time series (28%), 1 pre-test post-test with concurrent controls (6%), 4 pre-test post-test without concurrent controls (22%), and 2 post-test only (11%). Studies employed a broad variety of outcome measures (Supplementary Table 2) over varied time periods, limiting their comparability. For example, when reporting portal use, studies variably reported login-days, total logins, activation, or another measure, and time periods varied from “per month” to “per 2 years.”

Table 2 reports the study demographics, eligibility criteria, setting, risk of bias, and intensity of intervention. Sample sizes of prospective studies ranged from 14 to 503 participants. Because retrospective studies often relied on portal system use data, their sample sizes included more than 10 000 or even 100 000 participants. Four out of 18 studies (22%) did not report on participants' race, and 8 (44%) did not report on ethnicity. One study (6%) included English- and Spanish-speakers, 8 (44%) included only English-speakers, and 9 (50%) did not report on language. All studies excluded pediatric populations except 1 study of adolescents. All interventions were limited to the outpatient setting except 3 that included inpatients. Intensity of intervention varied widely across studies. An example of a low-intensity intervention was one-time assistance with credentialing,⁸⁹ whereas an example of a high-intensity intervention was training participants across 4 weekly 2-hour sessions.⁸⁷

Table 2.

Characteristics of included studies

Variable	Ali et al. 201877	Ancker et al. 201778	Casey 201679	Graetz et al. 201880	Greysen et al. 201881	Kim et al. 200582	Leisy et al. 201783	Leveille et al. 201684	Lyles et al. 201885
Study Population
Total sample size	23	129 738	100	135 153	97	24	70	44 951	93
Age (mean)	41a	42a	65	61a	46a	65	61a	51	54
Female (%)	83	62	66	47	55	−	56	62	52
Race (%)
White	60	38	92	52	55	−	−	−	39
Black	5	23	2	8	20	−	−	−	29
Other	35	38	6	40	25	−	−	−	32
Latino ethnicity (%)	25	27	4	16	9	−	−	−	12
Eligibility Criteria
Illness status	≥ 1 chronic condition	Any	≥ 1 chronic condition	Diabetes	Hospitalized	Any	Ophthalmic	Any	≥ 1 chronic condition
Age range	18-95	>18	40-85	>18	>18	Adult	Adult	>18	>18
Primary language	English only	English or Spanish	English only	−	English only	−	English only	−	English only
Study Setting
Level of care	Primary or specialist	Primary	Primary	Primary or specialist	Tertiary	Primary or specialist	Specialist	Primary	Primary
Clinical setting	Outpatient	Outpatient	Outpatient	Outpatient	Inpatient	Outpatient	Outpatient	Outpatient	Outpatient
Other details	Academic	Safety net	−	−	Academic	Residential	Academic	Two sites	Safety net
Quality Assessment
Risk of bias	Medium	Low	Unclear	Low	Medium	High	Medium	Medium	Medium
Intensity of intervention	Medium	High	Unclear	Low	Medium	High	Medium	Low	Medium

Variable	Mafi et al. 201686	McInnes et al. 201387	Navaneethan et al. 201788	Phelps et al. 201489	Ramsey et al. 201790	Shaw et al. 201791	Stein et al. 201892	Turvey et al. 201693	Weisner et al. 201694
Study Population
Total sample size	14 360	14	209	11 352	96	19	70	52	503
Age (mean)	52	57	68	53a	19	60	56	68	42
Female (%)	58	7	56	40	59	63	36	12	31
Race (%)
White	75b	57	75	−	0	73	76	92	61
Black	5	21	22	−	87	16	10	−	7
Other	20	21	3	−	13	11	14	−	32
Latino ethnicity (%)	−	14	−	−	5	−	11	−	20
Eligibility Criteria
Illness status	Any	HIV or HCV	CKD	CKD	Any	Cardiac	Any	≥ 1 chronic condition	Addiction
Age range	Adult	Adult	18-80	Any	13-25	18-75	>18	Adult	>18
Primary language	−	−	English only	−	−	English only	English only	−	−
Study Setting
Level of care	Primary	Primary	Primary or specialist	Specialist	Primary	Specialist	Tertiary	Primary or specialist	Specialist
Clinical setting	Outpatient	Outpatient	Outpatient	Outpatient or inpatient	Outpatient	Outpatient	Inpatient	Outpatient	Outpatient
Other details	Academic	Veterans	Academic	−	Academic	−	Safety net	Veterans	−
Quality Assessment
Risk of bias	Medium	Medium	Low	High	Medium	High	Medium	Unclear	Low
Intensity of intervention	Medium	High	High	Low	Low	Medium	Medium	Low	High

Abbreviations: HIV, human immunodeficiency virus; HCV, hepatitis C virus; CKD, chronic kidney disease.

^aMean age estimated from categorical data.

^bRace reported for only 1 of 2 study sites; the second unreported study site is described as “predominantly white.”

–Not reported or not applicable.

Risk of bias assessment

Four out of 18 studies (22%) had low risk of bias, 9 studies (50%) had medium, 3 studies (17%) had high, and 2 studies (11%) were unclear (Table 2). The most common sources of bias included: (1) failure in design or analysis to account for important confounding and modifying variables through matching, stratification, multivariable analysis, or other approaches [10 studies, 56%]; (2) differential length of follow-up between comparison groups [5 studies, 28%]; (3) if attrition was a concern, failure to handle missing data appropriately through intention-to-treat analysis, imputation, or other approaches [4 studies, 22%]; (4) failure to rule out impact from a concurrent intervention or an unintended exposure that might bias results [3 studies, 17%]; (5) failure to blind outcome assessors to the intervention or exposure status of participants [3 studies, 17%].

Intervention categorization using the SEIPS model

Figure 2 presents the SEIPS system components intervened on in each study. Out of 18 studies, 13 (72%) intervened on the individual (person) component, 5 (28%) on the tool component (ie, patient portal), 1 (6%) on the task component (eg, prescribing portal content), 2 (11%) on the environment component, and 4 (22%) on the organization component. Seven studies (39%) intervened on 2 components, but no study intervened on more than 2. Table 3 more deeply explores the different interventions and their relationships with the SEIPS system components. In the included studies, 13 out of 26 interventions (50%) involved training or assisting patients with portal use (person component).^{79,81–83,85,87–94} Out of 26 interventions 6 (23%) involved enhancing portal content,^84,88 providing mobile access,^78,80 Spanish translation,⁷⁸ or improving usability (tool component).⁷⁷ The remaining interventions involved prescribing portal use (task component),⁹¹ offering devices or internet connectivity (environment component),^81,82 increasing portal reminders (organization component),^86,92,93 or modifying organizational policy (organization component).⁷⁸

Figure 2.

SEIPS system components intervened on in included studies

Table 3.

Categories of interventions to increase patient portal use

No.	Intervention	Description	Included Studies	Additional Examples from the Literaturea
Category A. Person-based Interventions
A1	Assist patients	Training, technical assistance, or motivation for patients, from a physician, nurse, educator, or other professional	Casey79Greysen et al.8111 more82,83,85,87–94	Professional assistance with enrollment or system use44,95–97 Computer education for patients with limited technology experience49,98 Online tutorials on portal system use99,100
A2	Engage informal care providers	Portal co-access or assistance from an informal care provider, like healthcare proxies, family members, or peers	No studies	Enabling patients to selectively share content with informal care providers101–103 Portal co-access or planned access for informal care providers104–106 Peer support for or education on portal system use
A3	Engage formal care providers	Training, assistance, or motivation for providers, to encourage them to engage their patients in portals	No studies	Training for providers to enhance portal recruitment and reduce biases107,108 Additional messages or content from trusted providers to encourage use100,109 Gamification, such as competitions, to demonstrate the highest portal use
Category B. Tool-based Interventions
B4	Simplify content	Define complex terms, simplify readability of medical text, or offer education around clinical content	No studies	Infobuttons that redirect to educational content, such as MedlinePlus54,110,111 Hyperlinks that define or explain important medical terms or acronyms111,112 Tools that simplify medical text or reduce the literacy level of content68,69,113–116
B5	Enhance content	Include novel content, improve utility of existing content, or more transparency of existing medical record information	Leveille et al.84 Navaneethan et al.88	Direct or immediate release of lab test results or the entire medical record117 Novel features (medication plans,42,118–120 messaging,102,121 OpenNotes122,123) Enhance content using voice, graphics, or video124–127
B6	Portal translation	Translation of portal text into the user's preferred language, in part or in entirety	Ancker et al.78	Conduct machine or human translation of portal content128 Incorporate education or other content originally written in multiple languages110
B7	Improve usability	Use heuristic evaluation, participatory or user-centered design to create interfaces	Ali et al.77	Personalization of the portal interface or content to the user's illness129,130 Reduce cognitive load or task complexity within the portal interface51,52,131–137
B8	Better accessibility	Provide portal interfaces for users with disabilities, or limited literacy, technology experience, or broadband access	Ancker et al.78 Graetz et al.80	Offer paper versions or other low-technology versions Mobile access for the patient portal138,139 Accommodations for elderly or disabled persons, such as voice100,130,140
Category C. Task-based Interventions
C9	Prescribe tasks	Assign patients tasks within the portal to improve understanding of care	Shaw et al.91	Assign educational content prior to starting a new medication or procedure Patient-reported outcome tracking, such as after a surgical procedure141
Category D. Environment-based Interventions
D10	Provide technology	Offer devices or internet connectivity for patients to access their portals	Greysen et al.81 Kim et al.82	Integrate tablets into the hospital environment to support bedside access142 Public computers, internet, or workstations designed to support portal use
Category E. Organization-based Interventions
E11	Modify policy	Implement policy strategies to ensure all patients receive portal access	Ancker et al.78	Universal access or “opt-out” policies, which require that all patients receive information on portal activation or use143–145
E12	Increase exposure	Increase exposure to reminders and information about portal use	Mafi et al.86 Turvey et al.93 Stein et al.92	Include information about the portal in all after-visit or discharge summaries Better advertising strategies such as text messages or email reminders

^aIncludes studies that did not meet our eligibility criteria as additional examples.

Findings of individual studies

Table 4 summarizes the key findings of included studies. Supplementary Table 2 defines each outcome measure and reports how frequently it is used.

Table 4.

Summary of key findings for included studies

		Outcomes		Findings
Study	Main Comparison	Category	Description / Timing	Intervention group	Comparison group	P value
Portal Use
Casey 201679	Received education vs did not	Messages	Number of messages sent in the 4 weeks post-intervention	54 messages	12 messages	<.001
Graetz et al. 201880	After mobile access vs before	Login-days	Mean days user logged in per month over a one-year period	2.86 days/month	2.00 days/month	<.001b
		Timeliness	Percent of lab test results viewed within 7 days over a one-year period	63.8% (Non-white race)	58.8% (Non-white race)	<.001
				72.6% (White race)	72.3% (White race)	.439
Greysen et al. 201881	Received education vs did not	Logins	Percent of patients able to login without any assistance, the same day or 7 days after training	64% (same day)	60% (same day)	.65
				58% (7 days)	55% (7 days)	.86
			Mean number of logins within 7 days  post-discharge	3.48 logins	2.94 logins	.60
		Messages	Percent of patients able to view messages without any assistance, the same day or 7 days after training	92% (same day)	77% (same day)	.04
				48% (7 days)	38% (7 days)	.55
			Mean number of message clicks within 7 days post-discharge	5.98 clicks	3.98 clicks	.33
		Test results	Percent of patients able to view test results without any assistance, the same day or 7 days after training	86% (same day)	77% (same day)	.23
				44% (7 days)	38% (7 days)	.59
			Mean number of test result clicks within 7 days post-discharge	5.68 clicks	4.36 clicks	.49
Kim et al. 200582	Technical help vs none	Updates	Percent of user-made information updates occurring when help is available	58% of updates	42% of updates	Not reported
Leveille et al. 201684	After OpenNotes vs before	Binary use	Percent of patients using the portal 6-12 months before and after OpenNotes	78% used portala	84% used portal	<.001b
		Login-days	Percent of login-days for record seeking6-12 months before and after OpenNotes	35% record seekinga	24% record seeking	<.001b
Lyles et al. 201885	In-person vs online training	Activation	Percent of patients who enrolled within 3-6 months of training	19% enrolled	20% enrolled	.9
		Logins	Percent of enrolled patients who logged in once or more within 3-6 months of training	21% logged in	20% logged in	.8
	Any training vs none	Activation	Percent of patients who enrolled within 3-6 months after training	20% enrolled	8% enrolled	<.001
		Logins	Percent of enrolled patients who logged in once or more within 3-6 months of training	21% logged in	9% logged in	<.001
Mafi et al. 201686	Email reminders vs none	Notes	Effect of discontinuing reminders on viewing Visit Notes within 30 days	1.00 (reference group)	0.20 (risk ratio)	<.001b
McInnes et al. 201387	Received education vs did not	Overall use	Mean score on a self-reported 4-item portal use scale directly after training, and again 3 months after training	2.00 (end of training)	0.36 (before training)	<.001
				1.36 (3 months later)	0.36 (before training)	.01
Navaneethan et al. 201888	Patient navigator vs none	Login-days	Mean total number of days user logged over the 2-year study period	70 days /2 years	45 days /2 years	.10
		Logins	Percent with more than 48 logins over the 2-year study period	70% (>48 logins)	46% (>48 logins)	.04
		Clicks	Median number of clicks on the portal over the 2-year study period	427 clicks	269 clicks	.08
Phelps et al. 201489	Help with credentialing vs none	Initial login	Effect of provision of assistance with credentialing on completing an initial login	3.22 (odds ratio)a	1.00 (reference group)	<.001b
Ramsey et al. 201790	MyChart Geniuses vs none	Sign up	Percent of patients who signed up for the portal during the 8-month study period	86% signed upa	59% signed up	<.001b
		Activation	Percent of signed-up patients activating their account in the 8-month study period	20% activateda	77% activated	<.001b
Shaw et al. 201791	Received education plus video assignment vs did not	Binary use	Number reporting use 30 days after intervention vs 6 months before	4 out of 19	2 out of 19	Not reported
Stein et al. 201892	Received education plus reminder emails vs did not	Activation	Percent of inpatients who registered an account ≥2 weeks post-discharge	48% enrolled	11% enrolled	<.01
		Logins	Percent who self-reported an attempt to login ≥2 weeks post-discharge	60% logged in	35% logged in	.05
Turvey et al. 201693	Received education vs did not	Blue Buttonc	Percent who gave outside providers content generated with Blue Button	90%	17%	<.001
Weisner et al. 201694	Received education vs did not	Login-days	Mean days user logged in per month in the 6 months post-intervention	1.7 days/month	1.1 days/month	.001
		Messages	Mean messages from provider per month in the 6 months post-intervention	0.6 messages/month	0.4 messages/month	.02
		Test results	Mean login-days per month for test results in the 6 months post-intervention	0.3 days/month	0.2 days/month	<.001
Predictors of Portal Use
Ali et al. 201877	Portal version 2 vs version 1	Usability	Mean System Usability Scale score	81.9 out of 100	69.2 out of 100	.049
Greysen et al. 201881	Received education vs did not	Satisfaction	Percent satisfied with using the tablet to access and navigate the portal	88% (same day)	83% (same day)	.48
Leisy et al. 201783	After tutorial vs before	Comfort	Self-reported percent increase in comfort with login in previously unenrolled patients	77% (after)	Not reported (baseline)	Not reported
Lyles et al. 201885	After any training vs before	Intended use	Percent with intention to use the portal 3–6 months after training	53% intend to use	72% intend to use	.01
		Confidence	Percent with high confidence in their ability to use the portal without help, 3–6 months after training	77% confident	67% confident	.53
		Skills	Percent with self-reported skill in portal use, 3-6 months after training	78% skilled	63% skilled	.03
Disparities in Portal Use
Ancker et al. 201778	After new policy vs before	Age	Percent of patients >65 years old who received an offer or who repeatedly used the portal in 2014 (after) vs 2011 (before)	97% received offers	27% received offers	<.001b
				11% repeated use	9% repeated use	<.001b
		Race	Difference in percent of non-black vs black patients who received an offer or who repeatedly used the portal in 2014 (after) vs 2011 (before)	0.0% difference in offers	2.1% difference in offers	<.001b
				1.5% difference in use	2.9% difference in use	<.001b

^aEstimates calculated from published data by systematic review authors.

^bP value calculated by systematic review authors.

^cBlue Button is a system for patients to download their own health records.

Patient portal use

Fifteen out of 18 studies (83%) addressed an outcome related to portal use, including portal enrollment (aka activation, credentialing, or initiation), logins, timely use, clicks, persistent use, and use of features. Commonly reported outcome measures included login-days, portal activation, binary portal use [yes/no], total portal logins, portal features viewed, and secure messages sent. Supplementary Table 2 contains definitions of each measure.

Ten out of 18 studies (56%) reported on how technical training or assistance for patients impacted portal use.^{79,81,85,87–92,94} Eight of the 10 reported or permitted calculation of statistical significance, of which 6 demonstrated benefit (the intervention increased portal use), 1 demonstrated neutrality (ie, the intervention did not impact portal use), and 1 demonstrated mixed results (ie, the intervention both increased and decreased aspects of portal use). Lyles et al.⁸⁵ found that any type of technical training increased activation (20% vs 8% control, p < .001) and binary use (21% vs 9% control, p < .001), but found no differences between in-person and online training. McInnes et al.⁸⁷ reported that training increased patients' scores on a self-reported 4-item portal use scale (mean of 2.00 after vs 0.36 before, p < .001). The increase persisted 3 months after training (mean of 1.36 at 3 months vs 0.36 before, p = .01). Navaneethan et al.⁸⁸ found that logins significantly increased when patient navigators¹⁴⁶ offered portal training and ongoing technical support (estimated median of 49 vs 37 usual care, 36 portal only, 41 navigator and portal, p = .04). Phelps et al.⁸⁹ reported that patients from health centers providing credentialing assistance had higher odds of completing an initial login (odds ratio 3.22, 95% CI: 2.17–4.76), although risk of bias was high. Stein et al.⁹² found that 1 education session for hospitalized patients, along with 2 follow-up email reminders, increased portal registration (48% vs 11% control, p < .01) but not attempted logins (60% vs 33% control, p = .05). Weisner et al.⁹⁴ reported that login-days per month significantly increased after 6 group education sessions on patient engagement and health information technology resources (mean of 1.7 vs 1.1 control, p < .001). Greysen et al.⁸¹ found that 1 individual education session for hospitalized patients did not significantly increase same-day ability to login (64% vs 60% control, p = .65) or logins within 1 week post-discharge (mean of 3.48 vs 2.94 control, p = .60). Ramsey et al.⁹⁰ reported that trained portal educators (MyChart Geniuses) signed up significantly more patients (86% vs 59% general population, p < .001), but significantly fewer patients that signed up activated their portal accounts (20% vs 77% general population, p < .001).

Five out of 18 studies (28%) observed how technical training or assistance for patients impacts use of specific features.^{79,81,82,93,94} Four of the 5 reported statistical significance, 3 for benefit (ie, increased use of features) and 1 neutral (ie, no impact on use of features). Casey⁷⁹ reported that patients sent more secure messages in the month after education (frequency of 54 vs 12 control, p < .001), although risk of bias was unclear. Turvey et al.⁹³ studied Blue Button, a portal-based system for patients to download their health records, including their continuity of care document, although risk of bias was unclear. Patients shared their continuity of care document significantly more frequently with outside providers after Blue Button training (90% vs 17% control, p < .001). Weisner et al.⁹⁴ found that 6 education sessions significantly increased secure messages per month (mean of 0.6 vs 0.4 control, p = .02) and login-days per month for laboratory test results (mean of 0.3 vs 0.2 control, p < .001). Greysen et al.⁸¹ found that 1 education session did not significantly increase clicks on secure messages (mean of 5.98 vs 3.98 control, p = .33) or clicks on laboratory test results (mean of 5.68 vs 4.36 control, p = .49) within 1 week post-discharge.

Three studies of interventions besides patient education reported or permitted calculation of statistical significance, 2 for benefit and 1 mixed. Graetz et al.⁸⁰ studied the impact of adding mobile access to computer-only access. Adding mobile access increased login-days per month (0.78 login-days more [adjusted], 95% CI: 0.74–0.83). Adding mobile access also increased timeliness, defined as percent of test results viewed within 1 week, among non-whites (63.8% vs 58.8% control, p < .001) but not among whites (72.6% vs 72.3% control, p = .439). Mafi et al.⁸⁶ studied the effect of email reminders on patients viewing their doctor's notes. Note-viewing declined substantially at 1 institution when email alerts ceased (relative risk 0.20, 95% CI: 0.17–0.23), but persisted at another institution where alerts continued (relative risk 0.94, 95% CI: 0.89–1.00). Leveille et al.⁸⁴ found that portal use decreased after OpenNotes (78% after vs 84% before, p < .001), although the statistical significance may have resulted from the large sample size and may not indicate any meaningful clinical difference. However, the percentage of login-days dedicated to record-seeking increased after OpenNotes (35% after vs 24% before, p < .001).

Predictors of patient portal use

Seven out of 18 studies (39%) reported on predictors of portal use, including offers of enrollment, patient-assessed usability, patient perceptions, and patient intended use.^{77,79,81,83,85,90,93} Three of the 7 studies reported statistical significance, 1 for benefit, 1 neutral, and 1 mixed. Ali et al.⁷⁷ found that an iterative user evaluation improved portal usability (mean System Usability Score 81.9/100 after vs 69.2/100 before, p = .049). Greysen et al.⁸¹ reported that an education session for hospitalized patients did not significantly improve satisfaction with portal access through hospital-provided tablets (88% vs 83% control, p = .48). Lyles et al.⁸⁵ found that technical training significantly increased self-reported skill in portal use (78% after vs 63% before, p = .03), but not self-reported confidence (77% after vs 67% before, p = .53). Furthermore, technical training significantly decreased intention to use the portal (53% after vs 72% before, p = .01).

Disparities in patient portal use

Only 1 study reported on how an intervention impacted health-equity–related disparities in portal use. Ancker et al.⁷⁸ studied a universal access policy, or policy declaring that all patients must be offered portal enrollment. Before the policy's implementation, vulnerable groups were less likely to receive offers of portal enrollment and subsequently use the portal. The vulnerable groups included the elderly, racial minorities, and the uninsured or publicly insured. Three years post-intervention, only insurance status remained a significant predictor in multivariate models.

DISCUSSION

A growing body of literature suggests that patient portals can prevent medical errors,^6–11 increase shared decision-making,^12–17 and improve at least certain health outcomes.^18,19 Unfortunately, more than 100 studies document disparities in portal use,^28,40–56 and interventions will be critical to ensure portals do not disproportionately benefit more advantaged populations. Despite this, our results suggest that few studies have evaluated interventions to reduce disparities in portal use. Due to the strong evidence of disparities in use, the limited research on addressing them, and the need to ensure all populations benefit from portals, we recommend that researchers shift from identifying disparities in portal use to systematically addressing them. Additionally, we recommend that future studies measure interventions' impact on disparities in use directly, as most studies to date have not. Finally, categorization using the SEIPS model demonstrated that most interventions to date addressed only the individual (person) component, and lacked coverage of the other components as well as combinations of components. To enhance impact, we recommend that future interventions affect, or at least consider the repercussions on, multiple components.

Out of 18 included studies, 15 assessed the intervention's impact on portal use and 7 on predictors of use. Surprisingly, only 1 study⁷⁸ assessed impact on disparities in use. To generate the best evidence on how interventions impact disparities in portal use, future studies should measure these disparities directly. This may include disparities on age, sex, race, ethnicity, preferred language, insurance status, income, level of education, technology access, technology experience, health literacy, numeracy, functional literacy, illness status, and disability status. Surprisingly, almost half of included studies did not report participants' race and ethnicity. At minimum, studies should report participants' age, sex, race, and ethnicity, which will enable readers to better interpret results and determine generalizability.

Technical training and assistance programs for patients currently have the best evidence for increasing portal use in vulnerable populations. Other interventions have not been sufficiently studied to draw conclusions. Thirteen out of 18 studies focused on patient education, either alone (7 studies) or in combination with other interventions (6 studies). In other research domains such as patient safety, training is considered a weak action because it affects 1 individual at a time without reducing the systemic drivers of error¹⁴⁷ or, in this case, the systemic drivers of inequity. In contrast, strong actions eliminate potential sources of error (or inequity) from a system. For example, a weak action may involve training a patient to mitigate issues related to portal usability, whereas a strong action would involve re-designing the interface to eliminate usability issues. Additional examples of strong actions may include: (1) free or low-cost internet access via smartphone or broadband, (2) data delivery through 2G and 3G networks in addition to 4G, (3) creating accessible and easily understandable policies, and (4) ensuring software adheres to accessibility, legibility, and readability standards for persons with disabilities and elderly persons.¹⁴⁸ Importantly, strong actions have been demonstrated to be more sustainable as they facilitate system-wide impact,¹⁴⁹ as opposed to impact on an individual-by-individual basis. Strength of action frameworks designed for patient safety do, however, acknowledge that weak actions may be necessary stopgap solutions while stronger actions are implemented.¹⁴⁷

The interventions we reviewed were heterogeneous in type and intensity, and could be categorized using various approaches. Categorization based on the SEIPS model was not meant to be an all-encompassing approach, but was meant to inform concerned researchers, clinicians, and administrators on the gaps in the current literature. Interestingly, few studies intervened on multiple components of the work system (person, tool, task, environment, and organization) or combined multiple intervention types. The SEIPS model stresses the importance of considering the tightly coupled, interactive nature of system components.^73–75 Future work should explore composite approaches that address multiple components and leverage multiple types of interventions to maximize impact. As an example, the recent PRISM (Personal Reminder Information and Social Management) randomized controlled trial evaluated a multi-component intervention to improve social support for older adults.^150,151 Participants received computers (technology component) with iteratively designed programs (task component), and received internet access (environment component), computer use training (individual component), and organizational support (organization component) as needed. The intervention demonstrated efficacy for improving social support. The efficacy of similar multi-component interventions for improving portal use remains to be studied.

The included studies reported several unintended consequences of interventions. Ramsey et al.⁹⁰ found that fewer patients signed up by MyChart Geniuses activated their portal accounts. One potential reason is that MyChart Geniuses target patients with lower technology literacy than the general population, and the intervention is insufficient to overcome technology-literacy–based barriers to activation. This hypothesis is consistent with previous research suggesting that technical assistance with activation is insufficient to overcome barriers to subsequent use.¹⁴³ Leveille et al.⁸⁴ reported that portal use decreased after OpenNotes, and Lyles et al.⁸⁵ found technical training significantly decreased intention to use the portal. Reasons for these unintended consequences remain to be explored.

In the included studies, measures of portal use varied greatly in definition and in timing. To create comparable evidence, the field will need to develop standardized measures or metrics of portal use. Single measures many not provide the best overall picture of portal use, and composite metrics may be needed. For example, logins may not accurately reflect use in situations where patients login infrequently, but spend hours browsing after each login. Common metrics from the web analytics domain include downloads, installations, acquisition, user growth rate, retention rate, churn rate, stickiness, session length, and daily or monthly active use.¹⁵² Commonly used web and mobile analytics software may help researchers record additional metrics of portal use.

The included studies almost always excluded non-English speakers and hospitalized patients. Therefore, results may not apply to these populations. The studies we examined were conducted in various outpatient settings in the US, including academic, safety net, and veterans hospitals. Therefore, the findings are more likely to apply to the outpatient setting. Five out of 18 studies had high or unclear risk of bias. In a recent review, Showell⁴⁰ identified common sources of selection bias in studies of portal users, including: (1) exclusion of participants with critical illness, (2) exclusion of non-English speakers, and (3) exclusion of participants with limited technology experience. Recruiting these populations is resource-intensive and time-consuming,¹⁵³ but necessary to reduce selection bias and ensure generalizability.

Limitations

A potential limitation of our review is incomplete retrieval of relevant research. Because we included a broad variety of study designs, intervention types, and outcome measures, developing an inclusive search strategy proved difficult. Occasionally, Medical Subject Headings did not include relevant terms (for example, no term for “patient portal use” exists). We mitigated these limitations by collaborating with an experienced librarian and incorporating supplemental search strategies such as table-of-contents review of pertinent journals. However, we cannot exclude the possibility that we missed potentially eligible studies. Another potential limitation is publication bias and selective reporting. We do not have information about unpublished studies or outcomes, limiting our certainty about the potential for publication bias. Several studies did not report statistical significance for outcomes, limiting what we could extract from the literature. In 3 included studies, the primary outcome differed from the portal-related outcome, meaning the portal-related outcome was potentially underpowered, less detailed, or analyzed in a post-hoc manner.

CONCLUSION

Disparities in patient portal use may worsen existing health inequities and prevent portals from benefiting all populations. More than 100 studies have described disparities in portal use, however, our review suggests that far fewer have evaluated interventions to overcome disparities. We found that most interventions focused on the individual, rather than including the portal-, task-, environment-, or organization-based components, which could increase their effectiveness. Additional research is urgently needed to identify effective, cross-cutting interventions that reduce disparities in portal use.

FUNDING

This work was supported by the National Library of Medicine (R01LM012964, PI: Ancker; T15LM007079, trainee: Grossman), the National Institute of Nursing Research (R00NR016275, PI: Masterson Creber), and the Agency for Healthcare Research and Quality (R01HS21816, PI: Vawdrey).

CONTRIBUTORS

LVG, DKV, and JSA collaboratively conceptualized this review. DW conducted the literature searches. LVG, RMC, and JSA conducted the initial and full-text screening. LVG, RMC, NCB, and JSA performed the data extraction, risk of bias assessment, and analyses. LVG drafted the manuscript, and all authors contributed to refining all sections and critically editing the paper.

SUPPLEMENTARY MATERIAL

Supplementary material is available online at Journal of the American Medical Informatics Association.

CONFLICT OF INTEREST STATEMENT

None declared.