Inpatient Portals for Hospitalized Patients and Caregivers: A Systematic Review

Abstract

Patient portals, web-based personal health records linked to electronic health records (EHRs), provide patients access to their healthcare information and facilitate communication with providers. Growing evidence supports portal use in ambulatory settings; however, only recently have portals been used with hospitalized patients. Our objective was to review the literature evaluating the design, use, and impact of inpatient portals, which are patient portals designed to give hospitalized patients and caregivers inpatient EHR clinical information for the purpose of engaging them in hospital care. Literature was reviewed from 2006 to 2017 in PubMed, Web of Science, CINALPlus, Cochrane, and Scopus to identify English language studies evaluating patient portals, engagement, and inpatient care. Data were analyzed considering the following 3 themes: inpatient portal design, use and usability, and impact. Of 731 studies, 17 were included, 9 of which were published after 2015. Most studies were qualitative with small samples focusing on inpatient portal design; 1 nonrandomized trial was identified. Studies described hospitalized patients’ and caregivers’ information needs and design recommendations. Most patient and caregiver participants in included studies were interested in using an inpatient portal, used it when offered, and found it easy to use and/or useful. Evidence supporting the role of inpatient portals in improving patient and caregiver engagement, knowledge, communication, and care quality and safety is limited. Included studies indicated providers had concerns about using inpatient portals; however, the extent to which these concerns have been realized remains unclear. Inpatient portal research is emerging. Further investigation is needed to optimally design inpatient portals to maximize potential benefits for hospitalized patients and caregivers while minimizing unintended consequences for healthcare teams.

Engaging patients and their caregivers in care improves health outcomes^1–3 and is endorsed by leading health-care organizations as essential to improving care quality and safety.^4–6 Patient engagement emphasizes that patients, caregivers, and healthcare providers work together to “promote and support active patient and public involvement in health and healthcare and to strengthen their influence on healthcare decisions.”⁷ Patient portals, web-based personal health records linked to electronic health record (EHR) data, are intended to promote engagement by providing patients and their caregivers with timely electronic access to their healthcare information and supporting communication through secure messaging with their healthcare team.⁸ The use of patient portals has also been suggested as a way for patients and/or caregivers to identify and intercept medical er rors, thus having the potential to also improve patient safety.^8,9

As a requirement for meaningful use, access to health information through patient portals in the ambulatory setting has increased dramatically.¹⁰ Studies evaluating the use of these patient portals to promote patient-centered care are growing, but evidence supporting their impact on improved health outcomes is currently insufficient.^11–15 Although research and policy focus on the use of patient portals in the ambulatory setting, recent literature suggests that patient portals may be used to share inpatient clinical information to engage patients and their caregivers during their hospitalization.^16–18 Before the widespread use of patient portals in the inpatient setting is endorsed, systematic research is needed to understand optimal portal design requirements, if and how these portals are used, and whether their use provides value to the hospitalized patient and/or caregiver.⁸

Prior literature summarized early findings regarding the use of various technologies designed to engage hospitalized patients.^17,19,20 In this systematic review, we describe the emerging literature examining the design, use, and impact of inpatient portals for hospitalized patients and/or caregivers over the last 10 years. Inpatient portals are defined here as electronic patient portals tethered to EHRs that are designed to provide hospitalized patients and/or caregivers secure access to personalized, inpatient clinical information with the intent of engaging them in their hospital care. After analyzing and summarizing these data, we then identify knowledge gaps and potential future research directions.

METHODS

Search Strategy, Study Selection, and Analysis

This systematic review included available, peer-reviewed, and grey literature published from January 1, 2006, to August 8, 2017, in PubMed, Web of Science (including the Institute of Electrical and Electronics Engineers Xplore), Cochrane, CINAHLPlus, and Scopus databases. Terms and phrases, including those found in the Medical Subject Heading (MeSH) index, were used to identify studies evaluating (1) patient portals (“health record, personal [MeSH],” “personal health record,” “patient portal,” “inpatient portal,” “ipad,” “tablet,” or “bedside information technology”), (2) engagement (“engagement,” “empowerment,” “participation,” “activation,” or “self-efficacy”), and (3) in the hospital (“inpatient [MeSH],” “hospital [MeSH],” “hospitalized patient [MeSH],” or “unit”). MeSH terms were used when applicable. Based on previous literature, free-text terms were also used when subject headings were not applied consistently, such as with terms related to engagement.^17,21 Studies were excluded if they were not written in English, if they evaluated portals exclusively in the emergency department or ambulatory setting, and/or if they described future study protocols. Studies describing general inpatient technology or evaluating portals used in the hospital but not tethered to inpatient EHR clinical data were also excluded.

By using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines,²² 2 researchers (M.K. and P.H.) completed the literature search and potential article screening. Results were aggregated and studies were screened and excluded from full review based on title and abstract information. Additional studies were included after reference list review. During a full review of included studies, 2 researchers independently extracted data, including the study objective, design, setting, sample, data collection instruments, outcomes, and a description of results. Guided by our study objective, findings were reconciled by consensus and analyzed and described according to the following 3 themes: (1) inpatient portal design, (2) inpatient portal use and usability, and (3) the impact of inpatient portal use on patient or caregiver and healthcare team outcomes as defined by retrieved studies.

The quality of studies was evaluated by the same 2 researchers independently by using the Downs and Black checklist for assessing the methodological quality of randomized and nonrandomized healthcare interventions.²³ Qualitative studies describing the development of portal prototypes and/or portal redesign efforts were excluded from these analyses. Discrepancies were resolved by consensus. Because of the wide variability in study designs, populations, and outcomes, a meta-analysis of pooled data was not performed.

RESULTS

Of the 731 studies identified through database searching and reference review, 36 were included for full-text review and 17 met inclusion criteria (Figure; Table 1). Studies excluded after full-text review described portal use outside of the inpatient setting, portals not linked to hospital EHR clinical data, portals not designed for inpatients, and/or inpatient technology in general. The inpatient portal platforms, hardware used, and functionalities varied within included studies (Table 2). The majority of studies used custom, web-based inpatient portal applications on tablet computers. Most provided information about the patients’ hospital medications, healthcare team, and education about their condition and/or a medical glossary. Many included the patient’s schedule, hospital problem list, discharge information, and a way to keep notes.

FIG.

Article selection flow chart adapted from Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

TABLE 1.

Summary of Inpatient Portal Literature Included

Authors, year	Study Objectives	Study Design	Sample	Results
Vawdrey et al., 201116	To assess patient’s knowledge of inpatient care and usefulness of portal prototype	Qualitative, interviews	5 postop patients on the cardiac unit	Patients perceived portal use would improve satisfaction and engagement. They found it useful but had varying levels of comfort using it. Patients identified unmet needs, including the ability to send messages, give feedback, enter outpatient medications, and see additional information about their healthcare team.
Weyand et al., 201124	To develop, implement, and evaluate the usability of a NICU decision support tool	Qualitative, multiphase	Neonatal experts; 8 parents of former NICU patients	Parents found the portal easy to use, would use the tool, and made suggestions for improvement, such as a glossary describing medications and side effects.
Caligtan et al., 201225	To identify data elements to define requirements for a bedside communication tool prototype	Qualitative, multiphase	41 healthcare team members, 7 inpatients; 30 nurses, 30 inpatients	37 information requirements were identified. Patients indicated the need for a daily plan, schedule, recovery goals, and room/hospital information. Nurses were more interested in safety. Other information requested included discharge information, education, medications, and healthcare team names/photos.
Wilcox et al., 201226	To assess needs of patients to inform the design of inpatient medication electronic views	Qualitative, interviews	11 inpatients, 6 nurses on cardiac step-down unit	Patients and nurses agreed on value. General themes emerged regarding the need for medication tracking, progress, decision-making, education, information, and formatting. Patients indicated the need for information about medication dosage, frequency, administration, photos, criticality, and education (alternatives, indications, side effects).
Dykes et al., 201327	To build and test an electronic bedside communication center prototype	Qualitative, multiphase	Patients/caregivers, volunteers; 8 inpatients, 3 families	Most participants would use the prototype, were satisfied with it, and found it useful and easy to use. Recommendations for improvement were made, including the need to involve the patient in communication and development of the care plan.
Dykes et al., 201428	To identify workflow and design enhancements of an electronic bedside communication center to develop a patient-centered toolkit	Qualitative, multiphase	12 advisory council; 18 nurses, 10 physicians; 5 inpatients, 2 families	Participants confirmed prior needs (above). Participants desired tools within the portal to communicate their goals, problems, concerns, and care preferences directly with the care team along with giving feedback on how well the care team was assisting them to meet these goals.
Pell et al., 201529	To evaluate patient and healthcare team experiences using a portal before and after implementation	Before and after study without control	50 inpatients, 28 clinicians, 14 nurses	Patients who used it were positive about it improving empowerment, understanding, reassurance, and their ability to follow health recommendations. Patients didn’t report having more knowledge about discharge timing. Most clinicians thought it would increase their workload and that patients would worry more. These concerns decreased postportal implementation.
Yoo et al., 201530	To design a smart bedside station terminal based on patient/caregiver experiences and healthcare team workflow	Qualitative, multiphase	Multiple inpatients, care-givers, nurses, clinicians, researchers	Participants describe user needs and design components that went in to the development of the bedside terminal. These include information regarding access to inpatient health information and a schedule, addressing privacy issues, integrating into hospital processes, and improving the patient-caregiver relationship.
Dalal et al., 201631	To evaluate a patient-centered toolkit, including enrollment strategy, use and usability, and content of patient-generated messages	Cross-sectional	119 inpatients, 120 caregivers in a medical ICU or oncology unit	Participants found the portal usable, useful, and identified adoption barriers and strategies to promote use. Most frequently used functionalities included goals, results, care team, messages, and medications. 66% and 41% of participants entered a daily and overall goal. Messages included concerns, preferences, needs, and questions.
Kaziunas et al., 201632	To explore the needs of patients/caregivers to design and develop a bone marrow transplant roadmap	Qualitative, observations/interviews	17 caregivers of pediatric bone marrow transplant patients > 10 y/o	Participants identified 3 stages of the caregiving experience that may be improved by using the portal: (1) navigating the health system and communicating with the healthcare team, (2) managing caregiving challenges, and (3) transitioning from inpatient to outpatient care.
Kelly et al., 201633	To assess inpatient portal use, parent perceptions of impact on care safety, quality, and communication	Cross-sectional	90 parents of children <12 y/o on medical surgical unit	Most parents were satisfied, found it easy to use and useful, and increased their ability to monitor and care for child. Less perceived it improved communication. 8% found a medication error by using the portal.
Maher et al., 201634	To examine user views, needs, and wants to design and develop bone marrow transplant roadmap	Qualitative, multiphase	11 caregivers, 8 pediatric bone marrow transplant patients >10 y/o	Participants were generally satisfied with functionalities and found the portal useful. Recommendations for improvement were suggested, such as using it to improve the discharge transition through a “continuing the journey” icon and helping with emotional issues.
O’Leary et al., 201618	To assess the effect of using an inpatient portal on patient knowledge and activation	Nonrandomized trial	102 general medical inpatients on control unit, 100 on intervention unit	80% of intervention patients used it, 76% said was easy to use, and 71% said it was useful. More intervention patients could name their physician and role, but patient activation and knowledge of nurse names, planned tests and procedures, and medication changes were not significantly different between groups.
O’Leary et al., 201635	To evaluate patient and provider perceptions of an inpatient portal and identify barriers to use and enhancements	Qualitative, interviews/focus groups	18 inpatients, 21 providers	Patients found portal information useful and enjoyed entertainment. Patient enhancement suggestions included more information on medications and results and the ability to record questions. Providers perceived that portal use improved engagement but enhancements may overwhelm patients and their communication and workflow.
Wilcox et al., 201636	To evaluate the usability, use, and usefulness of hospital medication tool for patients to inform its redesign	Qualitative, multiphase	20 post-op inpatients, 2 families; 5 pharmacists	An interactive inpatient medication-tracking tool was refined. 70% of patients used it to review medications and log questions and comments. 90% found it useful. Improvements were suggested, such as providing a medication schedule, administration methods, and lay term explanations.
Woollen et al., 201637	To investigate patients’ use, experiences, and information needs using an inpatient portal	Qualitative, interviews	14 postop cardiac inpatients and families on a step-down unit	86% of patients used it and 93% wanted more information even if not fully understandable. Most perceived portal use helped address their needs and increased understanding. Most useful features included medications and care team information. Enhancements were suggested, including physician notes, operative reports, medical condition information, test results, and patient-friendly education.
Kelly et al., 201738	To evaluate healthcare team perceptions before and after implementation of a tablet-based inpatient portal	Repeated cross-sectional	94 healthcare team members on general care unit pre- then 70 postimplementation	All healthcare team respondents perceived challenges, including parents would have too many questions, parents would know test results before the healthcare team, staff would be skeptical, and there would not be enough technical support. All perceived challenges were significantly reduced after implementation.

NOTE: Abbreviations: ICU, intensive care unit; NICU, neonatal intensive care unit; y/o, years old.

TABLE 2.

Inpatient Portal Platform, Hardware Used, and Patient and Caregiver-Facing Functionalities Specified in Each Included Study

There has been a recent increase in inpatient portal study publication, with 9 studies published during or after 2016. Five were conducted in the pediatric setting and all but 130 with English-speaking participants. Twelve studies were qualitative, many of which were conducted in multiple phases by using semi-structured interviews and/or focus groups to develop or redesign inpatient portals. Of the remaining studies, 3 used a cross-sectional design, 1 used a before and after design without a control group, and 1 was a nonrandomized trial. Studies were rated as having medium-to-high risk of bias because of design flaws (Table 1 in supplementary Appendix). Because many studies were small pilot studies and all were single-centered studies, the generalizability of findings to different healthcare settings or patient populations is limited.

Inpatient Portal Design

Most included studies evaluated patient and/or caregiver information needs to design and/or enhance inpatient portals.^16,24–37 In 1 study, patients described an overall lack of information provided in the hospital and insufficient time to understand and remember information, which, when shared, was often presented by using medical terminology.³⁰ They wanted information to help them understand their daily hospital routine, confirm and compare medications and test results, learn about care, and prepare for discharge. Participants in multiple studies echoed these results, indicating the need for a schedule of upcoming clinical events (eg, medication administration, procedures, imaging), secure and timely clinical information (eg, list of diagnoses and medications, test results), personalized education, a medical glossary, discharge information, and a way to take notes and recognize and communicate with providers.

Patients also requested further information transparency,^34,37 including physicians’ notes, radiology results, operative reports, and billing information, along with general hospital information,¹⁶ meal ordering,³³ and video conferencing.²⁷ ln designing and refining an inpatient medication-tracking tool, participants identified the need for information about medication dosage, frequency, timing, administration method, criticality, alternative medications or forms, and education.^26,36 Patients and/or caregivers also indicated interest in communicating with inpatient providers by using the portal.^{16,27,28,30–37} In 1 study, patients highlighted the need to be involved in care plan development,²⁷ which led to portal refinement to allow for patient-generated data entry, including care goals and a way to communicate real-time concerns and feedback.²⁸

Studies also considered healthcare team perspectives to inform portal design.^{25,26,28,30,35,37} Although information needs usually overlapped, patient and healthcare team priorities differed in some areas. Although patients wanted to “know what was going to happen to them,” nurses in 1 study were more concerned about providing information to protect patients, such as safety and precaution materials.²⁵ Similarly, when designing a medication-tracking tool, patients sought information that helped them understand what to expect, while pharmacists fo cused on medication safety and providing information that fit their workflow (eg, abstract medication schedules).³⁶

Identified study data raised important portal interface design considerations. Results suggested clinical data should be presented by using simple displays,²⁸ accommodating real-time information. Participants recommended links^16,29 to per sonalized patient-friendly³⁷ education accessed with minimal steps.²⁶ Interfaces may be personalized for target users, such as patient or proxy and younger or older individuals. For example, older patients reported less familiarity with touch screens, internal keyboards, and handwriting recognition, favoring voice recognition for recording notes.²⁷ This raised questions about how portals can be designed to best maintain patient privacy.²⁵ Interface design, such as navigation, also relied heavily on hardware choice, such as tablet versus mobile phone.²⁸

Inpatient Portal Use and Usability

Most patient and/or caregiver participants in included studies were interested in using an inpatient portal, used it when offered, found it easy to use, useful, and/or were satisfied with it.^{16,18,24–37} Most used and liked functionalities that provided healthcare team, test result, and medication information.^22,33,37 In the 1 identified controlled trial,¹⁸ researchers evaluated an inpatient portal given to adult inpatients that included a problem list, schedule, medication list, and healthcare team information. Of the intervention unit patients, 80% used the portal, 76% indicated it was easy to use, and 71% thought it provided useful information. When a portal was given to 239 adult patients and caregivers in another study, 66% sent a total of 291 messages to the healthcare team.³¹ Of these, 153 provided feedback, 76 expressed preferences, and 16 communicated concerns. In a pediatric study, an inpatient portal was given to 296 parents who sent a total of 36 messages and 176 requests.³³ Messages sent included information regarding caregiver needs, questions, updates, and/or positive endorsements of the healthcare team and/or care.

Impact of Inpatient Portal Use

Multiple studies evaluated the impact of inpatient portal use on patient and/or caregiver engagement, empowerment, activation, and/or knowledge, which had mixed results. Most adult patients interviewed in one study had positive experiences using a portal to answer their questions between physician visits and learn about, remember, and engage in care.³⁷ A majority of adult inpatient portal users in another study agreed that portal use helped them feel in control and understand their condition; however, they did not report having improved discharge timing knowledge.²⁹ In a pediatric study, most parent inpatient portal users agreed use improved their ability to monitor, understand, and make decisions about their child’s care.³³ In the controlled trial,¹⁸ a higher percentage of portal intervention patients could identify their physician or role; however, patient activation was not statistically different between intervention and control patients.

Results from included studies also evaluated the impact of portal use on communication. Some suggest inpatient portal use may replace and/or facilitate verbal communication between patients, caregivers, and providers.³⁵ In a pediatric study, 51% of parent portal users reported it gave them the information they needed, reducing the amount of questions they had for their healthcare team.³³ Similarly 43% of 14 adult inpatient portal users in another study thought the portal could replace at least some face-to-face communication.³⁷ Some providers indicated portal use enhanced rounding discussion quality.³⁵ Another study suggested that patient-provider communication via electronic messaging may provide benefits for some patients and not others.³⁷

Multiple studies evaluated patient, caregiver, and/or health-care team perceptions of the impact of inpatient portal use on detection of errors and patient safety.^29,31,33,35 In adult inpatients, 6% agreed portal use could help them find errors.²⁹ In a pediatric study, 8% reported finding at least 1 medication error by using the portal, and 89% thought use reduced errors in their child’s care.³³ One patient in a qualitative study of adult inpatients cited an example of a dosing error discovered by using the portal.³⁷ Healthcare providers in another study also reported that use facilitated patient error identification.³⁵

Included studies evaluated the potential impact of portal use on patient anxiety, confusion, and/or worry, and the work of healthcare teams. In 1 study, nurses voiced concerns about giving information subject to change or that couldn’t always be achieved because of competing hospital priorities, such as discharge timing.²⁵ They also worried about giving medical information that would create cognitive overload for patients and/or require professional interpretation. Although providers in another study perceived little negative impact on their workflow after portal implementation, they worried about the potential of adding other information to the portal.³⁵ For example, they were concerned that the future release of abnormal test results or sensitive data would lead to confusion and more time spent answering patient questions. Physicians also worried that secure messaging could be overused by patients, would be used to inappropriately express acute concerns, or might adversely affect verbal communication. Providers in 2 studies expressed concerns about potential negative implications of portal use on their work before implementation, which were subsequently reduced after portal implementation.^29,38 Conversely, no parent portal users in another study thought portal information was confusing.³³ One parent participant noted portal use may actually decrease anxiety: “Access to their medical information gives patients and their caregivers perspective and insight into their hospital care and empowers them with knowledge about [what is going on], which reduces anxiety.”³⁷

DISCUSSION

We identified multiple studies evaluating the design, use, and impact of inpatient patient portals for hospitalized patients and caregivers. Based on the information needs identified by patients and healthcare team participants, multiple key content and design recommendations are suggested, including presenting (1) timely, personalized clinical and educational infor mation in lay terms, (2) the care trajectory, including care plan and patient schedule, and (3) a way to recognize and communicate with the inpatient healthcare team. Design challenges still exist, such as translating medical terminology from EHRs into patient-friendly language, proxy access, and portal integration across transitions. Data from identified studies suggest hospitalized patients and caregivers are interested in and willing to use inpatient portals, but there is less information about the use of each functionality. Evidence supporting the role of inpatient portal use in improving patient and/or caregiver engagement, knowledge, communication, and the quality and safety of care is currently limited. Included studies indicate that healthcare team members had concerns about using portals to share clinical information and communicate electronically in the hospital. The extent to which these concerns translate to demonstrable problems remains to be seen.

Early studies focus on patient and caregiver information needs and portal interface design. Although the necessity for certain core functionalities and design requirements are becoming clear,²⁰ best practices regarding the amount and timing of information released (eg, physician notes, lab results), optimal hardware decisions (eg, large-screen displays, hospital-owned tablets, bring-your-own-device model), and details around secure-messaging implementation in the acute hospital setting are still lacking. Future work is needed to understand optimal patient-provider communication architectures that support improved synchronous and asynchronous messaging and privacy-preserving approaches to the design of these systems to handle patient-generated data as it becomes more commonplace. Although patient participants in these studies were generally satisfied using inpatient portals, many indicated the need for even more transparency, such as the release of results in real time and inclusion of physician notes (even if they could not be fully comprehended).³⁷ As the movement of sharing notes with patients in the ambulatory setting grows,³⁹ it will inevitably extend to the inpatient setting.⁴⁰ Further research is needed to understand the impact of increased transparency on health outcomes, patient anxiety, and inpatient healthcare team workload. Although the majority of studies described the design and/or use of custom portal platforms, EHR vendors are now developing inpatient portals that integrate into preexisting systems (eg, MyChart Bedside, Epic Systems). This will increase the likelihood of broad inpatient portal adoption and may facilitate multicenter trials evaluating the impact of their use.

The next steps will need to focus on the evaluation of specific inpatient portal functionalities and the impact of their use on objective process and outcome measures by using rigorous, experimental study designs. Akin to ambulatory portal research, measures of interest will include patient activation,^41,42 patient and/or caregiver satisfaction,⁴³ care processes (eg, length of stay, readmissions), and patient safety (eg, safety perceptions, adverse drug events, hospital-acquired conditions, and diagnostic errors). More than a mechanism for unidirectional sharing information from providers to the patient, inpatient portals will also provide a platform for the reciprocal exchange of information from the patient to the provider through patient-generated data, such as goal setting and feedback. Patients may play a larger role in reporting hospital satisfaction in real time, reconciling medications, contributing to the treatment plan, and identifying medical errors. As portals are integrated across the care continuum,²⁰ our understanding of their impact may become more clear.

In this review, only 5 studies were conducted in the pediatric hospital setting.^{24,32–34,38} With hospitalized children experiencing 3 times more harm from medical errors than adults,⁴⁴ engaging parents in inpatient care to improve safety has become a national priority.⁴⁵ Giving patient portals, or “parent portals,” to parents of hospitalized children may provide a unique opportunity to share healthcare information and promote engagement, a direction for future study. There is also a research gap in evaluating adolescent inpatient portal use. Future portals may be designed to incentivize young children to learn about their hospitalization through games linked to health-related education.

Finally, as patients and caregivers begin using inpatient portals, there will almost certainly be consequences for health-care teams. Understanding and anticipating human and work system factors influencing inpatient portal adoption and use from the perspectives of both patients and healthcare teams are needed.^46,47 Engaging healthcare team members as valuable stakeholders during implementation and measuring the impact of portal use on their workload is necessary, especially as portal use spreads beyond pilot units. The success of inpatient portals is dependent upon both the positive benefits for patients and their acceptance by healthcare teams.⁴⁸

Limitations exist in conducting a systematic literature review.⁴⁹ The conceptual definition of a portal for hospitalized patients and patient/caregiver engagement is evolving; therefore, our definition may not have captured all relevant studies. We intentionally did not include all inpatient technology, as we were interested in a narrow definition of portals designed for inpatients that provided clinical information from the inpatient EHR. Because of rapid technology changes, we also limited our search to studies published within the last 10 years; prior literature has been described elsewhere.¹⁷ We excluded non-English language studies, limiting our ability to capture the full scope of inpatient portal research. These patients already experience healthcare delivery disparities, widened by the inaccessibility of innovative health information technologies.⁵⁰ Future studies would be enhanced with the inclusion of these participants.

Inpatient portal research is in its infancy but growing rapidly. Studies to date are primarily focused on portal design and have small sample sizes. Early findings suggest that patients and caregivers are, in general, enthusiastic about using inpatient portals. Further research is needed, however, to determine the impact of inpatient portal use on patient engagement and hospital-care quality, safety, and cost.

Appendix Table 1

Appendix Table 1.

Quality assessment for included studies using a modified^* Downs and Black checklist²³

Authors, year	Study design	Reporting (Maximum 11)	External validity (Maximum 3)	Bias (Maximum 7)	Confounding (Maximum 6)	Power (Maximum 1)
Pell et al., 201529	Before-after study without control	4	0	0	0	0
Dalal, et al.,201631	Cross-sectional	6	0	2	0	0
Kelly et al., 201633	Cross-sectional	5	1	1	0	0
O'Leary, et al., 201618	Non-randomized trial	7	1	5	3	1
Kelly et al., 201738	Repeated cross-sectional	4	1	1	0	0

^*Higher scores reflect less bias. The power assessment was modified from a 0-5 to a 0-1 scale, where the item was scored "1" if a power calculation or sample size calculation was present and "0" if there was no power/sample size calculation or an explanation of the appropriateness of the number of subjects.