A web-based, patient-centered toolkit to engage patients and caregivers in the acute care setting: a preliminary evaluation

Abstract

We implemented a web-based, patient-centered toolkit that engages patients/caregivers in the hospital plan of care by facilitating education and patient-provider communication. Of the 585 eligible patients approached on medical intensive care and oncology units, 239 were enrolled (119 patients, 120 caregivers). The most common reason for not approaching the patient was our inability to identify a health care proxy when a patient was incapacitated. Significantly more caregivers were enrolled in medical intensive care units compared with oncology units (75% vs 32%; P < .01). Of the 239 patient/caregivers, 158 (66%) and 97 (41%) inputted a daily and overall goal, respectively. Use of educational content was highest for medications and test results and infrequent for problems. The most common clinical theme identified in 291 messages sent by 158 patients/caregivers was health concerns, needs, preferences, or questions (19%, 55 of 291). The average system usability scores and satisfaction ratings of a sample of surveyed enrollees were favorable. From analysis of feedback, we identified barriers to adoption and outlined strategies to promote use.

INTRODUCTION

There is increasing support for providing patients with tools to actively participate in their care. ^1–5 Meaningful patient engagement can enhance patient-provider interactions, promote behavioral change, and improve understanding of and adherence to the plan of care. ^2,4–7 These efforts should result in improved self-management, more favorable outcomes, and lower costs. ^8–10

The acute care environment presents unique opportunities and challenges for engaging patients and caregivers in developing and executing the plan of care. ^1,3 Patients often receive inconsistent education regarding their plan of care. ^11–13 The members of the care team typically maintain the plan of care, but patients’ goals, priorities, and preferences may not be efficiently conveyed to them. ^14,15 Finally, the informal dialog that transpires among patients and health care providers is fragmented; it is increasingly difficult for patients to communicate with and receive a consistent message from their providers as the plan of care evolves. ^12,16,17 Lack of a patient-centered plan of care can contribute to poor patient-reported outcomes, suboptimal utilization of resources, and preventable harm. ¹⁸

Catalyzed by public reporting of patient satisfaction and experience, health care organizations have been increasingly implementing health information technologies (HIT) to engage patients and caregivers in clinical settings. ^{5, 6,19–24} The bulk of experience for engaging patients using portals is in ambulatory settings; there are considerable knowledge gaps in hospital settings. ^25,26 For example, patient portals typically do not ask patients to establish goals, priorities, and preferences, or provide a mechanism to facilitate communication with the rapidly changing inpatient care team. ^27,28 Also, implementing HIT poses organizational challenges, and limited data exist regarding the extent to which caregivers are willing to use online portals when patients are critically ill or incapacitated. ^29–31

We have designed and developed a patient-centered toolkit (PCTK) that provides patients and caregivers tools to participate in their plan of care during hospitalization. ^32–34 The PCTK is a suite of web-based patient-facing and provider-facing tools designed to facilitate collaborative decision making by providing access to tailored educational content and facilitating patient-provider communication. ^34–40 The PCTK is accessible by patients, caregivers, and providers from any web-enabled device connected to the hospital’s secure intranet. In this study, we evaluate our enrollment strategy, use and usability of patient tools, and the content of patient-generated messages.

METHODS

Setting and Participants

The PCTK was developed at the Brigham and Women’s Hospital, a 793-bed acute care hospital affiliated with Partners HealthCare. The PCTK is a core component of an institutional initiative and framework for care delivery—Promoting Respect and Ongoing Safety through Patient-centeredness, Engagement, Communication, and Technology (PROSPECT). PROSPECT is supported by the Gordon and Betty Moore Foundation and governed by a multidisciplinary steering committee consisting of researchers, clinicians, and key stakeholders including patient advocates. The goals of PROSPECT are to prevent adverse events, facilitate care plan concordance, reduce unnecessary healthcare resource utilization, and improve patient satisfaction and experience. A full description of PROSPECT and the PCTK ( figures 1 and 2 ) has been reported elsewhere. ^34–41

Figure 1:

Patient Tools. Patients and authorized caregivers can access the PCTK using a secure logon process that requires a personal email address and creation of a password at enrollment. We use structured coded data from the electronic health record and infobuttons to support tailoring of self-management education to patients’ diagnoses, problems, test results, and medications. Patients and caregivers can (1) navigate their plan of care (diagnoses/problems, care team goals, schedule); (2) establish recovery goals, input preferences, and rate priorities; (4) access their medications and test results; (5) review validated educational content; (6) identify care team members; (7) submit questions directly to their providers; (8) complete a validated checklist that informs providers of their discharge preparedness; and (9) view tailored safety tips and reminders (eg, fall prevention risk). All patient information (eg, names, clinical data, etc.) is hypothetical.

Figure 2:

Provider Tools—Plan of Care (2a) and Patient Thread (2b). Providers can access the PCTK from clinical applications typically used in the hospital. From the default “Plan of Care” tab ( figure 2 a), providers can quickly view the Patient Plan of Care summary page to (1) review health concerns, overall and daily goals, and care preferences inputted by patients; (2) update problems viewable by patients; (3) update/establish care team goals; and (4) update the patient’s schedule. Providers can identify current care team members (“Care Team”), message with other care team members (“Provider Thread”), and respond to questions and concerns submitted by the patient or authorized caregiver (“Patient Thread,” figure 2 b). The messaging tools are available to providers on mobile devices (iOS, Apple Inc, Cupertino, California; and Android, Google Inc, Mountain View, California). All patient information (eg, names, clinical data, etc.) is hypothetical.

Our study was approved by the Partners’ institutional review board. We implemented the PCTK in medical intensive care (MICU) and oncology units. All patients (and caregivers) admitted to intervention units for at least 24 hours were eligible to use the PCTK. We excluded patients admitted to a study unit for less than 24 hours.

Implementation and Enrollment

We initiated enrollment in July 2014. During the first 6 months after implementation, we refined, tested, and updated the PCTK based on feedback received from patients, caregivers, and providers. The majority of these updates included usability enhancements and bug fixes to improve stability and reliability of performance of key components discovered postimplementation. We completed our evaluation in January 2015.

Patient/Caregiver Enrollment

Research assistants enrolled patients over a 12-hour period during a typical 5-day workweek. They approached eligible patients within 24 hours of admission to an intervention unit. When an eligible patient was not capable of participating (eg, intubated, altered mental status, non-English speaker, blind, etc), a caregiver (a designated health care proxy) was identified and approached. Patients/caregivers were offered access to the PCTK on hospital-issued mobile devices (iPad Air; Apple, Inc, Cupertino, California). In order to enroll, they were required to provide an email address (username) and create a secure password. During a 30-minute enrollment session, patients/caregivers were trained to (1) access the PCTK; (2) update goals, preferences, and concerns; (3) view care team designated goals, problems, and schedule of events; (4) click infobuttons to access educational content; (5) navigate to educational content using infobuttons; and (6) send messages to the care team. We set the expectation that messaging tools were to be used for nonurgent communication of questions and concerns that arise over the course of the day after meeting with clinical staff, but not to convey acute clinical symptoms (eg, chest pain). Patients/caregivers were informed that they could expect a response within 24 hours, most likely in person or during goals of care meetings that typically occur for critically ill patients.

Teach-Back Sessions

Research assistants offered structured teach-back to all enrolled patients/caregivers starting in September 2014. Approximately 48 to 72 hours after enrollment, patients/caregivers were asked to demonstrate their ability to perform all activities taught during enrollment (see above). Patient/caregiver users were encouraged to provide suggestions for improvement via the feedback page. Research assistants recorded all feedback obtained verbally during enrollment and teach-back sessions.

Measurements and Data Collection

Usage of Patient Tools. We calculated the total number of enrollees who had entered an overall goal, daily goal, preference, or health concern at least once, as well as the total number of times each patient portal page was accessed by any enrollee. We measured the total number of messages sent to the care team per individual patient, the number of messages viewed by notified providers based on an analysis of read receipts, and the number of provider responses.

System Usability and Satisfaction Survey. During January 2015, all enrollees were asked to complete the 10-item System Usability Scale ⁴² and rate overall satisfaction on a 5-point Likert scale.

Statistical and Qualitative Analyses. Descriptive statistics were used to report patient demographic data, quantify usage and patient-inputted data (goals, preferences, concerns), and report usability and satisfaction survey data. The proportion of all patients (either patient or caregiver) enrolled by type of intervention unit (oncology vs MICU) was compared using the Fisher exact test (a 2-sided P -value < 0.05 was considered signficant). A similar comparison was performed for caregivers who enrolled on the patient’s behalf. The frequency of patient portal page visits by category (eg, goals, medications, care team, etc) was calculated from the total number of times each page was accessed by any enrollee. The content of messages sent by patients/caregivers was analyzed and grouped by themes using a 2-person consensus approach. Numerical ratings for each item of the System Usability Scale were summed, averaged over all participants, and reported on a scale of 0 to 100 (acceptable score > 70). ⁴² Satisfaction scores were dichotomized among participants who agreed or strongly agreed versus those who did not agree. All patient feedback collected was analyzed similar to message content.

RESULTS

Of the 496 MICU (mean [SD] age, 59 [17.7] years; 51% male; 64% Caucasian) and 380 oncology patients (mean [SD] age, 60 [13.8] years; 61% male; 84% Caucasian) eligible to participate, 220 (44%) and 71 (19%), respectively, were not approached for the following reasons: inability to identify a health care proxy when the patient was incapacitated (107), patient or research staff unavailable (87), inappropriate time or imminent demise (50), non-English speaking (34), and other (13). Of the 276 MICU patients approached, 103 (37%; mean [SD] age, 56 [15.6] years; 49% male; 76% Caucasian) were enrolled (26 patients, 77 caregivers). Of the 309 oncology patients approached, 136 (44%; mean [SD] age, 59 [13.9] years; 63% male; 89% Caucasian) were enrolled (93 patients, 43 caregivers). Three-hundred and forty-six patients (or caregivers) declined for the following reasons: leaving unit or being discharged (146), personal preference (139), and other (61). We observed a nonsignificant trend that favored greater enrollment of patients admitted to oncology compared with MICU units (44% vs 37%; P = .11). There were significantly more caregivers enrolled in MICU compared with oncology units (75% vs 32%; P < .01).

Of the 239 patient/caregivers, 200 (84%) and 39 (16%) used the PCTK for 1 to 4 days and 5 to 10 days, respectively, after enrollment. One hundred and fifty-eight (66%) inputted a daily goal; and 97 (41%) inputted an overall goal comprising the following: “be cured” 34 (35%), “improve/maintain health” 33 (34%), “live longer” 23 (24%), “be comfortable” 4 (4%), and “other” 3 (3%). Of the 239 patient/caregivers, 76 (32%) communicated preferences, 16 (7%) expressed new health concerns, and 153 (64%) provided real-time feedback regarding how well the care team was meeting patients’ goals. The most frequently visited patient portal pages ( figure 3 ) were goals (22%), test results (14%), care team members (9%), medications (9%), and messages (8%). The least frequently visited page was problem education (1%). One hundred and fifty-eight of 239 (66%) sent at least 1 message to providers (291 messages in total, ∼1.8 messages per patient/caregiver). Of the 291 messages sent by these patients/caregivers, 204 (70%) were viewed by at least 1 notified provider and 83 messages were sent in response. Four clinical and 3 nonclinical themes ( table 1 ) were identified in these messages (∼1.16 themes per message).

Figure 3:

Categories of Use The categories of patient portal pages are listed on the x-axis . The number of page visits per category is enumerated on the y-axis.

Table 1.

Major themes identified in messages sent by patients/caregivers

Category	No. (%) of Messages (n = 291)	Examples
Clinical Themes
Health concerns, needs, preferences, or questions/clarifications	55 (19%)	“How much fluid is still around my lungs and heart?”
		“Will my mother be seen by a cardiologist regarding her A fib?”
Report of clinical status/symptoms or request for clinical updates	33 (11%)	“Stood and walked with walker. His right leg is stronger today.”
Tests, medications, or schedules	32 (11%)	“What are my MRI results?”
Care coordination	17 (6%)	“Our daughter will be in approximately at 12:30 pm and would like to meet with [the oncologist].”
Nonclinical themes
Initial study communication	109 (37%)	“Hello, I’m an iPad a user.”
Feedback or nonmedical requests	76 (26%)	“Our nurse today is doing an amazing job of listening to us and coordinating a plan of care.”
Errors or corrections	15 (5%)	Correcting typos

Abbreviations: MRI, magnetic resonance imaging; Afib, atrial fibrillation.

^a iPad (Apple Inc, Cupertino, California).

Thirty-two enrollees (16 patients and 16 caregivers) were approached to complete the system usability and satisfaction survey. Eighteen participated (56% response rate), including 10 patients (70% aged > 51 years; 50% male; 80% Caucasian) and 8 caregivers (87.5% aged > 51 years; 25% male; 87.5% Caucasian). The mean (SD) system usability score was 74.0 (16.7). Thirteen of 18 (72%) were satisfied or extremely satisfied (mean [SD] score = 4.06 [0.94]). Key themes for improvement were derived from feedback received from 67 and 27 patients/caregivers during teach-back sessions and via the feedback page ( box 1 ).

Box 1.Key themes for improvement from analysis of patient/caregiver feedback

Provide a history and trend-view of all test results (not just the current day’s results)

Ensure that care team goals and problems are kept up-to-date

Ensure schedule of events is updated and includes timing of imaging studies

Display all types of test results including imaging studies and currently suppressed results

Highlight abnormal test results

Enable functionality to order meals and food online

Provide caregivers access from home and after discharge

Provide more entertainment and non–health-related content

Incorporate video conferencing functionality

Disable the lock-screen passwords

Improve care team display when there are many providers

Make the status of the discharge checklist viewable to providers

DISCUSSION

We observed a modest enrollment of eligible patients (or caregivers) and a trend towards enrollment of noncritically ill oncology patients compared with critically ill MICU patients. Fewer MICU patients were approached for enrollment (due to inability to identify a health care proxy). Nonetheless, we were relatively successful at enrolling patients using available caregivers in the critical care compared with noncritical care setting. Enrolled patients most often used the PCTK to establish goals, view test results and medications, and identify care team members. The volume of patient-initiated messages for clinically themed purposes was low; the messaging tool was used primarily to report health concerns, needs, or preferences. Use of educational content was highest for medications and test results, but infrequent for problem education. Finally, the average system usability score and satisfaction rating reported by a sample of users surveyed after the PCTK was optimized were favorable. From analysis of feedback, we outlined key barriers to adoption and strategies to promote future use ( table 2 ).

Table 2.

Barriers to adoption and strategies to promote use

Barriers to Adoption	Strategies to Promote Use
Not approached: patient lacked capacity, no health care proxy, non-English speaking, unavailable, etc. Declined: leaving the unit, personal preference (eg, prefer to use ambulatory patient portal), other (caregiver cannot access outside the hospital)	Offer acute care patient portal access from the ambulatory portal, providing a seamless experience (ie, a single enterprise patient portal). Encourage patients to designate a health care proxy and identify caregivers prior to hospitalization. Encourage patients/caregivers to enroll in the patient portal prior to hospitalization (eg, during ambulatory visits). Ensure electronic patient/caregiver contact information (eg, email) is available for registration.
Caregivers cannot access outside of hospital	Optimize web and mobile applications for access from home computers and personal mobile devices. Provide tools that allow patients to invite caregivers to access their portal. Offer a HIPAA-compliant mobile application for quick download.
Security requirements for patient portal access on hospital tablets	Optimize logon process by minimizing number of login screens and different passwords to remember. Allow a “bring-your-own device” strategy to minimize the need to encrypt, monitor, and track hospital-issued devices.
Technological and device issues	Ensure battery is fully charged and backup power/charger is available. Educate support staff on how to troubleshoot simple issues (eg, restarting device). Leverage mobile device APIs to enhance interactivity (eg, notification APIs).
Lack of clinician knowledge of patient portal tools and functionality	Promote awareness of key functionality, tools, and benefits for patients via multimedia broadcasts and campaigns. Real-time training. For example, create short videos uploaded to the web (eg, YouTube, San Bruno, California) and make them accessible from application or notification emails via hyperlinks.
Infrequent use during day	Train nursing staff to engage patients in using portal to establish goals, self-educate, and review test results and medications. Leverage device’s reminders and notification tools to facilitate engagement. Drive patients to use the device by taking away paper-based options (eg, food menu).
Information not up-to-date	Inform providers of the benefits of updating problems, care team goals, and schedule of events as a means of managing patients’ expectations and providing real-time education. Link patient schedules to administrative systems for key items (eg, imaging studies, procedures, etc).

Abbreviations: HIPPA, Health Insurance Portability and Accountability Act; API, application program interface.

Our relative success at enrolling critically ill or incapacitated patients can be explained by our proactive attempts at identifying “care partners” willing to use an online portal to participate in the patient’s plan of care. ³¹ The use of the PCTK to establish hospital goals is not surprising because patients and caregivers are directed to enter goals upon initial logon, and we reinforce this activity during teach-back. We attribute the lower-than-anticipated use of patient-initiated messaging to our efforts at setting expectations at the time of enrollment, lack of timely response from providers (though some responses were offered in person) and inability for caregivers to access the system outside the hospital. The minimal use of problem educational content is surprising; although we have implemented a new workflow to maintain and update active medical and nursing problems, patients perceive that this does not happen routinely. In contrast, patients’ use of medication and test result educational content is higher, and this content is maintained and synchronized with our electronic health record (EHR). Finally, we attribute the high degree of satisfaction by users to our ongoing collaboration with key institutional stakeholders and an iterative, participatory design and development process, both involving patient-advisors. ^35,39

As one of few reported attempts at using a patient portal to engage patients/caregivers in the acute care setting, we believe our study has important implications. ²⁵ First, acute care patient portals should be configured for access by caregivers so that they can participate in plan of care decision making when patients are either incapacitated or too ill, as what typically happens during early hospitalization. This is consistent with the emerging consensus to engage caregivers via online patient portals. ³¹ Second, offering patients a mechanism to establish and communicate goals directly to providers should improve concordance rates with providers and the execution of actions relevant to those goals. ¹⁴ In our preliminary analysis, the patient, nurse, and physician each selected the same overall hospital goal in just 20% of cases, suggesting much room for improvement. ⁴³ Third, although health care providers feared that patients and caregivers would send many messages and expect immediate responses via the patient-provider messaging tool, this was not the case—many messages were positive endorsements of care recevied. ³⁹ Fourth, in order for patients to effectively engage in self-management education via the patient portal, medical and nursing problems linked to educational content must be synchronized with the “source-of-truth” in the EHR, and this must be actively maintained and updated. In general, accuracy of EHR-problem lists tend to be subpar. ^44–47

Our study has several limitations. First, it was conducted at a single institution with a proprietary EHR; however, we are currently examining how to sustain innovative features through integration with our forthcoming vendor-based EHR (Epic Systems Corporation, Verona, Wisconsin). Second, although we identified surrogate decision makers as proxies in patients who were unconscious or otherwise incapacitated, there were many patients for whom we could not identify a proxy (eg, unavailable or inappropriate time) or whose proxy declined enrollment (eg, the PCTK was only available within the hospital and on hospital-issued devices). In the future, we plan to offer access outside the hospital and on personal mobile devices; this should substantially improve our ability to engage caregivers wherever they may be. Third, we did not assess efficacy of educational content or patient activation—both are important measures of patient engagement. Fourth, we did not include advanced health care planning tools for critical care and oncology patients, but this is an area of future interest. Finally, we restricted access to clinical content alone. According to the pyramidal model for acute care patient portals established by Prey et al, ²⁵ we could have expected greater and more sustained use if we had included access to entertainment options (eg, Internet access, medically-related multimedia content).

In summary, we have observed modest use of an acute care patient portal by patients and care partners as a platform for participating in the plan of care and communicating with providers. Although we have experienced challenges throughout implementation, we do not believe these are insurmountable. We think that such portals will become the norm as patients and care partners come to expect more interaction on a real-time basis. The lessons learned from our experience should be useful to any institution attempting to implement an acute care patient portal in a research or operational setting. Future studies should rigorously examine impact on clinical outcomes including patient/caregiver self-efficacy, care plan concordance, adverse events and preventable harm, resource utilization, and patient satisfaction.

CONTRIBUTORS

All authors have contributed sufficiently and meaningfully to the conception, design, and conduct of the study; data acquisition, analysis, and interpretation; and/or drafting, editing, and revising the manuscript. The secure messaging tools that were integrated into the web-based patient-centered toolkit were supported by CareThread, Inc., a business associate of Partners HealthCare, Inc.

FUNDING

This work was supported by a grant from the Gordon and Betty Moore Foundation (GBMF). GBMF had no role in the design or conduct of the study; collection, analysis, or interpretation of data; or preparation or review of the manuscript. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of GBMF. GBMF grant (GBMF3914).

COMPETING INTERESTS

None.