How outcomes are achieved through patient portals: a realist review

Abstract

Objective

To examine how patient portals contribute to health service delivery and patient outcomes. The specific aims were to examine how outcomes are produced, and how variations in outcomes can be explained.

Methods

We used a realist review method, which aims to describe how ‘an intervention works, for whom, and in what circumstances’ by analyzing patterns between context, mechanism, and outcomes. We reviewed 32 evaluation studies of patient portals published since 2003.

Results

The reviewed evaluations indicate that as a complement to existing health services, patient portals can lead to improvements in clinical outcomes, patient behavior, and experiences. Four different mechanisms are reported to yield the reported outcome improvements. These are patient insight into personal health information, activation of information, interpersonal continuity of care, and service convenience. The vast majority of evaluations were conducted in integrated health service networks in the USA, and we detected no substantial variation in outcomes across these networks.

Discussion and conclusions

Patient portals may impact clinical outcomes and health service delivery through multiple mechanisms. Given the relative uniformity of evaluation contexts, we were not able to detect patterns in how patient portals work in different contexts. Nonetheless, it appears from the overwhelming proportion of patient portal evaluations coming from integrated health service networks, that these networks provide more fertile contexts for patient portals to be effective. To improve the understanding of how patient portals work, future evaluations of patient portals should capture information about mechanisms and context that influence their outcomes.

Background and significance

Health service organizations increasingly implement patient portals based on the belief that patient portals will enhance patient engagement, health outcomes, service efficiency, and convenience.¹ A patient portal is a ‘secure website for patients, typically maintained by provider practices, that offers access to a variety of functions linked to a physician's EHR [electronic health record] including secure messaging, protected health information (eg, lab results, medication lists, and immunizations), appointment scheduling, and tethered PHRs [personal health records]; more advanced portals may offer programs for self-management or patient questionnaires’.¹ In 2012, Ammenwerth et al² published a meta-analysis of the impact of patient portals, identifying six types of outcomes or processes to which patient portals can contribute. These include clinical outcomes, health resource consumption, patient adherence, patient–provider communication, patient empowerment, and patient satisfaction. Although providing a synthesis of the contribution of patient portals to these six outcomes and processes, the review method used by Ammenwerth et al does not explain how patient portals contribute to these outcomes and processes, nor does it explain why some patient portals are successful in doing so while others fail. The variation in outcomes reported by Ammenwerth et al, as well as by others, calls for a scientific analysis to provide explanations for variation in patient portal outcomes. This study seeks to provide such explanations using a so-called realist review method.

A realist review seeks to describe how ‘an intervention works, for whom, and in what circumstances’.³ It does so by analyzing the relationships between context, mechanism, and outcomes.⁴ In adhering to this terminology, we use the word outcome to refer to the six outcome and process measures defined by Ammenwerth.² Mechanisms are the often hidden workings of the intervention that cause outcomes.⁵ Context refers to the conditions in the environment where the intervention works that activate the mechanisms.⁵ Contrary to more traditional literature synthesis methods, the realist approach does not aim to assess the outcomes of interventions, but rather to explain them. This aim allows for including diverse study designs, since each may help explain how the intervention works.³ A realist approach unravels the workings of complex interventions in their specific contexts.⁶ As such, the approach has recently been used on complex and diverse interventions such as joint health and safety committees,⁷ school feeding programs,⁸ culturally appropriate diabetes education programs,⁹ and internet-based medical education.¹⁰ Patient portals are also complex interventions and often continuously adapted to meet the needs of the users and the healthcare organizations involved.¹¹ Given such idiosyncratic conditions, the effects of a patient portal may not be replicated from one context to another.¹² As the number of patient portal implementations increases across a variety of settings, understanding the mechanisms that explain the results achieved in different contexts gains importance.

Objective

Based on the realist line of thinking, the review aims to synthesize and analyze evaluations of patient portals to explain the reported outcomes. Our two main research questions are:

1. By what mechanism(s) do patient portals contribute to outcomes?

2. How can variations in outcomes across different contexts be explained?

Materials and methods

We adhered to the realist review method described in the RAMESES publication standards, published in 2013.⁴ The reporting of our methods and results follows these standards.

Exploratory review of how patient portals work

We started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes.^{1 2 13–21} Based on this review, we identified six main ways in which patient portals may affect service delivery and outcomes. These include improving: patient access to information and services; patient decision-support; coordination of care around the patient; interpersonal continuity of care; health services efficiency; and service convenience to patients and caregivers. The aim of a realist review is to test and refine such ‘educated guesses’ against the data in the evaluation studies included in the realist review.¹⁰ In the following we describe how these studies were selected.

Searching process

We searched PubMed, LISTA, PsycINFO, and Scopus for peer-reviewed literature in English published between January 2003 and August 2013 (see table 1 for search queries). Assisted by a librarian, we searched for literature on both patient portals and electronic personal health records (PHRs). We included both these terms, since they are so related that literature often addresses them interchangeably. Given the definition of patient portals presented in the introduction, both entities are relevant to our review: when logged into a patient portal, patients may see their PHR, which stores data from information exchanges (such as secure messaging) made available via the portal.

Table 1.

Search queries

Search terms in title/abstract	Restrictions	Hits from combined search with all terms
Patient portal* Patient web portal* Patient online portal* Patient internet portal* Personal health record (PHR)* AND electronic PHR* AND online PHR AND electronic PHR AND online Shared electronic medical record* Electronic PHR*	Time period: 2003–2013 Language: English Type of publication: Peer-reviewed	PubMed: 150
		PsycINFO: 53
		LISTA: 48
		Scopus: 279
		Total: 530

The asterisk (*) after a search term indicates that we searched for variations of the truncated term. In the displayed search queries, that enabled us to capture both the singular and the plural form of a term, eg ‘patient portal’ and ‘patient portals’.

Selection and appraisal of documents

Two reviewers selected the articles to be included in the review through a two-step process. First, each article's title and abstract were reviewed, and articles were excluded if they did not present evaluations of electronic health record (EHR)-linked patient portals. The primary reviewer (TOT) reviewed all articles, while the second reviewer (AdB) reviewed a random sample of 10%. The agreement rate was 97.4%, and disagreement was resolved through discussion. In the second round, we read the full text to determine whether our extended set of inclusion criteria was met. Building on Ammenwerth et al's review, we included only evaluations addressing the six outcome or process categories classified in that review. In addition, we based our appraisal on whether the studies identified mechanisms by which the interventions were expected to work. The second reviewer received a random sample of 16%, and the agreement rate between reviewers about whether the inclusion criteria were met was 86%. Agreement was reached through discussion and 25 articles were selected. Through snowballing, an additional seven studies were identified, bringing the number to 32.

Data extraction

We extracted data from the articles and created six data tables, one for each outcome category. The data used to populate the cells included: article information; study objective; intervention description; patient portal characteristics; outcomes; contextual factors; proposed mechanisms; evaluation characteristics; and finally, study characteristics. For the studies that reported multiple outcomes, we posted relevant information into multiple outcome tables. However, we did not include information regarding second or tertiary outcomes that were only superficially reported, thus disregarding mechanisms linked to weaker or coincidental outcomes.

Analysis and synthesis processes

For each outcome category, we organized information to bring forward by what mechanism patient portals contribute to outcomes and the variation in outcomes across different contexts.

First, we identified the outcomes reported in each study, and organized outcome data according to intervention type and study design (randomized controlled trials (RCTs), observational, and qualitative). The organization of data in terms of intervention type was especially important, since we included evaluations of an entire patient portal, a patient portal module, or a program offered through a module. We reasoned that all interventions provide valuable information on the contribution of a patient portal, but were aware that the outcomes should be compared with caution. With respect to study design, we classified the strength of the patient portals’ effect on each outcome (see online supplementary appendix 1). For the RCTs, we separated the reported effects into studies reporting no statistically significant effect; those with some significant effect, but lacking consistency across different outcome measures and/or population subgroups; those with a significant effect not sustained over time; and finally those with significant effects across multiple outcome measures and sustained over time.

Second, for each outcome category, we identified the mechanisms mentioned by the authors. Most of the studies did not describe in detail the mechanisms believed to be in play, and careful reading of the text was required to identify the mechanisms proposed to link the patient portal intervention to an outcome. Moreover, we assessed whether these mechanisms were empirically tested; that is, whether the key concepts of the mechanism were operationalized in the studies.

Finally, we gathered information on the organizational and healthcare system context in which the evaluated patient portals operated. Unfortunately, the influence of context on the intervention was rarely described in detail at the service unit level (culture, working relationships, operational policies and procedures, incentive systems) or the personal patient–provider level (guidelines, communication, relationship). Most studies contained some large-scale information, such as the type of organization, the number of providers and patients using the patient portal, the comprehensiveness of the EHR, and existing disease management efforts within the organization. User characteristics such as age distribution, ethnicity, language proficiency, and gender distribution were well described in the studies.

Results

Document flow diagram

Figure 1 illustrates the paper selection flow that led to a total of 32 evaluation studies.

Figure 1.

Paper flow diagram.

Document characteristics

Eleven studies evaluated the effect of a patient portal, portal module, or program offered through a patient portal on clinical outcomes. Eight studies examined the impact on healthcare utilization, and six looked into the effect on patient adherence. Patient–provider communication was an end-point in five evaluations, patient empowerment in eight, and patient satisfaction in five. Eighteen of the studies were RCTs, 11 were observational studies, three were qualitative studies using focus groups or interviews, and one used mixed methods (see online supplementary appendix 1). Thirty studies were from the USA, one was from Canada, and one from the Netherlands. Eleven articles were published before 2010 and 21 after 2010, indicating a recent and steady increase in the evaluations of patient portals and electronic PHRs. (See online supplementary appendix 2 for tables with article information and excerpts from the data about study objective, methods, context, mechanisms, and outcomes.)

Main findings

Outcomes

The evaluations indicated that patient portals could lead to improvements in clinical outcomes, patient adherence, patient–provider communication, patient empowerment, and patient satisfaction with health services. In total, 15 of the 18 RCTs demonstrated significant positive effects on these outcomes, albeit in some studies the effects were not strong or consistent over time. The majority of observational and qualitative studies provided indications consistent with these demonstrated effects. No studies found serious adverse consequences of patient portal implementations. However, interestingly, an often-hypothesized potential of patient portals to lower health resource consumption through substitution of services was not validated. On the contrary, of the eight studies that addressed healthcare utilization, five concluded that higher health resource consumption occurred after the implementation of a patient portal, and two found no change. Only one of the eight studies documented lower healthcare utilization due to a patient portal.²² However, this finding has been contradicted by a more recent study conducted within the same integrated delivery system.²³ These results suggest that patient portals became complements rather than substitutes to existing health services.

More than half of the patient portal evaluations were targeted at chronic disease patients. This was particularly the case in the clinical outcome category, where all 11 evaluations focused on management of diabetes, hypertension, depression, chronic musculoskeletal pain, or mobility difficulty. This focus on chronic conditions is not surprising since several studies mentioned that the patient portal supported comprehensive self-disease management programs.^24–29 Interestingly, several studies mentioned a ‘ceiling effect’ due to the breadth and quality of these existing disease management programs, which could explain why their results were often not strong or consistent. As one study concludes: ‘As control approaches the upper end of feasibility, further improvements may be limited’.²⁴

Furthermore, several evaluations suffered from inadequate study samples, relatively short follow-up periods, and used different instruments to assess outcomes. These methodological problems may also have inhibited the strength of results and suggest that caution be taken in generalizing the findings from the studies. However, although the outcomes are important to understanding patient portals’ contributions to healthcare systems and the patients they serve, as noted earlier, assessments and comparisons of outcomes are not key to realist reviews. Rather, the focus is on explaining how and why the identified outcomes occur. In the sections below, we attempt to provide answers to those questions.

Mechanisms

By classifying and aggregating authors’ hypothesized mechanisms by which patient portals affect outcomes, we identified four such mechanisms. We recognized a mechanism if it was mentioned in at least half of the evaluations within a certain outcome category. Importantly, it should be noted that none of the studies described their hypothesized mechanisms in detail, nor did they empirically test whether these mechanisms did in fact lead to the reported outcomes. Further, as a consequence of the research designs used in the respective studies, data were not collected on the actual operationalization of these mechanisms. Hence, the mechanisms we identified are based are solely on researchers’ hypotheses about the patient portal workings, provided that these were not counter to the corresponding study outcomes. There were no correlations across mechanisms, study designs, or intervention type (whole portal, module, or a program). Below, we describe the four identified mechanisms and provide examples from the data.

Worth noting is that the four identified mechanisms do not include care coordination or provider efficiency, which we had identified as possible mechanisms in our exploratory review.

1. Patient insight into information: Several studies highlighted that having access to personal information will enable and motivate patients and their caregivers to be involved in its application and in ensuring its accuracy and comprehensiveness. This mechanism was mentioned in more than 50% of evaluations of patient empowerment, clinical outcomes, and patient adherence.

   Patient empowerment: ‘Making electronic health records available across the Internet is viewed as an important step toward consumer empowerment, because without adequate information patients are not able to achieve sufficient levels of desired autonomy and self-efficacy’.³⁰

   Clinical outcomes: ‘Access to effective and tailored patient education, electronic patient–provider communication, and tailored patient education, electronic patient–provider communication, and the wealth of clinical information and web-based resources contained within modern PHRs could lead to improvements in chronic disease outcomes through improved patient-centered care and self-management’.³¹

   Patient adherence: ‘The use of a secure, interactive personal health record (PHR) tethered to an EHR can provide an avenue for patients to review and update health information and has the potential to improve adherence to guidelines’.³²

2. Activation of information: Several authors described how decision-support tools provide new and effective ways of using and presenting information. Primarily reminders sent to patients through patient portals were reported to increase the effectiveness and targetability of information. This mechanism was mentioned as being important for achieving patient adherence in more than half of the evaluations.

   Patient adherence: ‘To act on their choices, patients need written plans and logistical details. They need reminders when services are due, guidance to deal with inconsistent recommendations, and access to decision aids for choices that require shared decision-making’.³³

3. Interpersonal continuity of care: Easier and improved access for patients to contact their providers was proposed to enhance interpersonal continuity of care. As several studies mentioned, patient portals allow patients to communicate asynchronously with a preferred provider, enabling them to build an ongoing, personal relationship that includes mutual trust and responsibility.³⁴ Interpersonal continuity of care was suggested to improve clinical outcomes and patient satisfaction in some studies, whereas patient–provider communication alone was linked to interpersonal continuity of care in more than 50% of the studies.

   Patient–provider communication: ‘Our examples illustrate the cases in which patients and providers establish social bonds during the interactions facilitated by the patient portal system. As well, accumulated messages in the portal system about the same patient can provide rich trajectory information that help providers and the patient better understand her illness management from a long-term perspective’.³⁵

4. Service convenience: Finally, patient portal services that ease the navigation of the health system, facilitate contact, and decrease patient costs, were believed to bring added service convenience to patients. This mechanism was hypothesized in more than 50% of the studies to impact health resource consumption and patient satisfaction by making it easier to acquire services.

   Health resource consumption: ‘Internet portals may improve patient health and well-being by providing reliable and trusted MS [multiple sclerosis]-related information and resources, providing easy and reliable methods for patients to navigate an increasingly complex medical healthcare system, and providing a secure avenue for patients to communicate electronically with their MS provider regarding symptoms and disease management’.³⁶

   Patient satisfaction: ‘The portal was convenient: 81% believed that the portal saved them a telephone call, and 33% believed it saved them a visit to the clinic. The portal allowed patients to send messages at all hours; indeed, 73% of incoming messages were sent during non clinic hours’.³⁷

Authors regularly reported socio-demographic differences in achieving the outcomes. One study, for example, noted that ‘users were demographically different from nonusers (eg, fewer minorities and higher education), had more chronic illnesses, and were more up-to-date with care at baseline’.³³ This may suggest a refinement to some or all of the proposed mechanisms. The refinement could point to variations in how mechanisms are triggered in individuals based on their race/ethnicity, geographical location, health and online literacy, and health consciousness.

As indicated above, the hypothesized significance of these mechanisms differs per outcome. This is illustrated in figure 2. The arrows signify that a least 50% (thin arrows) and 75% (thick arrows) of authors hypothesized that a particular mechanism was in play to produce the expected or observed outcome.

Figure 2.

Hypothesized links between mechanisms and outcomes. The thin arrows signify that at least 50% of authors hypothesized that a given mechanism produced a given outcome. The thick arrows indicate that this was the case for at least 75% of authors.

Context

The vast majority of evaluated patient portals included in our review operated within health service networks in the USA. These health service networks were integrated delivery networks, academic hospitals providing integrated care, and multispecialty group practices. Although some of these were indeed truly integrated delivery networks, we refer to the three types of networks as ‘organized’. (See online supplementary appendix 3 for a list of these networks.) Only two patient portals were from outside the USA (Netherlands and Canada) and both distinguished themselves by operating independently within a hospital clinic or a hospital network.

The organized health service networks appeared to share some large-scale contextual characteristics, including the presence of comprehensively used EHRs and a focus on chronic disease management. The influence of the network configuration on patient portals received some attention in the material, as several research teams noted the impact of integration on the outcomes. One team remarked: ‘this study was conducted in an integrated health care system with shared records of medical and mental health care as well as established collaborative relationships between primary care and mental health providers’. Moreover, ‘the efficiency and clinical benefit of this program might be difficult to replicate outside of an integrated system’.³⁸

Context–mechanism–outcomes patterns

From a realist perspective, patterns between context, mechanisms, and outcomes of an intervention are central to understanding how and why the intervention had an effect. Based on such patterns, reviewers can explore the impact of the particular context on the mechanisms that were suggested to produce the outcomes. We did not identify context–mechanism–outcome patterns based on the reviewed literature, as we were unable to detect consistent differences regarding the outcomes or proposed mechanisms across contexts. Nor did study design, intervention type, or evaluation methods differ notably across contexts.

Discussion

Summary of findings

Using a realist lens, we set out to answer (1) by what mechanism(s) patient portals contribute to outcomes, and (2) how variations in outcome across different contexts can be explained.

To the first question, although not empirically tested, patient portal evaluators suggest at least four mechanisms to influence clinical outcomes, health service utilization, patient adherence, patient–provider communication, patient empowerment, and satisfaction. This supports our premise that patient portals are complex interventions that work through multiple pathways to generate multiple outcomes. The most frequently reported mechanisms were patient insight into information and interpersonal continuity of care. Seventy-five per cent of authors hypothesized that patient insight was in play to enhance patient empowerment. Similarly, 75% of authors hypothesized that interpersonal continuity of care was a mechanism that explains how patient portals improve patient–provider communication.

Lack of variability in outcomes and mechanisms across relatively uniform contexts hinders a response to our second question. We give three possible explanations for this lack of detectable context–mechanism–outcome patterns in the included studies. First, as indicated above, conditions for patient portals in organized health service networks may be relatively similar, and consequently, the uniformity of evaluation contexts may have inhibited significant differences in mechanisms and outcomes. Second, the effect of a patient portal on outcomes may be limited because of a ceiling effect produced by existing disease management programs. Third, the study designs and evaluation methods did not allow for surfacing measurable differences, especially if these would be marginal due to the ceiling effect.

Nevertheless, the striking tendency of evaluated patient portals to operate within organized health service networks can provide some insights into the contextual characteristics conducive to patient portals. Large and organized health service networks with shared EHRs are well equipped to make the investment to establish high-functional patient portals that integrate information from the continuum of care.^{39 40} The probability that this investment will be returned is high since their large scale facilitates patient traffic, which is necessary to generate outcomes.^{41 42} Moreover, the business care for patient portals remains strong in organized networks, even if costs and benefits appear in different network components. Likewise, organized health service networks suffer less from internal operational barriers which hinder reaping joint benefits as attainable through the higher levels of collaboration and communication enabled by patient portals.^43–45 These attributes are likely to support a culture with a focus on quality improvement and patient-centered care, which may positively affect the application of patient portals across the care continuum.³⁹ In addition, existing research and quality improvement traditions incite and permit scientific evaluations of these technologies.³⁹

This examination of the impact of organizational aspects on patient portals prompts two reflections. First, the modest outcomes produced by most of the studies may be explained by the fact that the organized health service networks already provide well established patient-centered health services. As many of the reviewed studies noted, due to existing disease management and patient-engagement programs, the effect of adding a patient portal was only incremental. Furthermore, the high degree of interorganizational coordination found in organized health service networks could explain why the potential of patient portals to improve care coordination did not surface as an important mechanism in the reviewed studies. As care coordination commonly underlies the existing disease management programs, evaluators may have overlooked care coordination as a relevant contextual factor.

Second, the above-mentioned ceiling effect may not apply to fragmented contexts, which may therefore derive higher value from patient portals.⁴⁶ In fragmented contexts, patient portals may become a means to achieving a discontinuous improvement, for example towards care coordination, and thus to generating desired effects.⁴⁷ However, it appears from the lack of reported outcomes from fragmented healthcare systems that these systems are less conducive to achieving such improvements.⁴⁸ Patient portals seem to struggle in contexts that need them most.

Strengths and limitations

We found that the studies rarely detail the mechanisms by which an intervention is expected to work. Consequently, the studies are not designed to empirically test the mechanisms that could explain how patient portals improve outcomes. Combined with a scarcity of small-scale contextual information, individual studies tell us little of how and why patient portals create outcomes in different contexts. Thus, our ability to identify and aggregate the proposed mechanisms underlying each intervention is an important step in establishing the evidence base for the implementation of patient portals, and a major contribution of this realist review.

There are a number of limitations to the study. First, we only included evaluations with the six process and outcome measures identified by Ammenwerth et al, and may thus have excluded evaluations of other end-points. The lack of cost evaluations could, for example, explain why provider efficiency and productivity was not mentioned as an important mechanism in the reviewed studies. Second, we acknowledge a possible publication bias towards successful implementations, potentially having led to an omission of information regarding patient portal implementations from less effective contexts.⁴⁹

Future research directions

The reviewed evaluations are likely to be forerunners to a larger body of evaluations that may confirm our preliminary results. Future evaluations should describe the small- and large-scale contexts impacting the intervention to make apparent why an intervention may or may not have worked. Furthermore, our understanding of how and why patient portals work will benefit from more attention to the proposed mechanisms underlying patient portal interventions; for example, by empirically measuring whether key concepts to an given intervention mechanism have actually been operationalized as intended.

This review has indicated that patient portals may have even higher value propositions in more fragmented contexts, but that these contexts may be less favorable to patient portals. Therefore, a topic that deserves further exploration is how to implement and derive outcomes from patient portals in healthcare systems that are more fragmented than the ones captured in this review.

Conclusion

Patient portals affect clinical outcomes, health service utilization, patient adherence, patient–provider communication, patient empowerment, and patient satisfaction with health services by four mechanisms. These mechanisms are: patient insight into information, activation of information, interpersonal continuity of care, and service convenience. The significance of these mechanisms differs per outcome. Reported outcomes of patient portals derive mostly from large and organized health service networks. In highly organized health service networks, patient portals appear to be complements to disease management programs rather than substitutes for these services. Paradoxically, patient portals may have higher impact in more fragmented contexts that are less conducive to patient portal implementation and use.

The complexity of deriving outcomes from patient portals emphasizes the necessity of research that disentangles the mechanisms by which outcomes are produced in relation to their context. Research designs and evaluations reported to date are insufficient for this purpose. Moreover, reports on unsuccessful patient portals, which are equally important for such disentanglement, are lacking.