Abstract
Background
Patient portals are digital health tools adopted by health care organizations. The portals are generally connected to the electronic health record of the health care organization and offer patients functionalities such as access to the medical record, ability to order repeat prescriptions, make appointments, or message the health care provider. Patient portals may be beneficial for both patients and the health care system. Patient portals can widely differ from one context to another due to the differences in the portal functionalities and capabilities and it is anticipated that outcomes associated with the functionalities also differ. Current systematic reviews report outcomes associated with patient portal uptake but do not explicitly specify the patient portal functionalities.
Objective
The aim of this systematic review is to synthesize the evidence on health and health care quality outcomes associated with patient portal use among adult (18 years or older) patients. The review research questions are as follows: What kind of health outcomes do tethered patient portals and patient portal functionalities contribute to in adult patients (18 years or older)? and What kind of health care quality outcomes, including health care utilization outcomes, do tethered patient portals and patient portal functionalities contribute to in adult patients (18 years or older)?
Methods
The systematic review will be conducted by searching the MEDLINE, EMBASE, and Scopus databases for relevant literature. The review inclusion criteria will be studies about adult patients (18 years or older), studies only about tethered patient portals, and studies with or without a comparator. We will report patient portal–associated health and health care quality outcomes based on the patient portal functionalities. All quantitative primary study types will be included. Risk of bias of included studies will be assessed using the Cochrane Collaboration’s tool for assessing risk of bias in randomized trials and the National Heart, Lung, and Blood Institute’s quality assessment tools. Data will be synthesized using narrative synthesis and will be reported according to the patient portal functionalities, country, disease, and health care system model.
Results
Searches will be conducted in September 2019, and the review is anticipated to be completed by the end of June 2020.
Conclusions
This systematic review will provide an overview of health and health care quality outcomes associated with patient portal use among adult patients, providing detailed information about the functionalities of the portals and their associations with the outcomes. The review could potentially help patient portal evaluation studies by providing insights into outcomes associated with the different functionalities of patient portals.
Trial Registration
International Prospective Register of Systematic Reviews (PROSPERO) CRD42019141131; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=141131
International Registered Report Identifier (IRRID)
PRR1-10.2196/14975
Keywords: personal health record, patient portal, electronic health records, online access, patient records, systematic review
Introduction
Technology is affecting all aspects of health care systems. With the introduction of electronic health records (EHRs), patients are now able to access their medical records through patient portals, or personal health records (PHRs). Tethered patient portals or PHRs are connected to the EHR operated by the health care provider or organization and typically contain information about patient’s health records such as allergies, immunizations, medication, and upcoming appointments [1]. Some patient portals can also include information such as genetic data, preventative or customized medical advice [1], or offer functionalities such as ordering repeat prescriptions, messaging health care providers, and sharing health care record [2].
Patient portals are generally offered through the primary care providers, but can also be offered in hospital care, or during acute care [3]. The technology provides some benefits to patients and the health care system. For the health care system, portals may contribute to reducing phone call and visits [4], reducing emergency department visits [5], reducing hospital readmissions, improving the quality and efficiency of the health care system [6,7], and reducing health care service utilization in the long run and improving adherence to medical appointments [8]. For the patient, the use of patient portal may contribute to assisting in medical decision making [9], improving health care outcomes [6,10], improving adherence [11], improving patient- or person-centered care [4], improving patient satisfaction and increasing patient safety [12], and improving disease management and prevention [6,7]. In addition, patient portal functionalities such as viewing medical record may improve the relationship between patients and their providers [13]. Patient portals also make it possible to connect different emerging technologies such as wearable devices and mobile health (mHealth) technologies and collect the information in the patient record [14].
Although patient portals are associated with benefits to patients’ health and to the health care system, there are no definitive explanations about how they contribute to improved outcomes. A theoretical framework depicting how a patient portal access could improve health and health care quality experience is illustrated in Figure 1 using findings of qualitative studies. The benefits are often explained through the use of specific functionalities of the portal. For example, patients’ access to the health record with visit notes gives them ability to revisit their notes, which in turn improves their compliance with the care plan set out by the doctor [15,16] and also increases their confidence [17,18]. At the same time, access to the health record or test results gives the patients the chance to instantly and continuously review their record, increasing their knowledge about their condition [15] and improving their communication with the health care provider through increased ability to discuss their health care condition [15,16,18]. Other functionalities of patient portals such as e-messaging or secure messaging and patient education can improve patients’ involvement in their care [15] and their knowledge about their disease [19], respectively. Patients’ ability to refill medication through the portal allows them to efficiently refill medication, reducing the time they spend without medication, and as a consequence may improve their adherence and compliance with medication regime [17,20].
When reviewing patient portal studies it is important to consider possible sources of bias. Patient portal studies are often subject to bias due to the complex mechanisms involved between technology use and outcomes. Uptake usually varies between patients and often only registration on the portal is accounted for in studies. For example, while a meta-analysis reported a mean adoption rate of 51% (95% CI 42%-62%) from 40 studies about patient portals [21], a patient portal for laboratory results viewing with additional functionality of e-messaging received only 8.91% views out of the 208,635 tests released through the portal [22]. Second, registration or access to the portal does not guarantee use of the portal. A study among 301 patients with asthma reported that patients rarely used the patient portal on a regular basis and only about one-half used the portal in general [23]. In addition, studies cannot possibly take into account all of the interventions/factors other than the patient portal that may contribute to the patient outcomes [23].
Health care systems around the world are adopting patient portals [11,24,25]. In England, the National Health Service (NHS) made patient portals universally available in all general practices since 2015 [26]. The outcomes of the introduction of patient portals in England are yet to be examined, but before any outcomes can be expected, there needs to be an uptake of the technology by patients. Most published systematic reviews about patient portals focused on health outcome [11,24,25], patient experience outcomes such as patient engagement [27] or patient satisfaction [12], facilitators and barriers of use [21], and impact on quality [7]. However, most of these systematic reviews are now outdated due to the increased number of studies about patient portals in the last 1-2 years. Some reviews focused on patient portal adoption by a patient subgroup such as patients with diabetes [6,10], or cancer [2]. One meta-analysis reported patient portal adoption stratified by study setting (controlled versus real-world) but did not report patient portal functionalities of the included studies [21].
Although a number of published systematic reviews about patient portals are available, there are currently no reviews, that we know of, that focus on patient portal functionalities. When describing the outcomes, contextual factors such as the health care setting, patient type, and the functionalities of the patient portal should be considered as much as possible. Research reporting patient portal outcomes can provide valuable evidence if the context and the patient portal functionalities are clearly described and specified [6], because the context could contribute to the success of the technology. In addition, because patient portal adoption can generally be higher among patients with chronic diseases [28], the literature focused on studying the patient portal mostly by patient subpopulations such as those with diabetes or hypertension [17]. It is, therefore, essential to report user characteristics and contextual factors when considering patient portals.
There are a number of systematic review protocols of ongoing reviews about patient portals. Ammenwerth et al. [29] are planning to report patient portal outcomes based on the functionalities of the portal in which they will compare patient portals with access to the EHR alone with portals that have additional functionalities. However, the review will only include randomized controlled trials and cluster randomized controlled trials. Other upcoming reviews are examining patient portal user expectations [30]. An upcoming review by Petrovskaya et al. [31] plans to report patient and health care outcomes as a result of using patient portals among others; however, this will be an umbrella review and will only include other systematic reviews. Some other upcoming reviews will only focus on specific diseases such as diabetes, multiple sclerosis, and lower limb arthroplasty [32], or on one functionality of the patient portals (ie, having access to the medical record) [33], or will report only specific health care outcomes such as no-show appointments and emergency visits [34].
The aim of this systematic review is to synthesize the evidence on health and health care quality outcomes associated with patient portal use among adult (18 years or older) patients.
The review research questions are as follows:
What kind of health outcomes do tethered patient portals and patient portal functionalities contribute to in adult patients (18 years or older)?
What kind of health care quality outcomes, including health care utilization outcomes, do tethered patient portals and patient portal functionalities contribute to in adult patients (18 years or older)?
All research questions will be stratified by country, disease type, and health care system model.
Methods
Guidelines and Study Registration
This section will outline the methods of the systematic review using the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) guidelines [35] (Multimedia Appendix 1). This protocol is registered in PROSPERO, International Prospective Register of Systematic Reviews (registration number: CRD42019141131).
Inclusion Criteria
The review inclusion criteria with reference to participants, interventions, comparators, and outcomes (PICOS) framework [36] are reported in Table 1.
Table 1.
Characteristics | Criteria |
Population |
|
Intervention |
|
Comparator |
|
Outcomes | Primary outcomes:
|
Study types |
|
Exclusion Criteria
PHRs or patient portals that are not connected to the EHR.
The review will exclude patient portals that are designed only to deliver patient education or counselling.
Qualitative studies.
Information Sources and Search Strategy
The databases that will be searched include (1) MEDLINE (through Ovid), (2) EMBASE (through Ovid), and (3) Scopus. The search will have no restrictions or limits. The base search strategy was developed in the MEDLINE database (Multimedia Appendix 2) through multiple discussions with a medical librarian. The strategy will be modified and adjusted for each database according to the relevant keywords and subject headings in each database. Further studies will be identified through checking the references of eligible studies.
Study Records and Selection
We will use the Zotero (Roy Rosenzweig Center for History and New Media, USA) reference management software for removing duplicates and managing records. Studies will be scanned through the reading of the titles and abstracts in the first instance. Study title and abstracts will be assessed against the inclusion and exclusion criteria and studies that are clearly irrelevant will be excluded. Relevant studies and studies that did not provide enough detail in the abstract to judge their eligibility will then be assessed through full-text reading. Two members of the research team will independently perform title, abstract, and full-text screenings of the studies and discuss any discrepancies with a third reviewer. The study selection process will be recorded in a PRISMA flow diagram [35].
Data Extraction
Data extraction will be performed independently by two reviewers. A data extraction form was formulated to collect relevant data from the identified studies (Multimedia Appendix 3). The information collected in the data abstraction form will include (1) the last name of the author(s) and year of publication; (2) country of patient portal; (3) patient portal functionalities (the types of functionalities will not be defined prior to the review to avoid limiting the types that could be included. However, the functionalities, could include, but are not limited to functionalities mentioned in Figure 1, including accessing information through the portal, e-messaging, repeat prescription ordering, or appointment booking); (4) study type (controlled or real world); (5) patient characteristics, such as age group and sex; (6) any other patient characteristics specified in the study (such as patients with diabetes); (7) context-related factors (if any), such as type of health care setting (primary care or secondary care, or private or public health care); and (8) outcomes (health outcomes and health care quality outcomes will be recorded separately). Health outcomes could include any changes in disease indicators such as changes in blood pressure, plasma glucose concentration (hemoglobin A1c), or cholesterol levels. Health care quality outcomes could include any indicators of health care quality such as health care utilization rates, mortality rates, or disease-specific quality indicators. Study type is important because a previous meta-analysis found significant differences in adoption rates between controlled settings and real-world settings with patients being 10.8 times more likely to adopt the portals in controlled settings when compared with real-world settings (95% CI 3.2-36.3) [21].
Quality Appraisal
The risk of bias of randomized controlled trials will be assessed using the Cochrane Collaboration’s tool for assessing risk of bias in randomized trials [37]. Observational, cross-sectional, and quasi-experimental studies will be assessed using the National Heart, Lung, and Blood Institute (NHLBI) quality assessment tools for each study design [38]. The NHLBI tool helps assess the “internal validity of studies” and identify possible sources of bias [39]. The tool can be used to rate studies as good, fair, or poor in terms of risk of bias. Risk of bias assessment will be performed independently by two reviewers using the criteria from each of the risk of bias assessment tools depending on the type of study (Tables 1 and 2 in Multimedia Appendix 4). Summaries of risk of bias assessments will be presented as a table or a graph or in both formats for each study design. Results of the risk of bias assessment will be taken into account when interpreting findings from the studies; however, these results will not likely be used to exclude studies based on the risk of bias. These results will instead further help assess the confidence in outcomes concluded from the included studies.
Data Analysis and Synthesis
It is anticipated that performing a meta-analysis will be unlikely due to the heterogeneity of studies. Studies are likely to vary in terms of the methods used to collect and analyze the data, and in the outcomes reported. Alternatively, a narrative synthesis method will be used following guidelines suggested by Popay et al. [40] and the Cochrane Consumers and Communication Review Group (data synthesis and analysis document) [41]. The analysis will start by exploring the differences between and within the studies and identifying patterns in study outcomes. The relationship between studies, and gaps in the study findings will also be addressed in the synthesis.
Results
This systematic review is ongoing. The software searches started in September 2019. Data abstraction and data synthesis are expected to be completed by the end of June 2020. The review is anticipated to be completed by the end of June 2020. We are planning to disseminate the forthcoming systematic review in a peer-reviewed journal.
Discussion
Principal Study Findings
This systematic review will provide a comprehensive overview of the patient portal literature. To make it easy to compare between studies examining patient portals, we will categorize results by patient portal functionalities, disease, country, and health care system model. In the discussion section of the completed review, we will discuss implications from the studies, researcher assumptions, quality of the data, strengths and limitations of the review, and areas for future research.
Comparison With Previous Work
A common theme in current systematic reviews is the high heterogeneity between studies evaluating or assessing patient portals as well as the high heterogeneity between the functionalities offered by each portal [10]. Certain patient portal functionalities can have a higher association with improved health outcomes than others [6]. To control for the differences between the different portals, we will compare outcomes associated with portals based on the patient portal functionalities taking into account the disease or condition for which the portals are used for whenever possible. The outcomes of this review will inform a population-level analysis of patient portal use stratified by patient portal functionality, disease, country, and health care system model.
Limitations
It is a possibility that studies with low patient portal uptake do not report or publish their study findings. One way to deal with publication bias is by including both published and unpublished literature such as gray literature [42]. However, due to time constraints this review will only include published and peer-reviewed studies which subjects this review also to publication bias. The review will omit qualitative studies due to the drastic differences in methods and reporting of the results. However, this does not indicate that qualitative studies are not important to understand the outcomes related to patient portals. A systematic review focusing on qualitative studies reporting patient portal outcomes will help to further understand the mechanisms involved between patient portal use and health outcomes and can complement the findings of the forthcoming review.
Acknowledgments
We thank Ms Rebecca Jones (Medical Librarian) from the Imperial College Charing Cross Hospital Library for the fruitful discussions to refine the research questions and improve the search strategy. We also thank Imperial College Global Digital Health Unit members for their insights about the study protocol. This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Abbreviations
- EHR
electronic health record
- NHLBI
National Heart, Lung, and Blood Institute
- NHS
National Health Service
- PHR
personal health record
- PICOS
participants, interventions, comparators, and outcomes
- PRISMA-P
Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols
PRISMA-P 2015 Checklist.
Sample search strategy.
Sample data extraction form.
Sample risk of bias assessment forms.
Footnotes
Authors' Contributions: AA, GG, FG, and CC initiated the review design. SA and RHJ participated in the study design. AA drafted the manuscript. All authors read and approved the final manuscript. AA is the guarantor.
Conflicts of Interest: None declared.
References
- 1.Roehrs A, da CCA, Righi RDR, de OKSF. Personal Health Records: A Systematic Literature Review. J Med Internet Res. 2017 Jan 06;19(1):e13. doi: 10.2196/jmir.5876. http://www.jmir.org/2017/1/e13/ v19i1e13 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Aljabri D, Dumitrascu A, Burton MC, White L, Khan M, Xirasagar S, Horner R, Naessens J. Patient portal adoption and use by hospitalized cancer patients: a retrospective study of its impact on adverse events, utilization, and patient satisfaction. BMC Med Inform Decis Mak. 2018 Jul 27;18(1):70. doi: 10.1186/s12911-018-0644-4. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-018-0644-4 .10.1186/s12911-018-0644-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Sadasivaiah S, Lyles CR, Kiyoi S, Wong P, Ratanawongsa N. Disparities in Patient-Reported Interest in Web-Based Patient Portals: Survey at an Urban Academic Safety-Net Hospital. J Med Internet Res. 2019 Mar 26;21(3):e11421. doi: 10.2196/11421. http://www.jmir.org/2019/3/e11421/ v21i3e11421 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Rigby M, Georgiou A, Hyppönen H, Ammenwerth E, de KN, Magrabi F, Scott P. Patient Portals as a Means of Information and Communication Technology Support to Patient- Centric Care Coordination - the Missing Evidence and the Challenges of Evaluation. A joint contribution of IMIA WG EVAL and EFMI WG EVAL. Yearb Med Inform. 2015 Aug 13;10(1):148–59. doi: 10.15265/IY-2015-007. http://europepmc.org/abstract/MED/26123909 .me2015-007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Tsai R, Bell EJ, Woo H, Baldwin K, Pfeffer MA. How Patients Use a Patient Portal: An Institutional Case Study of Demographics and Usage Patterns. Appl Clin Inform. 2019 Jan;10(1):96–102. doi: 10.1055/s-0038-1677528. http://www.thieme-connect.com/DOI/DOI?10.1055/s-0038-1677528 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Coughlin SS, Williams LB, Hatzigeorgiou C. A systematic review of studies of web portals for patients with diabetes mellitus. Mhealth. 2017;3:23. doi: 10.21037/mhealth.2017.05.05. doi: 10.21037/mhealth.2017.05.05.mh-03-2017.05.05 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kruse CS, Bolton K, Freriks G. The effect of patient portals on quality outcomes and its implications to meaningful use: a systematic review. J Med Internet Res. 2015;17(2):e44. doi: 10.2196/jmir.3171. http://www.jmir.org/2015/2/e44/ v17i2e44 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Zhong X, Liang M, Sanchez R, Yu M, Budd PR, Sprague JL, Dewar MA. On the effect of electronic patient portal on primary care utilization and appointment adherence. BMC Med Inform Decis Mak. 2018 Dec 16;18(1):84. doi: 10.1186/s12911-018-0669-8. https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-018-0669-8 .10.1186/s12911-018-0669-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Jordan SR, Brungardt A, Phimphasone-Brady P, Lum HD. Patient Perspectives on Advance Care Planning via a Patient Portal. Am J Hosp Palliat Care. 2019 Feb 25;:1049909119832820. doi: 10.1177/1049909119832820. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Sun R, Korytkowski MT, Sereika SM, Saul MI, Li D, Burke LE. Patient Portal Use in Diabetes Management: Literature Review. JMIR Diabetes. 2018 Nov 06;3(4):e11199. doi: 10.2196/11199. http://diabetes.jmir.org/2018/4/e11199/ v3i4e11199 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Elers P, Nelson F. Improving healthcare through digital connection? Findings from a qualitative study about patient portals in New Zealand. Aust J Prim Health. 2018 Nov;24(5):404–408. doi: 10.1071/PY17116.PY17116 [DOI] [PubMed] [Google Scholar]
- 12.Mold F, de LS, Sheikh A, Majeed A, Wyatt JC, Quinn T, Cavill M, Franco C, Chauhan U, Blakey H, Kataria N, Arvanitis TN, Ellis B. Patients' online access to their electronic health records and linked online services: a systematic review in primary care. Br J Gen Pract. 2015 Mar;65(632):e141–51. doi: 10.3399/bjgp15X683941. http://bjgp.org/cgi/pmidlookup?view=long&pmid=25733435 .65/632/e141 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Legler A, Price M, Parikh M, Nebeker JR, Ward MC, Wedemeyer L, Pizer SD. Effect on VA Patient Satisfaction of Provider's Use of an Integrated Viewer of Multiple Electronic Health Records. J Gen Intern Med. 2019 Jan;34(1):132–136. doi: 10.1007/s11606-018-4708-z.10.1007/s11606-018-4708-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Paglialonga A, Patel AA, Pinto E, Mugambi D, Keshavjee K. The healthcare system perspective in mHealth. In: Andreoni G, Perego P, Frumento E, editors. mHealth Current and Future Applications. Cham: Springer International Publishing; 2019. pp. 127–142. [Google Scholar]
- 15.McAlearney AS, Sieck CJ, Gaughan A, Fareed N, Volney J, Huerta TR. Patients' Perceptions of Portal Use Across Care Settings: Qualitative Study. J Med Internet Res. 2019 Jun 06;21(6):e13126. doi: 10.2196/13126. https://www.jmir.org/2019/6/e13126/ v21i6e13126 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Dalal AK, Dykes P, Samal L, McNally K, Mlaver E, Yoon CS, Lipsitz SR, Bates DW. Potential of an Electronic Health Record-Integrated Patient Portal for Improving Care Plan Concordance during Acute Care. Appl Clin Inform. 2019 May;10(3):358–366. doi: 10.1055/s-0039-1688831. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Casillas A, Perez-Aguilar G, Abhat A, Gutierrez G, Olmos-Ochoa TT, Mendez C, Mahajan A, Brown A, Moreno G. Su salud a la mano (your health at hand): patient perceptions about a bilingual patient portal in the Los Angeles safety net. J Am Med Inform Assoc. 2019 Dec 01;26(12):1525–1535. doi: 10.1093/jamia/ocz115.5542899 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Reed ME, Huang J, Millman A, Graetz I, Hsu J, Brand R, Ballard DW, Grant R. Portal Use Among Patients With Chronic Conditions: Patient-reported Care Experiences. Med Care. 2019 Oct;57(10):809–814. doi: 10.1097/MLR.0000000000001178. [DOI] [PubMed] [Google Scholar]
- 19.Truong TM, Lipschultz E, Danahey K, Schierer E, Ratain MJ, O'Donnell PH. Assessment of Patient Knowledge and Perceptions of Pharmacogenomics Before and After Using a Mock Results Patient Web Portal. Clin Transl Sci. 2019 Sep 06; doi: 10.1111/cts.12681. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Hawkes JE, Mittal M, Davis M, Brixner D. Impact of Online Prescription Management Systems on Biologic Treatment Initiation. Adv Ther. 2019 Aug;36(8):2021–2033. doi: 10.1007/s12325-019-01000-w. http://europepmc.org/abstract/MED/31168763 .10.1007/s12325-019-01000-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Fraccaro P, Vigo M, Balatsoukas P, Buchan IE, Peek N, van DVSN. Patient Portal Adoption Rates: A Systematic Literature Review and Meta-Analysis. Stud Health Technol Inform. 2017;245:79–83. [PubMed] [Google Scholar]
- 22.Foster B, Krasowski MD. The Use of an Electronic Health Record Patient Portal to Access Diagnostic Test Results by Emergency Patients at an Academic Medical Center: Retrospective Study. J Med Internet Res. 2019 Jun 28;21(6):e13791. doi: 10.2196/13791. https://www.jmir.org/2019/6/e13791/ v21i6e13791 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Apter AJ, Bryant-Stephens T, Perez L, Morales KH, Howell JT, Mullen AN, Han X, Canales M, Rogers M, Klusaritz H, Localio AR. Patient Portal Usage and Outcomes Among Adult Patients with Uncontrolled Asthma. J Allergy Clin Immunol Pract. 2020 Mar;8(3):965–970.e4. doi: 10.1016/j.jaip.2019.09.034.S2213-2198(19)30862-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.de LS, Ross P, Shifrin M, Hercigonja-Szekeres M, Seroussi B. A comparison of approaches to providing patients access to summary care records across old and new europe: an exploration of facilitators and barriers to implementation. Stud Health Technol Inform. 2013;192:397–401. [PubMed] [Google Scholar]
- 25.Ammenwerth E, Schnell-Inderst P, Hoerbst A. The impact of electronic patient portals on patient care: a systematic review of controlled trials. J Med Internet Res. 2012;14(6):e162. doi: 10.2196/jmir.2238. http://www.jmir.org/2012/6/e162/ v14i6e162 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.NHS England About GP Online Services. 2019. [2019-04-04]. https://www.england.nhs.uk/gp-online-services/about-the-prog/
- 27.Irizarry T, DeVito DA, Curran CR. Patient Portals and Patient Engagement: A State of the Science Review. J Med Internet Res. 2015;17(6):e148. doi: 10.2196/jmir.4255. http://www.jmir.org/2015/6/e148/ v17i6e148 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Turner K, Hong Y, Yadav S, Huo J, Mainous AG. Patient portal utilization: before and after stage 2 electronic health record meaningful use. J Am Med Inform Assoc. 2019 Oct 01;26(10):960–967. doi: 10.1093/jamia/ocz030.5427963 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Ammenwerth E, Lannig S, Hörbst A, Muller G, Schnell-Inderst P. Adult patient access to electronic health records. Cochrane Database Syst Rev. 2017;2017(6):CD012707. doi: 10.1002/14651858.CD012707. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Riippa I, Linna M, Rönkkö I, Kröger V. Use of an electronic patient portal among the chronically ill: an observational study. J Med Internet Res. 2014 Dec 08;16(12):e275. doi: 10.2196/jmir.3722. http://www.jmir.org/2014/12/e275/ v16i12e275 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Petrovskaya O, Lau F, Antonio M. Synthesizing and translating evidence on patient portals: a protocol for an umbrella review with stakeholder engagement. PROSPERO 2018 CRD42018096657. 2018. [2019-04-25]. https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=96657 .
- 32.Bulbul A, Parkinson A, Desborough J, Smith J. A systematic review of patient attitudes towards the use of patient portals for the management of type 1 diabetes, multiple sclerosis and lower limb arthroplasty. PROSPERO 2019 CRD42019120942. 2019. [2019-04-25]. https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=120942 .
- 33.Tapuria A, Porat T, Kalra D, D'souza G, Curcin V. The impact of promoting patient access to their electronic health care record: a systematic review. PROSPERO 2019 CRD42019117426. 2019. [2019-04-25]. https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=117426 .
- 34.Avdagovska M, Olson K, Gramlich L, Vohra S, Ballermann M, Paul P. The impact of personal health records portals on no-show appointments and emergency room visits: a realist review protocol. PROSPERO 2017 CRD42017065027. 2017. [2019-04-25]. https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=65027 .
- 35.Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009 Jul 21;6(7):e1000097. doi: 10.1371/journal.pmed.1000097. http://dx.plos.org/10.1371/journal.pmed.1000097 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Centre for Reviews and Dissemination . CRD's Guidance for Undertaking Reviews in Health Care. England: Centre for Reviews and Dissemination, University of York; 2009. [2020-07-05]. https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf . [Google Scholar]
- 37.Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JAC, Cochrane BMG, Cochrane SMG. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. doi: 10.1136/bmj.d5928. http://europepmc.org/abstract/MED/22008217 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.National Heart, Lung, and Blood Institute (NHLBI) Study Quality Assessment Tools. Bethesda, MD: US Department of Health & Human Services; 2018. [2020-07-05]. https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools . [Google Scholar]
- 39.National Heart, Lung, and Blood Institute (NHLBI) Background: Development and Use of Study Quality Assessment Tools. Bethesda, MD: US Department of Health & Human Services; 2018. [2020-07-04]. https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools . [Google Scholar]
- 40.Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M. Guidance on the Conduct of Narrative Synthesis in Systematic Reviews: A Product from the ESRC Methods Programme. 2006. Jan 20, [2020-07-05]. https://www.lancaster.ac.uk/shm/research/nssr/research/dissemination/publications/NS_Synthesis_Guidance_v1.pdf .
- 41.Ryan R, Cochrane Consumers Communication Review Group Cochrane Consumers and Communication Review Group: Data Synthesis and Analysis. 2013. Jun 20, [2020-07-05]. https://www.lancaster.ac.uk/shm/research/nssr/research/dissemination/publications/NS_Synthesis_Guidance_v1.pdf .
- 42.Song F, Hooper. Loke Y. Publication bias: what is it? How do we measure it? How do we avoid it? OAJCT. 2013 Jul;2013:71–81. doi: 10.2147/OAJCT.S34419. [DOI] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
PRISMA-P 2015 Checklist.
Sample search strategy.
Sample data extraction form.
Sample risk of bias assessment forms.