Abstract
Purpose:
Electronic portals provide patients with real-time access to personal health records. Use of this technology by individuals with cancer is particularly intensive. We therefore examined patterns of use of electronic portals by clinic staff at a National Cancer Institute–designated comprehensive cancer center.
Methods:
We identified and characterized cancer center providers and clinic staff who performed electronic activities related to MyChart, the institution’s personal health records portal, from 2009 to 2014. Total MyChart actions and messages received were quantified and characterized according to type, timing, and staff category.
Results:
Two hundred eighty-nine employees were included in our analysis: 85 nurses (29%), 79 ancillary staff (27%), 49 clerical/managerial staff (17%), 47 physicians (16%), and 29 advanced practice providers (10%). These individuals performed 740,613 MyChart actions and received 117,799 messages. Seventy-seven percent of actions were performed by nurses, 11% by ancillary staff, 6% by advanced practice providers, 5% by physicians, and 1% by clerical/managerial staff. From 2011 to 2014, staff MyChart activity increased approximately 10-fold. On average, 6.3 staff MyChart actions were performed per patient-initiated message. In 2014, nurses performed an average of 3,838 MyChart actions and received an average of 589 messages, compared with 591 actions and 87 messages in 2011 (P < .001). Sixteen percent of all actions occurred outside clinic hours.
Conclusion:
Cancer center employee effort related to an electronic patient portal has increased markedly over time, particularly among nursing staff. Because further uptake of this technology is expected, it is critical to consider potential effects on clinical resources, employee and patient satisfaction, and patient safety.
INTRODUCTION
Electronic patient health portals are clinician-tethered, internet applications that provide a direct and secure means for patients to receive and convey information relevant to their health care. Common functions include real-time access to personal medical records (ie, medical histories, medications, and laboratory and radiology results), disease-specific educational resources, and patient-to-provider electronic messaging.1,2 In conjunction with broader expansion of electronic medical records, the prevalence and use of these tools are increasing in the United States.3,4 Increased transparency of care, heightened patient engagement in medical decision making, and strengthened doctor-patient relationships are established goals of patient health portal implementation.1,5-8 Furthermore, patient health portals have been shown to improve communication, patient satisfaction, and adherence to medical treatments.2,5,7-9
MyChart is an electronic patient health portal used by multiple major medical centers. It provides a secure, online portal for patients to access their personal health record within the Epic electronic medical record (Verona, WI). MyChart became available at the University of Texas Southwestern Medical Center (UT Southwestern) through some clinical departments as early as 2005 and was implemented in the Harold C. Simmons Comprehensive Cancer Center in 2009. Since that time, all patients at the Simmons Cancer Center have been offered MyChart access at the time of registration.
Considering the critical disease nature and intensity of oncology practice, understanding the use of portal technology by individuals with cancer and their health care providers is central to the goals of patient satisfaction and safety. We previously reported predictors and intensity of MyChart use by a cohort of almost 6,500 patients with cancer at our center. We found that use increased markedly over a recent 5-year period.10 Compared with nononcology populations, these patients used the electronic portal four to 10 times more frequently.10-12
How this growth in electronic communication has affected oncology practices is not clear. Within the field of oncology, use of such technology raises particular concerns. First, patient viewing of laboratory and radiology results related to disease status outside the context of a provider visit could result in anxiety and misunderstanding.13 Separately, given the aggressive nature of many malignancies, clinical updates sent by patients electronically may be more likely to represent medical urgencies than in other fields. Although health care providers recognize the potential benefit of real-time patient access to medical information, some also express substantive concern regarding safety and added workload associated with electronic messaging.14-17 Already, concerns have emerged regarding the growing proportion of time clinicians devote to electronic medical record data and order entry, which in some fields seems to well exceed that spent in direct patient contact.18 Given these considerations, we determined the intensity and patterns of patient health portal use by providers and staff at a National Cancer Institute (NCI) –designated comprehensive cancer center.
METHODS
Study Population
This study was approved by the Simmons Cancer Center Protocol Review and Monitoring Committee and the UT Southwestern Institutional Review Board (STU 062012-025). Using human resources data records, our team identified clinic staff and providers used by the Simmons Cancer Center between 2009 and 2014. Each employee was identified by a unique Epic username (which also served as the MyChart identifier). Professional titles were categorized as follows: physicians (faculty, fellows, and residents), nurses, advanced practice providers (nurse practitioners and physician assistants), clerical/managerial staff, and ancillary staff. Oncology clinic types were categorized as medical oncology, surgical oncology, and radiation oncology. User professional titles were defined at the time of data extraction (December 2015). Thus, the analysis did not account for changes in professional role or clinic type over the duration of this study. We excluded staff not directly affiliated with the cancer center (ie, radiology technologists) and unidentified users.
Data Abstraction and Analysis
Physicians, advanced practice providers, nurses, ancillary staff, and clinic administrators interface with MyChart using personal Epic accounts. Patient-to-clinic, clinic-to-patient, and provider/staff-to-provider/staff messages populate an in basket. Users may perform a number of electronic actions in relation to these messages, which we categorized as the following: message review, postpone/pend message, patient chart review, encounter creation (ie, appointment creation/cancellation, medication refill, and telephone call), message/encounter completion, new message creation (ie, forward, reply, and new message), and other. Message and action threads are dynamic. For instance, if a nurse or advanced practice provider receives a patient communication that requires physician attention, messages may be sent between these individuals while remaining electronically linked to the original patient message. Thus, it is possible for multiple users to perform multiple actions in reference to a single message.
Using MyChart data table Eow_command_audit, we extracted MyChart actions (electronic functions) performed and messages received by the Simmons Cancer Center employee cohort between 2009 and 2014. This data table was selected because it provided the most detailed and complete collection of user actions. Employees who did not demonstrate at least one user action or received message within this table were excluded from the cohort.
We aggregated total actions performed in relation to received messages. Messages received from patients were quantified. Because a single patient-initiated message could be viewed by multiple employees, the first individual to open the message was considered the primary recipient and was used in message-oriented analysis. We adopted this approach to prevent over-representation of the true number of messages sent to the clinic. We then characterized actions and messages by user professional title, date, time of day, and day of week. Because of limitations inherent to the Eow_command_audit data table, we were unable to isolate messages initiated de novo by clinic employees to patients or providers/staff. Thus, all messages analyzed in this study were sent from patients to clinic staff, and all actions were conducted in relation to these messages. Messages sent from clinic staff (to patients and other employees) were included in the action count, but only when in reference to patient-initiated messages.
We applied descriptive statistics to nurses, who exhibited the greatest use of MyChart. Mean and median action/message counts for individual nurses were calculated for 2011 and 2014 and compared using t tests (assessing for a significant increase in actions and messages during this time frame). Mean and median action/message counts were also compared between clinical divisions (medical oncology, radiation oncology, and surgical oncology) using t tests. All statistical analysis and data summarization were performed using Python 2.7.3 and R 2.15.0 (Python Software, Beaverton, OR).
RESULTS
Of 3,416 employees identified in the 2009 to 2014 human resources database, 2,827 were excluded because of employment outside of Simmons Cancer Center and 103 were disqualified because of incomplete information within the database. Among the 486 remaining cancer center employees, 289 (59%) were identified as MyChart users (MyChart actions performed or messages received between 2009 and 2014). Staff positions and clinical division affiliations of the MyChart user cohort are listed in Table 1.
Table 1.
Between 2009 and 2014, 740,613 MyChart actions were performed by clinic providers and staff. Of these actions, 77% (566,651) were performed by nurses, 11% (81,873) by ancillary staff, 6% (44,755) by advanced practice providers, 5% (39,036) by physicians, and 1% (8,298) by clerical/managerial staff. Cancer center employees received 117,799 messages: 72% were received by nurses, 15% by ancillary staff, 7% by physicians, 4% by advanced practice providers, and 2% by clerical/managerial staff. On average, 6.3 staff MyChart actions were performed per patient-initiated message.
Figure 1A depicts the total number of user actions and received messages occurring annually between 2009 and 2014. The annual average actions per message are listed in this figure as well. Both actions and messages increased approximately 10-fold between 2011 and 2014. Figures 1B and 1C show the average number of user actions and messages per year over the duration of the study. The average number of user actions and messages increased each year for all positions except ancillary staff, which showed a slight decrease in 2014.
Because nurses had, by a considerable margin, the greatest rates of MyChart use, we further characterized their MyChart activity in the most recent period of the study. Appendix Figure A1 (online only) depicts the total number of MyChart actions and messages engaged by individual nurses in the year 2014. In 2014, nurses performed an average of 3,838 MyChart actions (median, 1,226) and received an average of 589 messages (median, 238) compared with an average of 591 actions (median, 239) and an average of 87 messages (median, 25) in 2011 (P < .001). Nurse use of MyChart was greatest in medical oncology clinics in 2014, accounting for 86% of nurse actions. Nurses in the medical oncology clinics also had the greatest intensity of use: eight nurses (20%) with > 10,000 actions per year and a maximum individual action count of 22,055. Surgical oncology clinic nurses had the greatest mean action count at 5,482 (median, 6,293) compared with medical oncology clinic nurses at 5,090 (median, 2,820) and radiation oncology clinic nurses at 691 (median, 286). The difference between surgical oncology clinic and medical oncology clinic nurses was not statistically significant (P = .78).
The timing of staff MyChart actions and received messages is shown in Figure 2. One percent of nurse actions and messages occurred on weekends; 13% and 12% were performed between 5:00 pm and 8:00 am. Among physicians, 11% of actions and 14% of messages were performed on weekends, with 30% and 31% occurring between 5:00 pm and 8:00 am; 14% of actions and 16% of received messages occurred between 8:00 pm and 7:00 am. In total, 16.4% of staff MyChart actions occurred outside of business hours (weekends and outside the hours of 8:00 am to 5:00 pm), of which 73.6% were performed by nurses and 13.2% by physicians.
DISCUSSION
Since Congress enacted the Health Information Technology for Economic and Clinical Health Act in 2009, patient health portal implementation and enrollment have expanded rapidly.3,4,17,19 The primary care literature is replete with data supporting positive aspects of patient health portals, including enhanced patient-provider communication, enriched comprehension of medical conditions, heightened engagement in medical decision making, and improved outcomes for certain chronic diseases.8,17,20-25 Adverse consequences of patient health portal use have been suggested as well, including undue clinic burden, lack of financial reimbursement, and issues surrounding patient safety.10,26
Application of patient health portals to the field of oncology, in which patient empowerment and communication are central tenets, requires careful consideration. This study analyzed patient health portal use by oncology providers and clinic staff at an NCI-designated comprehensive cancer center. Over the 6 years of our analysis, 289 employees of the Harold C. Simmons Comprehensive Cancer Center at UT Southwestern performed > 740,000 portal actions and received > 117,000 electronic messages. Nurses accounted for the majority of these actions. Sixteen percent of staff portal actions were performed outside clinic hours. Reflecting the tendency for patient-oriented electronic messages to generate a multistaff communication string, on average each from-patient message resulted in 6.3 staff actions. Over the last 4 years of our study period, overall use increased approximately 10-fold.
This substantial increase in staff effort mirrors the increased uptake and intensity of use by patients. In an earlier study, we demonstrated a near five-fold increase in the number of patients enrolled, with a 10-fold increase in total patient account log-ins, suggesting that patients and their caregivers use this technology more than twice as much as they did in the recent past.10 Compared with other populations, we found that patients with cancer used electronic portals far more frequently—12 times that of urology patients and more than four times that of families of children cared for in a congenital heart disease program.9,12 The considerable increase in portal use by both patients and staff likely reflects multiple factors: increased use of electronic text-based communication at a societal level; increased comfort with electronic communication, resulting in use of additional features; and change in institutional electronic health record policy resulting in availability of additional portal functions over time. For instance, the auto-release function for laboratory and imaging results was activated at our center during the study period.
As might be expected from clinic staffing structure, the bulk of work related to the patient health portal fell to nurses. By the final year of data collection, nurses performed an average of > 3,800 actions and received almost 600 messages annually. Among nurses exhibiting the heaviest use of this technology, there were > 22,000 actions performed annually. The number of MyChart actions varied widely among nurses, with nurses in medical oncology clinics demonstrating the greatest intensity of MyChart use. This distribution likely reflects the various roles nurses have within a cancer center as well as the nature of care delivered by various subspecialties.
It is notable that 197 (41%) employees in our cohort did not perform any MyChart actions during the study period, including 51% of physicians, 44% of clerical/managerial staff, 41% of ancillary staff, 36% of nurses, and 22% of advanced practice providers. For clerical/managerial and ancillary staff, these presumably reflect job categories that do not involve direct outpatient clinical care. It seems possible that the population of nurses who did not engage in MyChart activities represent those who do not participate in longitudinal patient-specific care, such as infusion nurses. There are a number of potential explanations for the substantial proportion of nonusing physicians. Some physicians may have only administrative or inpatient responsibilities. In addition, some physicians may fully delegate MyChart responsibilities to nurses and advanced practice staff. Although these clinicians likely interact with MyChart via nurse/advanced practice messaging, if physicians did not directly participate in from-patient electronic conversations, they were not included in the data set. Accordingly, it is possible that our study cohort may under-represent the true impact of MyChart-related activities on physician practice.
How nurses and other staff perceive this new role in clinical care remains poorly understood. Physicians across several specialties have acknowledged positive attributes of patient health portals, including improved access to care, enhanced communication, and heightened patient engagement.16 Conversely, they have also expressed concern regarding lowered threshold at which patients initiate communication with providers, usability of electronic platforms, cost of implementation, and workload implications.15-17,27,28 Future mixed-method studies are needed to explore implications for nurse and other staff time and effort and impact on in-person patient care activities and clinic work flow.
Staff use of patient portals outside clinic hours raises potential concerns about staff workload, professional satisfaction, and patient safety. In this analysis, 16% of user actions occurred on the weekend or on weekdays before 8:00 am or after 5:00 pm. Two percent of messages received by nurses and physicians occurred on weekends, and 14% occurred between 5:00 pm and 8:00 am. Although the MyChart electronic portal in use at our institution includes specific warnings not to use the feature for medically urgent issues, such direction requires patients to triage their own clinical developments. In primary care practice, it has been shown that approximately 4% of electronic messages are sent for potentially high-risk symptoms.26 Given the acuity of the underlying disease and the toxicity of therapies, it seems possible that this figure could be considerably higher in oncology populations.
How electronic portals affect clinical productivity and practice patterns remains poorly understood. Studies in primary care settings have reported modest increased productivity attributed to secure web-based messaging.14,29 It has also been suggested that numbers of routine and urgent care clinic visits are conserved despite availability of open access to personal medical records.25,30,31 Paradoxically, a large matched cohort study performed by Kaiser Permanente demonstrated substantially greater use of multiple health care resources (including office visits, telephone encounters, after-hours clinic visits, emergency department visits, and hospitalizations) after implementation of a patient health portal.32 Less is known about practice impact in oncology. Medical record access has been examined with regard to effects on anxiety and disease-specific education.27,33 However, there are few available data on workflow effects. It is possible that first-hand and immediate access to laboratory data and radiology results could lead to an increase in electronic messaging, telephone encounters, and clinic visits to address questions or anxiety arising from the information. Alternatively, some patients might believe the electronic availability of surveillance imaging results, for instance, eliminates the need for an associated clinical encounter, with undetermined consequences for clinical outcomes as well as practice revenue.
This study has a number of limitations. Although we present a quantitative analysis of staff portal use, as defined by numbers of actions and messages, we are not able to determine the amount of time employees devoted to these tasks or the impact on clinic work flow. Furthermore, because of inherent limitations within the available data tables, we were unable to capture messages initiated de novo by providers, capturing only those received from patients. As described, the first individual to open a given message was considered the primary recipient and was used in message-oriented analysis. We adopted this approach to prevent over-representation of the true number of messages sent to the clinic. As a byproduct, the true number of messages received by providers was likely under-represented. For instance, messages triaged by a nurse then sent to a physician were not included in this count. Further research into interemployee messaging patterns is needed. Finally, generalizability of our findings may be limited by the study cohort, because patients seeking care at NCI-designated comprehensive cancer centers may differ from the broader population by race, geographic location, and socioeconomic status.34,35
In conclusion, given the disease severity, longitudinal course, and data-intensive practice of oncology, understanding how patients and providers use electronic portals is key to practice quality and safety. Although this technology has been implemented widely in the last decade, few studies have directly examined the impact of electronic patient portals on cancer care providers. In the present analysis, we identified a dramatic and sustained increase in staff use of an electronic patient portal at an NCI-designated comprehensive cancer center. Most of this work falls to nurses, and a substantial proportion occurs outside clinic hours. Future research into the impact of this technology on staff and patient satisfaction, use of other health care resources, practice finances, and clinical outcomes will be essential as implementation and use continue to expand.
ACKNOWLEDGEMENT
Supported by National Cancer Institute (NCI) Cancer Clinical Investigator Team Leadership Award No. 1P30 CA142543-01 supplement (D.E.G.), NCI Midcareer Investigator Award in Patient-Oriented Research No. K24CA201543-01 (D.E.G.), National Institutes of Health (NIH) Grant No. 5R01CA152301 (Y.X.), Cancer Prevention and Research Institute of Texas Grant No. RP101251 (Y.X.), and the University of Texas Southwestern Center for Patient-Centered Outcomes Research, Agency for Healthcare Research and Quality Grant No. 1R24HS022418-01 (S.J.C.L.). Biostatistical support was provided by the Biostatistics and Bioinformatics Shared Resource at the Harold C. Simmons Cancer Center, University of Texas Southwestern Medical Center, Dallas, TX, which is supported in part by NCI Cancer Center Support Grant No. 1P30 CA142543-03. Research reported in this publication was also supported by the National Center for Advancing Translational Sciences of the NIH under Award No. TL1TR001104 as well as Clinical and Translational Science Award NIH Grant No. UL1-RR024982. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Appendix
AUTHOR CONTRIBUTIONS
Conception and design: Andrew L. Laccetti, Yang Xie, Simon J. Craddock Lee, David E. Gerber
Collection and assembly of data: Andrew L. Laccetti, Beibei Chen, Jennifer Cai, Samantha Gates, Yang Xie
Financial support: David E. Gerber
Administrative support: David E. Gerber
Data analysis and interpretation: Andrew L. Laccetti, Beibei Chen, Jennifer Cai, Samantha Gates, Yang Xie, David E. Gerber
Manuscript writing: All authors
Final approval of manuscript: All authors
AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST
Increase in Cancer Center Staff Effort Related to Electronic Patient Portal Use
The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or jop.ascopubs.org/site/misc/ifc.xhtml.
Andrew L. Laccetti
No relationship to disclose
Beibei Chen
No relationship to disclose
Jennifer Cai
No relationship to disclose
Samantha Gates
No relationship to disclose
Yang Xie
No relationship to disclose
Simon J. Craddock Lee
No relationship to disclose
David E. Gerber
Stock or Other Ownership: Gilead Sciences
Research Funding: ImmunoGen (Inst), ArQule (Inst), Synta (Inst), Genentech (Inst), Celgene (Inst), ImClone Systems (Inst), BerGenBio (Inst), Karyopharm Therapeutics (Inst)
Patents, Royalties, Other Intellectual Property: Royalties from Oxford University Press, Royalties from Decision Support in Medicine
Travel, Accommodations, Expenses: Eli Lilly, ArQule
REFERENCES
- 1.Tang PC, Ash JS, Bates DW, et al. Personal health records: Definitions, benefits, and strategies for overcoming barriers to adoption. J Am Med Inform Assoc. 2006;13:121–126. doi: 10.1197/jamia.M2025. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Archer N, Fevrier-Thomas U, Lokker C, et al. Personal health records: A scoping review. J Am Med Inform Assoc. 2011;18:515–522. doi: 10.1136/amiajnl-2011-000105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Weingart SN, Rind D, Tofias Z, et al. Who uses the patient internet portal? The PatientSite experience. J Am Med Inform Assoc. 2006;13:91–95. doi: 10.1197/jamia.M1833. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Hsu J, Huang J, Kinsman J, et al. Use of e-Health services between 1999 and 2002: A growing digital divide. J Am Med Inform Assoc. 2005;12:164–171. doi: 10.1197/jamia.M1672. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Davis Giardina T, Menon S, Parrish DE, et al. Patient access to medical records and healthcare outcomes: A systematic review. J Am Med Inform Assoc. 2014;21:737–741. doi: 10.1136/amiajnl-2013-002239. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Feeley TW, Shine KI. Access to the medical record for patients and involved providers: Transparency through electronic tools. Ann Intern Med. 2011;155:853–854. doi: 10.7326/0003-4819-155-12-201112200-00010. [DOI] [PubMed] [Google Scholar]
- 7.Fisher B, Bhavnani V, Winfield M. How patients use access to their full health records: A qualitative study of patients in general practice. J R Soc Med. 2009;102:539–544. doi: 10.1258/jrsm.2009.090328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Ross SE, Lin CT. The effects of promoting patient access to medical records: A review. J Am Med Inform Assoc. 2003;10:129–138. doi: 10.1197/jamia.M1147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Honeyman A, Cox B, Fisher B. Potential impacts of patient access to their electronic care records. Inform Prim Care. 2005;13:55–60. doi: 10.14236/jhi.v13i1.579. [DOI] [PubMed] [Google Scholar]
- 10.Gerber DE, Laccetti AL, Chen B, et al. Predictors and intensity of online access to electronic medical records among patients with cancer. J Oncol Pract. 2014;10:e307–e312. doi: 10.1200/JOP.2013.001347. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Cathala N, Brillat F, Mombet A, et al. Patient followup after radical prostatectomy by Internet medical file. J Urol. 2003;170:2284–2287. doi: 10.1097/01.ju.0000095876.39932.4a. [DOI] [PubMed] [Google Scholar]
- 12.Burke RP, Rossi AF, Wilner BR, et al. Transforming patient and family access to medical information: Utilisation patterns of a patient-accessible electronic health record. Cardiol Young. 2010;20:477–484. doi: 10.1017/S1047951110000363. [DOI] [PubMed] [Google Scholar]
- 13.Giardina T, Modi V, Parrish D, et al: The patient portal and abnormal test results: An exploratory study of patient experiences. Patient Experience Journal 2:20, 2015 [PMC free article] [PubMed]
- 14.Emont S. Measuring the Impact of Patient Portals: What the Literature Tells Us. Oakland, CA: California Health Foundation; 2011. [Google Scholar]
- 15.Siteman E, Businger A, Gandhi T, et al: Clinicians recognize value of patient review of their electronic health record data. AMIA Annu Symp Proc 2006:1101, 2006 [PMC free article] [PubMed] [Google Scholar]
- 16.Nazi KM. The personal health record paradox: Health care professionals’ perspectives and the information ecology of personal health record systems in organizational and clinical settings. J Med Internet Res. 2013;15:e70. doi: 10.2196/jmir.2443. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Kruse CS, Argueta DA, Lopez L, et al. Patient and provider attitudes toward the use of patient portals for the management of chronic disease: A systematic review. J Med Internet Res. 2015;17:e40. doi: 10.2196/jmir.3703. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Hill RG, Jr, Sears LM, Melanson SW. 4000 clicks: A productivity analysis of electronic medical records in a community hospital ED. Am J Emerg Med. 2013;31:1591–1594. doi: 10.1016/j.ajem.2013.06.028. [DOI] [PubMed] [Google Scholar]
- 19.Feldman H, Rodriguez ES. The future of oncology care with personal health records. Am Soc Clin Oncol Educ Book. 2012:e66–e69. doi: 10.14694/EdBook_AM.2012.32.135. [DOI] [PubMed] [Google Scholar]
- 20.Goldzweig CL, Orshansky G, Paige NM, et al. Electronic patient portals: Evidence on health outcomes, satisfaction, efficiency, and attitudes: A systematic review. Ann Intern Med. 2013;159:677–687. doi: 10.7326/0003-4819-159-10-201311190-00006. [DOI] [PubMed] [Google Scholar]
- 21.Byczkowski TL, Munafo JK, Britto MT. Variation in use of Internet-based patient portals by parents of children with chronic disease. Arch Pediatr Adolesc Med. 2011;165:405–411. doi: 10.1001/archpediatrics.2011.55. [DOI] [PubMed] [Google Scholar]
- 22.Grant RW, Wald JS, Schnipper JL, et al. Practice-linked online personal health records for type 2 diabetes mellitus: A randomized controlled trial. Arch Intern Med. 2008;168:1776–1782. doi: 10.1001/archinte.168.16.1776. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Earnest MA, Ross SE, Wittevrongel L, et al. Use of a patient-accessible electronic medical record in a practice for congestive heart failure: Patient and physician experiences. J Am Med Inform Assoc. 2004;11:410–417. doi: 10.1197/jamia.M1479. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.McCarrier KP, Ralston JD, Hirsch IB, et al. Web-based collaborative care for type 1 diabetes: A pilot randomized trial. Diabetes Technol Ther. 2009;11:211–217. doi: 10.1089/dia.2008.0063. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Ross SE, Moore LA, Earnest MA, et al. Providing a web-based online medical record with electronic communication capabilities to patients with congestive heart failure: Randomized trial. J Med Internet Res. 2004;6:e12. doi: 10.2196/jmir.6.2.e12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.North F, Crane SJ, Stroebel RJ, et al. Patient-generated secure messages and eVisits on a patient portal: are patients at risk? J Am Med Inform Assoc. 2013;20:1143–1149. doi: 10.1136/amiajnl-2012-001208. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Wiljer D, Leonard KJ, Urowitz S, et al. The anxious wait: Assessing the impact of patient accessible EHRs for breast cancer patients. BMC Med Inform Decis Mak. 2010;10:46. doi: 10.1186/1472-6947-10-46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Gravis G, Protière C, Eisinger F, et al. Full access to medical records does not modify anxiety in cancer patients: Results of a randomized study. Cancer. 2011;117:4796–4804. doi: 10.1002/cncr.26083. [DOI] [PubMed] [Google Scholar]
- 29.Liederman EM, Lee JC, Baquero VH, et al. The impact of patient-physician Web messaging on provider productivity. J Healthc Inf Manag. 2005;19:81–86. [PubMed] [Google Scholar]
- 30.Ralston JD, Hirsch IB, Hoath J, et al. Web-based collaborative care for type 2 diabetes: A pilot randomized trial. Diabetes Care. 2009;32:234–239. doi: 10.2337/dc08-1220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Green BB, Cook AJ, Ralston JD, et al. Effectiveness of home blood pressure monitoring, Web communication, and pharmacist care on hypertension control: A randomized controlled trial. JAMA. 2008;299:2857–2867. doi: 10.1001/jama.299.24.2857. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Palen TE, Ross C, Powers JD, et al. Association of online patient access to clinicians and medical records with use of clinical services. JAMA. 2012;308:2012–2019. doi: 10.1001/jama.2012.14126. [DOI] [PubMed] [Google Scholar]
- 33.Cahill JE, Lin L, LoBiondo-Wood G, et al. Personal health records, symptoms, uncertainty, and mood in brain tumor patients. Neurooncol Pract. 2014;1:64–70. doi: 10.1093/nop/npu005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Onega T, Duell EJ, Shi X, et al. Determinants of NCI Cancer Center attendance in Medicare patients with lung, breast, colorectal, or prostate cancer. J Gen Intern Med. 2009;24:205–210. doi: 10.1007/s11606-008-0863-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Onega T, Duell EJ, Shi X, et al. Influence of NCI cancer center attendance on mortality in lung, breast, colorectal, and prostate cancer patients. Med Care Res Rev. 2009;66:542–560. doi: 10.1177/1077558709335536. [DOI] [PMC free article] [PubMed] [Google Scholar]