Abstract
Few studies have explored adoption of patient portals for pediatric patients outside primary care or disease-specific applications. We examined use of patient-provider messaging in a patient portal across pediatric specialties during the three years after implementation of pediatric portal accounts at Vanderbilt University Medical Center. We determined the number of patient-initiated message threads and clinic visits for pediatric specialties and percentage of these outpatient interactions (i.e., message threads + clinic visits) done through messaging. Generalized estimating equations measured the likelihood of message-based interaction. During the study period, pediatric families initiated 33,503 messages and participated in 318,386 clinic visits. The number of messages sent (and messaging percentage of outpatient interaction) increased each year from 2,860 (2.7%) to 18,772 (17%). Primary care received 4,368 messages (3.4% of outpatient interactions); pediatric subspecialties, 29,135 (13.0%). Rapid growth in messaging volume over time was seen in primary care and most pediatric specialties (OR>1.0; p<0.05).
Introduction
Patient portals are secure online applications that give patients convenient, 24-hour access to personal health information with an Internet connection.1 Government regulations like the Health Information Technology for Economic and Clinical Health Act (HITECH) and Meaningful Use (MU) have helped drive the development and implementation of patient portals.2,3
Secure messaging between patients and their providers is one of the most popular features of patient portals.4 Research indicates clinical care is delivered through portal message exchanges —for example, patients may report new problems or request treatment.5,6 A recent systematic review suggests secure messaging can improve medication adherence and clinic attendance.7 Other studies report messaging may increase patient satisfaction, improve efficiency for patients and providers, ease reporting of laboratory results, reduce costs, and increase productivity.4,8
Much of the research about secure messaging through patient portals has focused on adult primary care or chronic disease settings.4,9–14 At our institution, we have demonstrated broad adoption of secure messaging across diverse clinical specialties, with the majority of use occurring in medical or surgical specialties.15,16 A systematic review of patient portal implementation in pediatrics17 highlighted that existing research has largely focused on subpopulations of children with chronic diseases or ongoing medical conditions,18–20 or have been done in primary care. The review recognized an ongoing need for more quantitative research on volume of messaging, frequency of use over time, and the impact of portal use on other types of patient contact following portal adoption.17
In this study, we aimed to characterize adoption of secure messaging by pediatric specialists, patients and their families after broad deployment of a portal at a large academic medical center. We specifically focused on uptake of messaging by primary care pediatricians and pediatric subspecialists, and examined differences in messaging rates by adolescents and their parents or other caregivers.
Methods
Setting
This retrospective cohort study was conducted at Vanderbilt University Medical Center (VUMC), which includes the tertiary care Monroe Carell Jr. Children’s Hospital at Vanderbilt (MCJCHV). VUMC implemented the My Health at Vanderbilt (MHAV) patient portal in 2005, which is available to all patients seen at VUMC. MHAV provides typical patient portal functions including secure messaging, appointment scheduling, access to selected portions of the electronic health record (EHR), account and bill management, and delivery of personalized health information. MHAV accounts for pediatric patients and their families were launched in 2007 and were fully deployed across pediatric specialties by 2008.21 VUMC policies for pediatric patient and parent or guardian access are similar to those for portals at other major children’s hospitals.22 Parents and guardians may access their children’s health information through MHAV until the child turns 18 years of age. MHA V allows adolescents ages 13 years or older to have parent-controlled MHAV accounts with access to functions determined by the parent.
Study population
We examined all outpatient interactions at VUMC for pediatric patients between January 1, 2008 and December 31, 2010, the first three years after deployment of MHAV to pediatric patients and providers. Patients were stratified by age into three groups: children (<13 years), adolescents (≥13 and <18 years), and adults (≥18 years). MHAV users consisted of either VUMC patients (over the age of 13 years); delegates, individuals whom a patient designates to access MHAV on their behalf; or surrogates, parents or guardians who have access to their child’s or adolescent’s health information through MHAV. We defined an outpatient interaction as either a traditional face-to-face clinic visit or a patient-initiated message thread (i.e., initial message and all replies) between MHAV users and pediatric providers. For this study, we defined pediatric use as accessing of the MHAV portal by pediatric patients or any of their delegates or surrogates. Clinical providers have a designated message basket in which they receive messages from patients through MHAV, as well as through inter-departmental provider-to-provider messages. Providers access their message basket through the EHR. Each clinical unit and individual providers handle messages differently; some providers choose to answer their own messages, while other baskets are triaged first by administrative assistants or allied health professionals.
Measures
We collected data on all traditional outpatient clinic visits to pediatric providers during the study period. For each clinic visit, we recorded the age, gender, and race of the patient, and the specialty of the clinic. Pediatric specialties were categorized as primary care, cardiology, development, endocrinology, gastroenterology, genetics, hematology/oncology, infectious disease, neonatology, nephrology, neurology, pulmonology, and rheumatology. Visits to multidisciplinary clinics involving more than one specialty were excluded from analysis.
We also recorded the number of secure message threads initiated by pediatric accounts during the study period. For each message thread, we recorded the role of the sender (i.e., self, delegate, or surrogate); age, gender, and race of the pediatric patient; and specialty of the receiving provider. Messages threads were assigned to the same 13 specialties as the clinic visits. Messages sent to multidisciplinary or administrative message baskets were excluded.
Analysis
We determined the total number of unique MHAV users, message threads, recipient message baskets, and outpatient visits for each month of the study period and for each pediatric subspecialty. We constructed descriptive distributions and summary statistics for patients seen in clinic visits and patients about whom messages were sent. Generalized estimating equations with logit link controlling for age, gender, race, and time in months measured the likelihood of message-based versus clinic outpatient interaction across pediatric specialties. This analysis was conducted in R version 3.0.1, and the models were fit using the geeM package.23 The VUMC Institutional Review Board approved this study.
Results
Patient demographics
The demographic distribution of the pediatric patients who visited clinics or were the subjects of portal messages is shown in Table 1. Message-based interactions were more common among male and Caucasian patients (p<0.05). In models adjusting for patient demographics and clinical characteristics, the odds of a message-based rather than clinic-based interaction for African American and Hispanic patients were lower than those for Caucasian patients. Over half of all messages (58%, n=19,405) were in regard to children, with 23.8% (n=7,976) regarding adolescents and 18.3% (n=3,404) regarding adults.
Table 1.
Demographic distributions across pediatric outpatient interactions.
| Outpatient Encounters (N=318,386) N (% Total Encounters) | Message Threads (N=33,503) N (% Total Messages) | Odds Ratio of Message-based vs. Clinic-based Interactiona OR (95% CI) | |
|---|---|---|---|
| Gender | |||
| Female | 152,695 (48.0%) | 15,705 (46.9%) | 1 [Reference] |
| Male | 165,691 (52.0%) | 17,798 (53.1%) | 1.11 (1.08,1.14) |
| Race | |||
| African American | 69,333 (21.8%) | 1,999 (6.0%) | 1 [Reference] |
| American Indian/Eskimo | 525 (0.2%) | 14 (0.0%) | 0.85 (0.48, 1.50) |
| Asian/Pac. Islander | 7,059 (2.2%) | 423 (1.3%) | 2.14 (1.91, 2.40) |
| Caucasian | 156,358 (49.1%) | 28,131 (84.0%) | 4.29 (4.08, 4.52) |
| Hispanic/Latino | 31,253 (9.8%) | 179 (0.5%) | 0.21 (0.18, 0.25) |
| Unknown | 53,858 (16.9%) | 2,757 (8.2%) | 1.30 (1.22, 1.39) |
Odds ratios were computed using the generalized estimating equations model. This model controlled for age, gender, race, and time in months in order to measure the likelihood of message-based versus clinic outpatient interaction across pediatric specialties.
Clinic visit and messaging volumes
During the study period, pediatric patients participated in 318,386 outpatient clinic visits. The total number of visits across specialties was relatively stable over the course of the study (Figure 1A). Primary care had the most clinic visits at 122,773 visits during the study period, followed by endocrinology (37,337), hematology/oncology (34,831), and gastroenterology (22,506). Infectious disease, neonatology, and palliative care had the fewest clinic visits. 81 Palliative Care outpatient visits were excluded, as there were no message threads associated with the specialty. 3430 encounters with Allergy and Immunology and 148 messages and encounters associated with Adolescent Medicine were excluded as these services are multidisciplinary. For the purposes of the analysis, 47 encounters and messages associated with Heart Transplant were grouped with Cardiology.
Figure 1.
Volumes of outpatient interaction types. Number of (A) outpatient clinic encounters, (B) message threads, (C) recipient message baskets, and (D) the messaging percentage of outpatient interaction per month. Month 0 corresponds to January 2008; month 35 corresponds to December 2010.
From 2008 to 2010, pediatric patients and their families initiated 33,503 message threads, with the number of messages increasing from 2,860 in the first year to 11,871 and 18,772 in the second and third years, respectively (see Figure 1B). The number of provider baskets receiving messages increased from 346 in the first year to 701 and 1017 in the second and third years, respectively (Figure 1C). As there was only one message thread associated with Emergency Medicine, and this specialty is not associated with traditional outpatient visits, the message was excluded from the analysis. 894 messages associated with Allergy and Immunology were excluded for the reason stated above.
Users and user roles
Message threads were initiated by 19,049 unique users, including 13,848 surrogates (72.7%), 4,747 patients themselves (24.9%), and 454 delegates (2.4%), and received by 2,064 unique provider message baskets. The distribution of user types sending messages by month for children and adolescents is shown in Figure 2. Of the 10,747 users who messaged on behalf of children younger than age 13, 10,452 (97.3%) were surrogates and 261 (2.4%) were delegates. Forty-six messages (1.3%) were sent by the children themselves; in the early stages of pediatric account deployment, some children did have their own accounts created inadvertently. Of the 4,202 users who messaged on behalf of adolescents, 3,328 (79.2%) were surrogates, 99 (2.4%) were delegates, and 775 (18.4%) were the adolescents themselves. Although the number of adolescents using messaging increased from 103 in 2008 to 304 and 368 in 2009 and 2010, respectively, the percentage of total users who were adolescents decreased over time. Of the message exchanges with pediatric providers regarding adult patients, 68 (1.7%) of the 4,100 who messaged on behalf of adult accounts were surrogates, 94 (2.3%) were delegates, and 3,938 (96%) were the patients themselves.
Figure 2.
Distribution of users messaging on behalf of A. children and B. adolescents. Month 0 corresponds to January 2008; month 35 corresponds to December 2010.
Messaging across clinical specialties
Pediatric gastroenterology received the most messages (n=9,399; 28.1%) over the study period, followed by endocrinology (n=6,037, 18%), primary care (n=4,368; 13%), and neurology (n=3,717; 11.1%) (Table 2). Specialties receiving the fewest messages included development, infectious disease, genetics, and neonatology. With the exception of genetics, neonatology, and hematology/oncology, all pediatric specialties experienced significant growth (p-value<0.05; OR>1.0) in message volume over time during the study period when compared to primary care.
| Specialty | Total Outpatient Encounters (N=318,386) N (% Total Encounters) | Total Message Threads (N=33,503) N (% Total Messages) | % of Outpatient Interactions that are Messages | OR of Message-based vs. Clinic-based Interactiona OR (95% CI) |
|---|---|---|---|---|
| Primary Care | 122,773 (38.6%) | 4,368 (13.0%) | 3.4% | 1 [Reference] |
| Cardiology | 22,280 (7.0%) | 2,017 (6.0%) | 8.3% | 0.70 (0.58, 0.84) |
| Development | 11,969 (3.8%) | 574 (1.7%) | 4.6% | 0.26 (0.19, 0.36) |
| Endocrinology | 37,337 (11.7%) | 6,037 (18.0%) | 13.9% | 1.03 (0.91, 1.15) |
| Gastroenterology | 22,506 (7.1%) | 9.399 (28.1%) | 29.5% | 3.35 (3,01,3.73) |
| Genetics | 7,554 (2.4%) | 10 (0.0%) | 0.1% | 0.00 (0.00, 0.00) |
| Hematology/Oncology | 34,831 (10.9%) | 1,371 (4.1%) | 3.8% | 0.53 (0.45, 0.62) |
| Infectious Disease | 3,217 (1.0%) | 39 (0.1%) | 1.2% | 0.04 (0.02, 0.10) |
| Neonatology | 398 (0.1%) | 6 (0.0%) | 1.5% | 0.03 (0.01, 0.17) |
| Nephrology | 8,455 (2.7%) | 1,195 (3.6%) | 12.4% | 1.30 (1.10, 1.55) |
| Pulmonology | 15,185 (4.8%) | 2,781 (8.3%) | 15.5% | 1.76 (1.54, 2.00) |
| Rheumatology | 6,737 (2.1%) | 1,095 (3.3%) | 14.0% | 1.01 (0.80, 1.26) |
Odds ratios were computed using the generalized estimating equations model. This model controlled for age, gender, race, and time in months in order to measure the likelihood of message-based versus clinic outpatient interaction across pediatric specialties.
Over the study period, pediatric patients and their families used messaging for 10% of all outpatient interactions (i.e., message threads plus clinic visits). The proportion of pediatric outpatient interaction conducted through messaging rose from 2.7% in 2008 to 10.3% in 2009 and 14.6% in 2010 (Figure 1D). The proportion of message-based outpatient interaction for pediatric primary care was 3.4%, while the mean proportion of outpatient interaction across pediatric subspecialties was 19.1%. The specialties with the highest utilization of messaging as a form of outpatient interaction were gastroenterology (29.5% of outpatient interactions), pulmonology (15.5%), neurology (14.7%), and rheumatology (14%) (Table 2). The specialties with the lowest percentage of messaging-based outpatient interaction were neonatology (1.5%), infectious disease (1.2%), genetics (0.1%), and palliative care (0%). outpatient interactions by specialty.
Discussion
This study demonstrates rapid adoption of secure messaging through a patient portal by pediatric patients, families, and providers, with the total volume of messaging increasing by more than six times during the first three years after deployment of pediatric accounts. There was also substantial growth in the proportion of outpatient interaction involving messaging rather than in-person encounters, increasing from 2.7% in 2008 to 14.6% by 2010. In this study, a large number of adolescent patients exchanged messages with their providers; by 2010, 16% of all those users who were messaging on behalf of adolescents were adolescents themselves. To our knowledge, this study is the first to characterize adoption of portal messaging across a diverse set of pediatric patients and clinical specialties.
We found pediatric primary care experienced lower volumes and less growth in messaging relative to pediatric subspecialties, which differs from usage patterns that have previously been reported for other institutions, primarily in adults.24 Although primary care clinics handled over a third of all pediatric clinic visits, only 13% of all messages were sent to pediatric primary care providers. There are several possible reasons for this observation. First, children seeing primary care physicians tend to be healthier than children seeing subspecialists, with a large proportion of outpatient visits dedicated to well child visits which may not need a lot of follow-up or extended communications that would be well-suited to portal-based messaging. Second, acute issues seen by pediatric primary are issues that typically resolve without needing follow-up, such as viral infections. Third, the MCJCHV is a tertiary care center with a large rural referral base. Some families may have visited specialist clinics for specific needs and then followed up with pediatricians outside the VUMC network. The middle Tennessee area has many private pediatrics practices not affiliated with VUMC. These providers would not have access to the MHAV patient portal and may have their own patient portal; thus, such patients and their families would be less likely to use MHAV messaging.
In contrast, pediatric specialists, such as gastroenterologists, who managed a high volume of messages over the course of the study (29.5% of outpatient interactions), frequently provide care for children who have chronic diseases, such as short gut syndrome and inflammatory bowel disease, which require long-term nutrition and symptom management. Management of such long-term care may lend itself to message-based interactions between clinic visits. Gastroenterology is also a highly procedural specialty, and patients and their families may use the portal to follow up on the results of biopsies from endoscopies and other procedures. Pediatric endocrinology, neurology, and pulmonology, which also experienced high volumes of messaging, are also specialties that involve chronic conditions (e.g., diabetes, seizure disorders, and cystic fibrosis) that may require repeated interactions between families and providers for medication or symptom management, making secure messaging a valuable tool. In contrast, the specialties of pediatric genetics, development, and neonatology, which had lower volumes of clinic visits, may involve fewer interactions with patients and family, with little ongoing interaction after establishing a diagnosis. Some variation in adoption of secure messaging across specialties also may be due to individual differences in group practices at VUMC. Barriers to adoption of secure messaging that have been described for adult populations in the past have included providers’ workload concerns, clinical communication preferences, and clinical adoption inertia resulting from the information overload in a clinical environment that is already highly computerized.25 The reasons for the differences in usage of patient portal messaging by the pediatric primary care and subspecialty populations are not evident from our usage data alone. Our ongoing research is exploring the content of portal messages across specialties, which may yield further insight into these differences. Future research using qualitative methods such as interviews or focus groups with patients and physicians may elucidate the reasons for differences in usage.
Prior work has shown a complex relationship between the implementation of messaging and effects on the number of traditional clinic visits. Some studies reported declines in the number of traditional clinic visits after messaging implemention,10,11,26 suggesting that clinical care may be offset; others have observed no change or increases in office visits after messaging is implemented.27,28 Although we were not able to obtain data about the baseline number of clinic visits, across specialties, the number of clinic visits per month did not change significantly over the study period after the portal was implemented.
There are important considerations uniquely relevant to use of patient portals and secure messaging in pediatrics. Adolescents’ and parents’ legal rights to access patient records are not standardized and vary between states. MHAV requires parents to register to access their child’s accounts, which enabled us to contrast usage between patients, delegates, and surrogates. We found that, as expected, most of those messaging about children were surrogates (97%), with the rest being the small number of delegates who had been given access to their record. However, adolescents sent their own messages 18% of the time. Of all messages sent during the study, 18% were sent about adult patients, an observation due to the continued care of patients over the age of 18 by pediatric specialists, usually for congenital disorders or diagnoses common during childhood. Of those who messaged pediatric providers on behalf of adults, 96% were the adults themselves, 2% were surrogates, and 2% were delegates.
This study has several limitations. First, it was conducted at a single large academic medical center with a locally developed patient portal with specific features and attributes that may limit the generalizability of the findings to other institutions.21 Second, VUMC has approached pediatric accounts like other major children’s hospitals with policies that focus on preserving parental and guardian rights over those of adolescents21,22 (i.e., the “family engagement” model29). As such, parents have access to all of their child’s heath data and control access rights for their children. In contrast, the University of Florida implemented an adolescent patient portal with very limited access to adolescent patient health data by parents or guardians, also known as the confidentiality model.29 This approach was designed to comply with known adolescent care best practices as well as state laws that protect confidentiality of teenage patients. In this study, although the rate of portal and messaging use by adolescent patients themselves was high, our policies that allow ongoing parental access for adolescent patients may have limited adolescent use, especially for sensitive primary care issues such as birth control and sexually transmitted infections. In addition, it’s possible that some of the adolescents or adults who sent messages themselves were actually caregivers who were logged into the patient’s account. However, VUMC discourages this practice by requiring surrogate account use for young children and then keeping the surrogate accounts separate from the patient accounts when patients turn 13.
Our study is limited in that MHAV does not store complete sociodemographic information for all delegates and surrogates, and thus, our analysis included only the characteristics of the patient about whom the message was sent. Thus, we cannot explore whether there were age- or race-based disparities in usage of the caregivers. The focus of this work was messaging use across pediatric specialties, as age and racial disparities in portal usage have been explored elsewhere.30,31 In addition, this paper describes data from 2008 to 2010, and both healthcare institutions and consumers have had substantial experience with patient portals since that time. Thus, contemporary usage patterns, which may be influenced by the Affordable Care Act and Meaningful Use incentives and penalties, may vary. We selected this time period to focus on the usage of a portal immediately after implementation of accounts for pediatric patients and their caregivers. Thus, our results may inform other institutions that are at the early stages of pediatric portal adoption. Our research group is currently examining long-term trends in usage of the portal across specialties.
In this study, we only explored use of secure messaging, not looking at other functions, such as access of health education topics or access of selected portions of the EHR. This decision was made because messages are sent to baskets associated with a certain specialty, but it would have been difficult to determine the specialties associated with use of all other functions. We also assumed that the specialty associated with the initial message within each thread was the most appropriate specialty for the entire thread of messages, not accounting for potential transfers to other clinical specialties. We did not examine message content to determine what types of care or other services were provided through these messages, although this topic is the focus of ongoing research. This study did also not look at other forms of outpatient interaction beyond messaging and clinic visits, namely, telephone calls and non-secure emails. At VUMC, telephone calls are not systematically recorded, preventing quantification of call volumes; email interaction with patients is strongly discouraged. Finally, in our analysis, the GEE modeling approach allowed us to cluster records of message threads associated with the same patient, but we were unable to cluster encounters by patient. Therefore unaccounted correlations for interactions involving the same patient might affect the estimated standard errors. However, given the large sample size and high level of significance in the observed p-values, our findings are unlikely to be affected.
Conclusions
This study is one of the first to analyze use of secure messaging in a patient portal for pediatric patients across clinical subspecialties. After widespread deployment of pediatric accounts at our children’s hospital, most specialties experienced rapid growth in adoption of secure messaging. In contrast to studies of portal use for adult patients, we found that pediatric subspecialties demonstrated greater growth in messaging use as a form of outpatient interaction compared to primary care. Our research may inform institutions that are at the early stages of patient portal adoption or deployment of pediatric accounts. As different stages of MU unfold across the country and pediatric practices adopt patient portals and secure messaging, providers can expect to see rapid increase in the usage of messaging as a form of outpatient interaction, not only in primary care but also across pediatric subspecialties. In adopting patient portals at a children’s hospital, adolescents require unique considerations. Our study revealed enthusiastic adoption by adolescent patients across clinical specialties with use of patient-provider messaging by both adolescents and their parents. When implementing portal messaging for the pediatric population, institutions must take care not to violate state privacy laws and protect the privacy of adolescents. Additional research is needed to examine the types of information and care delivered through patient portal messages in pediatric specialties.
Figure 3.
Use of secure messaging across pediatric specialties. Number of office visits, number of message threads, and the percentage of outpatient encounters that are messages are shown for each specialty. Number of office visits and message threads are plotted on the left y-axis. % of outpatient encounters as messages is plotted on the right y-axis. Months are plotted on the x-axis from month 0 (January 2008) to month 35 (December 2010).
References
- 1.What is a patient portal? FAQs | Providers & Professionals | HealthIT.gov. http://www.healthit.gov/providers-professionals/faqs/what-patient-portalAccessed May 1, 2015.
- 2.Irizarry T, DeVito Dabbs A, Curran CR. Patient Portals and Patient Engagement: A State of the Science Review. J Med Internet Res. 2015;17(6) doi: 10.2196/jmir.4255. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Wright A, Feblowitz J, Samal L, McCoy AB, Sittig DF. The Medicare Electronic Health Record Incentive Program: Provider Performance on Core and Menu Measures. Health Serv Res. 2014;49(1pt2):325–346. doi: 10.1111/1475-6773.12134. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Goldzweig CL, Towfigh AA, Paige NM, et al. Washington (DC): Department of Veterans Affairs (US); 2012. Systematic Review: Secure Messaging Between Providers and Patients, and Patients’ Access to Their Own Medical Record: Evidence on Health Outcomes, Satisfaction, Efficiency and Attitudes. http://www.ncbi.nlm.nih.gov/books/NBK100359/Accessed May 1, 2015. [PubMed] [Google Scholar]
- 5.Barnett TE, Chumbler NR, Vogel WB, Beyth RJ, Qin H, Kobb R. The effectiveness of a care coordination home telehealth program for veterans with diabetes mellitus: a 2-year follow-up. Am J Manag Care. 2006;12(8):467–474. [PubMed] [Google Scholar]
- 6.Earnest MA, Ross SE, Wittevrongel L, Moore LA, Lin C-T. Use of a patient-accessible electronic medical record in a practice for congestive heart failure: patient and physician experiences. J Am Med Inform Assoc JAMIA. 2004;11(5):410–417. doi: 10.1197/jamia.M1479. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Lusignan S de, Mold F, Sheikh A, et al. Patients‘ online access to their electronic health records and linked online services: a systematic interpretative review. BMJ Open. 2014;4(9):e006021. doi: 10.1136/bmjopen-2014-006021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Wallwiener M, Wallwiener CW, Kansy JK, Seeger H, Rajab TK. Impact of electronic messaging on the patient-physician interaction. J Telemed Telecare. 2009;15(5):243–250. doi: 10.1258/jtt.2009.090111. [DOI] [PubMed] [Google Scholar]
- 9.Liederman EM, Lee JC, Baquero VH, Seites PG. Patient-Physician Web Messaging. J Gen Intern Med. 2005;20(1):52–57. doi: 10.1111/j.1525-1497.2005.40009.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Bergmo TS, Kummervold PE, Gammon D, Dahl LB. Electronic patient-provider communication: will it offset office visits and telephone consultations in primary care? Int J Med Inf. 2005;74(9):705–710. doi: 10.1016/j.ijmedinf.2005.06.002. [DOI] [PubMed] [Google Scholar]
- 11.Zhou YY, Garrido T, Chin HL, Wiesenthal AM, Liang LL. Patient access to an electronic health record with secure messaging: impact on primary care utilization. Am J Manag Care. 2007;13(7):418–424. [PubMed] [Google Scholar]
- 12.Turvey C, Klein D, Fix G, et al. Blue Button use by patients to access and share health record information using the Department of Veterans Affairs’ online patient portal. J Am Med Inform Assoc JAMIA. 2014;21(4):657–663. doi: 10.1136/amiajnl-2014-002723. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Wald JS. Variations in Patient Portal Adoption in Four Primary Care Practices. AMIA Annu Symp Proc. 2010;201(0):837–841. [PMC free article] [PubMed] [Google Scholar]
- 14.Goldzweig CL, Orshansky G, Paige NM, et al. Electronic patient portals: evidence on health outcomes, satisfaction, efficiency, and attitudes: a systematic review. Ann Intern Med. 2013;159(10):677–687. doi: 10.7326/0003-4819-159-10-201311190-00006. [DOI] [PubMed] [Google Scholar]
- 15.Cronin RM, Davis SE, Shenson JA, Chen Q, Rosenbloom ST, Jackson GP. Growth of Secure Messaging Through a Patient Portal as a Form of Outpatient Interaction across Clinical Specialties. Appl Clin Inform. 2015;6(2):288–304. doi: 10.4338/ACI-2014-12-RA-0117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Shenson JA, Cronin RM, Davis SE, Chen Q, Jackson GP. Rapid growth in surgeons’ use of secure messaging in a patient portal. Surg Endosc. 2015. Jun, [DOI] [PMC free article] [PubMed]
- 17.Bush RA, Connelly CD, Fuller M, Pérez A. Implementation of the Integrated Electronic Patient Portal in the Pediatric Population: A Systematic Review. Telemed J E-Health Off J Am Telemed Assoc. 2015. Aug, [DOI] [PMC free article] [PubMed]
- 18.Hsiao AL, Bazzy-Asaad A, Tolomeo C, Edmonds D, Belton B, Benin AL. Secure Web Messaging in a Pediatric Chronic Care Clinic: A Slow Takeoff of “Kids’ Airmail.’. Pediatrics. 2011;127(2):e406–e413. doi: 10.1542/peds.2010-1086. [DOI] [PubMed] [Google Scholar]
- 19.Borowitz SM, Wyatt JC. THe origin, content, and workload of e-mail consultations. JAMA. 1998;280(15):1321–1324. doi: 10.1001/jama.280.15.1321. [DOI] [PubMed] [Google Scholar]
- 20.Rosen P, Kwoh CK. Patient-physician e-mail: an opportunity to transform pediatric health care delivery. Pediatrics. 2007;120(4):701–706. doi: 10.1542/peds.2007-1094. [DOI] [PubMed] [Google Scholar]
- 21.Osborn CY, Rosenbloom ST, Stenner SP, et al. MyHealthAtVanderbilt: policies and procedures governing patient portal functionality. J Am Med Inform Assoc JAMIA. 2011;18(1):i18–i23. doi: 10.1136/amiajnl-2011-000184. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Bourgeois FC, Mandl KD, Shaw D, Flemming D, Nigrin DJ. Mychildren’s: Integration of a Personally Controlled Health Record with a Tethered Patient Portal for a Pediatric and Adolescent Population. AMIA Annu Symp Proc; 2009; pp. 65–69. [PMC free article] [PubMed] [Google Scholar]
- 23.McDaniel L, Henderson N, Rathouz P. Fast Pure R Implementation of GEE: Application of the Matrix Package. R J. 2013;5(1):181–187. [PMC free article] [PubMed] [Google Scholar]
- 24.Health IT Datasets from the ONC Health IT Dashboard. http://dashboard.healthit.gov/data/Accessed May 1, 2015.
- 25.Byrne JM, Elliott S, Firek A. Initial Experience with Patient-Clinician Secure Messaging at a VA Medical Center. J Am Med Inform Assoc JAMIA. 2009;16(2):267–270. doi: 10.1197/jamia.M2835. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Baer D. Patient-Physician E-Mail Communication: The Kaiser Permanente Experience. J Oncol Pract. 2011;7(4):230–233. doi: 10.1200/JOP.2011.000323. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Liss DT, Reid RJ, Gembowski D, Rutter CM, Ross TR, Fishman PA. Changes in Office Visit Use Associated With Electronic Messaging and Telephone Encounters Among Patients With Diabetes in the PCMH. Ann Fam Med. 2014;12(4):338–343. doi: 10.1370/afm.1642. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Palen TE, Ross C, Powers J, Xu S. Association of online patient access to clinicians and medical records with use of clinical services. JAMA. 2012;308(19):2012–2019. doi: 10.1001/jama.2012.14126. [DOI] [PubMed] [Google Scholar]
- 29.Thompson LA, Novak M. Adolescent Health and the Electronic Health Record: Can it be a social media tool for quality adolescent care? J Adolesc Health Off Publ Soc Adolesc Med. 2016;58(2):698–710. [PubMed] [Google Scholar]
- 30.Gordon NP, Hornbrook MC. Differences in Access to and Preferences for Using Patient Portals and Other eHealth Technologies Based on Race, Ethnicity, and Age: A Database and Survey Study of Seniors in a Large Health Plan. J Med Internet Res. 2016;18(3):e50. doi: 10.2196/jmir.5105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Ancker JS, Barrón Y, Rockoff ML, et al. Use of an Electronic Patient Portal Among Disadvantaged Populations. Journal of General Internal Medicine. 2011;26(10):1117–1123. doi: 10.1007/s11606-011-1749-y. [DOI] [PMC free article] [PubMed] [Google Scholar]




