Skip to main content
PMC home page
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2017 May 15.
Published in final edited form as: J Health Commun. 2015 Sep 2;21(1):85–91. doi: 10.1080/10810730.2015.1033118

A Mixed-Methods Study of Patient-Provider E-mail Content in a Safety-Net Setting

Jacob B Mirsky 1, Lina Tieu 1, Courtney Lyles 1, Urmimala Sarkar 1
PMCID: PMC5431571  NIHMSID: NIHMS784780  PMID: 26332306

Abstract

Objective

To explore the content of patient-provider e-mails in a safety-net primary care clinic.

Methods

We conducted a content analysis using inductive and deductive coding of e-mail exchanges (n=31) collected from January through November of 2013. Participants were English-speaking adult patients with a chronic condition (or their caregivers) cared for at a single publicly-funded general internal medicine clinic and their primary care providers (attending general internist physicians, clinical fellows, internal medicine residents, and nurse practitioners).

Results

All e-mails were non-urgent. Patients included a medical update in 19% of all e-mails. Patients requested action in 77% of e-mails, and the most common requests overall were for action regarding medications or treatment (29%). Requests for information were less common (45% of e-mails). Patient requests (n=56) were resolved in 84% of e-mail exchanges, resulting in 63 actions.

Conclusion

Patients in safety-net clinics are capable of safely and effectively using electronic messaging for between-visit communication with providers.

Practical Implications

Safety-net systems should implement electronic communications tools as soon as possible to increase healthcare access and enhance patient involvement in their care.

Keywords: secure messaging, e-mail, communication, safety-net, primary care

Introduction

The growth of the Internet in modern health care delivery (“Health Information Technology for Economic and Clinical Health (HITECH) Act,” 2009; “Health Information Technology: Initial Set of Standards, Implementation Specifications, and Certification Criteria for Electronic Health Record Technology; Final Rule,” 2010) has resulted in the wide dissemination of and United States federal financial incentives for online patient portals (Blumenthal & Tavenner, 2010). A patient portal is a secure website maintained by healthcare providers and linked to a patient’s electronic health records (EHR). Among many other functions, patient portals enable secure electronic messaging with providers that, compared to e-mail, protects personal health information (“The Value of Personal Health Records and Web Portals to Engage Consumers and Improve Quality,” 2012). In recent years, secure electronic messaging between patients and providers for between-visit communication has become standard of care (“Crossing the Quality Chasm: A New Health System for the 21st Century,” 2001). Both patient portals and secure electronic messaging improve communication ease and efficiency, shared decision-making, patient self-management, patient satisfaction, and chronic disease management (Delbanco & Sands, 2004; Fox, 2011; Harris, Haneuse, Martin, & Ralston, 2009; Liederman, Lee, Baquero, & Seites, 2005; Ralston, Hirsch, et al., 2009; Rosen & Kwoh, 2007; Zhou, Kanter, Wang, & Garrido, 2010). Secure electronic messaging additionally increases physician and patient satisfaction, provider productivity, and health care quality (Gaster et al., 2003; Harris et al., 2009; Pearl, 2014; Ralston, Rutter, et al., 2009; Rosen & Kwoh, 2007; Zhou et al., 2010). Consequently, secure electronic messaging is now utilized across the United States, with a rapid increase in uptake. Kaiser Permanente Northern California experienced an increase in secure electronic messages of 20% per year, on average, from 2008 to 2013 with an estimated 8.2 million messages sent in 2013 (Pearl, 2014). Unfortunately, many urban and safety-net settings experience barriers to online patient portal and secure electronic messaging use, including variations in enrollment depending on race, ethnicity, and age (Goel, Brown, Williams, Hasnain-Wynia, et al., 2011). Electronic communication also occurs in the form of e-mail between patients and providers outside of a secure electronic portal, especially because many health systems do not have patient portals in place. In our formative work with low-income, diverse patients, many who are not familiar with secure electronic messaging via a portal do use e-mail. Therefore, in asking about interest in secure electronic messaging, we use the term “e-mail” because it conveys the idea of non-visit communication in layperson’s terms. A recent study found that only 19% of patients in a safety-net public hospital used e-mail with providers, even though 71% were interested in electronic communication (Schickedanz et al., 2013), which could be due to a combination of limited resources as well as ineffective or nonexistent patient education about the potential benefits of electronic communication (Goel, Brown, Williams, Cooper, et al., 2011).

Despite the rise of secure electronic messaging, there has been little focus recently on the content of patient messages. Studies of the “early adopters” of electronic messages in the 1990s and early 2000s revealed that adult primary care patients used secure electronic messaging either to request information on medications, symptoms, and disease or for administrative tasks such as prescription refills and appointment scheduling (Delbanco & Sands, 2004; Roter, Larson, Sands, Ford, & Houston, 2008; Sittig, 2003; White, Moyer, Stern, & Katz, 2004). However, over a decade later, secure electronic messaging, and indeed all forms of electronic communication, are now much more prevalent in and relevant to an ever-growing patient base. In addition, the use of secure electronic messaging in underserved patient populations, such as those at safety-net hospital clinics, is virtually unstudied despite the notable linguistic and literacy barriers known to be prevalent in such care settings (“America’s Safety Net Hospitals and Health Systems, 2010,” 2012; Gage & Burch, 2009).

Therefore, we conducted an observational, qualitative e-mail content analysis in an academic, safety-net primary care setting to characterize electronic communication between providers and low-income patients.

Methods

Setting and Design

We conducted an observational study of electronic communication at the General Medicine Clinic at San Francisco General Hospital, a safety-net primary care clinic. The General Medicine Clinic is staffed by academic general internists, internal medicine housestaff, and nurse practitioners that care for a highly diverse, low-income, publicly-insured and uninsured population of about 6,500 unique patients. At the time of this study (January through November of 2013), there was no health-system patient portal in place and thus secure electronic messaging was not available to patients. As a result, all clinic patients had the option to send e-mail to their providers to enhance access between visits. We requested these e-mail correspondences from provider e-mail accounts as part of “usual care,” and we collected exchanges that were forwarded to us. All exchanges were initiated by a patient or caregiver e-mail and included a provider response. We asked providers to forward e-mails from patients and caregivers as well as their responses to investigators.

We received 23 “threads,” defined as e-mails exchanges over time, that contained 31 “strands,” defined as all e-mails related to at least one consistent issue, based on previously described taxonomy (Carrell & Ralston, 2005). One patient had two separate threads with the same provider. E-mails that were part of the same thread were classified as different strands if marked by a clear change in the issues discussed over time. As an example, if a patient replied to a provider e-mail from 30 days prior in order to initiate communication about a new issue, that would be a new “strand.” Provider (attending physicians [n=7], clinical fellows [n=3], residents [n=14], and nurse practitioners [n=7]) strands were with both patients (n=27) and caretakers (n=4; henceforth categorized as “patient(s)”). E-mails were stored on a secure database to safeguard protected health information and subsequently de-identified, removing names and contact information, prior to analysis. The institutional review board at the University of California, San Francisco, reviewed this study and found it to be exempt from committee approval.

Demographic analysis

We assessed the gender and age of patients. We reviewed all medical records to assess whether patients were prescribed at least one medication at the time of the first e-mail. We also determined whether patients had chronic illnesses, which are widely prevalent in our patient population.

Qualitative analysis

Two clinician reviewers (J.M, U.S.) assessed the content of the electronic messages to determine whether the messages contained a medically urgent situation that would have been more safely handled with a physician visit or immediate telephone call.

We then performed a content analysis of the e-mail strands. We employed deductive content analysis (Crabtree & Miller, 1999) incorporating a previously described taxonomy for requests for action or information and results of requests (Anand, Feldman, Geller, Bisbee, & Bauchner, 2005; Kravitz, Bell, & Franz, 1999). We selected only those categories that applied to our data set and combined some categories to simplify the analysis. Because the taxonomies in the literature for electronic messages predate the widespread use of patient portals, we also coded inductively, adding themes as they emerged (Charmaz, 2006). We reached thematic saturation after ten patient-provider e-mail exchanges but elected to continue coding the entire sample.

Qualitative analysis was conducted for classifying patient updates (e.g. symptom changes or results from subspecialty clinics), requests for action (administrative action; appointment; lab tests, x-rays, or other studies; medications or treatments; referral to non-physician; or referral to other physicians) or information (about an appointment; about diagnostic test indications, procedures, interpretations, or results; about drug therapy or nondrug therapy; about insurance, managed care, or financial issues; about physical symptoms, problems, or diseases), and results of requests (administrative paperwork, appointment, dosage change, information or clarification, medical examination, medical guidance, phone call, prescription, specialist consult).

Quantitative analysis

After coding the e-mails, we calculated the frequency of medical updates, requests for action, and requests for information in patient e-mails to providers. We report proportions of requests for actions versus information, the frequency of individual requests, and the frequency of resolved and denied requests. We calculated the relative frequency for each provider action result type. We also report quantitative components of the strands, including the number of e-mails and patient requests (for action or information) per strands as well as the temporal duration of strands (n=29). Descriptive statistics were calculated using Excel.

Results

There were 22 patients in the sample. Fifteen (68%) were female, and the average age was 58 (SD = 10). Review of medical records revealed that 21 (95%) of the 22 patients were prescribed at least one medication at the time of the first e-mail, and all patients had chronic illnesses such as hypertension, diabetes mellitus, hypothyroidism, and osteoarthritis. Clinician reviewers deemed all e-mails to be non-urgent and appropriate for electronic communication. Patients included a medical update in 6 (19% of all) strands. Examples of medical updates included symptom updates such as, “[the rash] itches a bit,” and subspecialty updates such as, “[I] saw the Endocrinologist 2 weeks ago. I think my tiredness and weight gain have to do with my Thyroid. The Endocrinologist disagrees.”

Of all requests (n=56), most were for action (77% of strands and 66% of all requests; Figure 1). Requests for information (n=19) were less common (45% of strands and 34% of all requests; Figure 2). The most common requests overall were for action regarding medications or treatments (n=16); action regarding lab tests, x-rays, other studies (n=7); action regarding a referral to other physicians (n=5); information regarding symptoms (n=5); information regarding tests or diagnostics procedures (n=5); and information regarding medications or treatments (n=5). Examples of requests for action are in Table 1 and examples of information requests are in Table 2.

FIGURE 1.

FIGURE 1

Open in a new tab

Types of requests for action from patients to providers

FIGURE 2.

FIGURE 2

Open in a new tab

Types of requests for information from patients to providers

TABLE 1.

Requests for provider action from patients: sub-types and example quotations

Request for Action Sub-Type Example Quotation
For laboratory testing, imaging tests, or other diagnostic studies “I forgot to bring my blood lab slip and was hoping that one of you could leave one for me at [building section] Ward 92 or GM [(General Medicine)] clinic. I need to do a Thyroid check. So T3, T4, and TSh [sic]. I’m planning to come Friday (tomorrow) mid-day. Please let me know if this is possible.”
For new medication, new form/dose/route of an old medication, refill or renewal of medication taken or prescribed in the past month “Need a refillGabapentin 300mg capsule 3 x daily”
For referral to a physician specialist “Oh one more thing could you get me a referral to orthopedic clinic so I can get the pain shots in my knees.”
For referral to a nonphysician “Also, we spoke about me speaking with someone within the clinic about my disability and dental resources. But I didn’t get any referrals.”
For other services “… my sister said I had some mail From Disability Insurance I been making all my appointment with social security Just waitting [sic] on there Decision I’m Dropping off the paper to you today for the state Disability so it will continue.”
For an appointment “Thank you can I a [sic] early appointment and I lost 45 pounds”
Open in a new tab

TABLE 2.

Requests for information from patients to providers: sub-types and example quotations

Request for Information Sub- Type Example Quotation
About physical symptoms, problems, or diseases “I wanted to ask whether mom ([name]) has hypothyroidism or hyperthyroidism? I believe it was hypo, but wanted to verify with you.”
About diagnostic test indications, procedures, interpretations, or results “I got an appointment with you on Tuesday the 19th. Should I get any blood test before I see you and we could talk about my problem?”
About drug therapy or nondrug therapy “First did you ever get anything from Dr. [Provider 1] or [Provider 2]? It was about the topical anti-inflammatory cream. This has been ever since 10/24/2012. [Provider 2] said she passed this on to you. “
About insurance, managed care, or financial issues “SORRY TO BOTHER YOU. I KNOW YOU ARE VERY BUST [sic]. BUT THESE MEDICARE FOLKS THAT WE TALKED ABOUT LAST WEEK HAVE YOU [sic] TO GET A HOLD OF ME. ITS BEEN A WEEK. AND MY MEDICARE WILL BE EFFECTIVE 04/01 WHICH IS NOT THAT FAR AWAY. WHEN TIME PERMITS COULD YOU PLEASE TRY AGAIN. THANKS YOU ROCK. I LL [sic] CHECK BACK LATER.......”
About an appointment “Question: when will be my next office visit with you.”
Not otherwise classified “Thank you [Provider]! Is [building section] 1m the general medicine clinic?”
Open in a new tab

Of all requests, 84% were resolved, 5% were denied, and 11% had no known outcome detailed in the strand. Providers fulfilled requests as in this example, “I know we have a printed list of sliding scale dentists in the community in our clinic. I will leave a copy of that list at the front desk tonight for you to pick up.” However, providers explicitly denied requests via e-mail as well such as, “In terms of how poorly your symptoms are currently, putting you on chronic Prednisone is not an option because the side effects are too high.” There were a total of 63 resulting actions and one request (a referral to another physician) without a known response within the strand. The most common results of requests (Figure 3) were a prescription (22%), a provider appointment (21%), information or clarification (16%), medical guidance (14%), and administrative paperwork (13%).

FIGURE 3.

FIGURE 3

Open in a new tab

Frequency of all requests

There was an average of 4.4 (SD = 2.5) e-mails per strand, with 19% of strands containing only two e-mails (i.e. 1 initial e-mail and 1 response). There was an average of 1.8 (SD = 1.1) requests per strand, and 45% of all strands contained more than one request. Strands lasted an average of 5.7 days (SD = 8.8) with a range of 0.5 hours to 45 days.

Discussion and Conclusion

Discussion

We completed the first content analysis of electronic messages sent between patients and providers in a safety-net healthcare setting. This is also the first electronic message content analysis performed, to our knowledge, since meaningful use incentives for patient portal use were implemented in the United States. Overall, the taxonomy for coding the electronic message content was consistent with previous literature. However, there were important differences.

The electronic messages that we analyzed were more structurally complex with more requests and e-mails per strand than in prior studies. This chronically ill population also had a higher proportion of medication-related requests than the relatively healthier populations from prior studies (Anand et al., 2005; Sittig, 2003). These differences may reflect the increased health needs and demands of adult, medically-complex, socially-vulnerable populations with economic, cultural, and linguistic access barriers. Such differences may also reflect increased reliance on and trust of electronic communication – in this age of smartphones, Facebook, and Twitter – compared with ten years prior (Fox, 2011).

E-mails were more focused on requesting provider action or information, rather than patients providing symptom updates, than previously described. Two-thirds of all requests in this setting were for provider actions, in contradistinction to prior studies in which the majority of requests were for information. Conversely, only 19% of strands contained an update on patient symptoms, which is significantly lower than in prior studies among parents of pediatric (25% (Anand et al., 2005)) and adult (41% (White et al., 2004)); “two thirds of patient e-mails” (Roter et al., 2008)) populations. This is critical because each request for action generated non-visit work for primary care providers. Earlier studies with fewer requests for action (and more symptom updates from patients) may have thus led to an underestimation of provider time needed for addressing issues raised in electronic communication.

Approximately 80% of our patient requests were resolved, in line with previous estimates. Two of the three requests that were denied in our sample reflected patient safety issues: prescribing topical non-steroidal anti-inflammatory agents in a patient on anticoagulant medications and prescribing chronic steroids for asthma. These instances highlight the importance of ensuring trained medical oversight of electronic communication.

This study further demonstrates that patients use electronic messages for appropriate health-related communication, given that clinician review of e-mails deemed them to be non-urgent. The duration of our strands ranged from less than one hour to 45 days, with an average of almost six days, which similarly reflects the lack of urgency of the messages and the ability of providers to respond to electronic messages in this safety-net primary care setting. This finding suggests that patients were able to assess which issues were appropriate for initiating electronic communication, and that providers require several days to address patient messages.

It is also important to distinguish observational studies of electronic communication, including the present study, from a growing body of randomized control trials for between-visit communication. Previous randomized controlled trials in other healthcare settings have reported that proactive secure electronic messaging, (Green et al., 2014; Green et al., 2008; Ralston, Hirsch, et al., 2009; Simon et al., 2011) as well as telephone support, (Allen et al., 2010; Derose et al., 2013; Huffman et al., 2014; Inglis et al., 2010; Kroenke et al., 2014) can improve disease self-management. However, these trials are often less representative of real-world clinical practice in which providers must balance many competing demands and lacking protected time for proactive communication. Our findings of high levels of medication refills and appointment requests, as opposed to broader self-management support discussions, is consistent with previous studies of between-visit communication outside the context of randomized control trials (Lyles, Grothaus, Reid, Sarkar, & Ralston, 2012).

There were several limitations to analysis in the current study. First, the small sampling size and lack of random sampling limits the direct comparison of our study to large content analyses from the past as well as generalizability to the clinic population. Second, the patients used in this study may themselves likely represent an “early adopter” sub-population of low-income, publicly insured or uninsured patients – particularly those with potentially greater technology access and higher computer literacy – making it additionally challenging to generalize to the clinic population as a whole. Third, the e-mail threads that we analyzed were in English, further limiting the scope of this study and generalizability to the larger multilingual population served in our clinic. Fourth, caregivers were included in this sample as “patients,” which is not truly reflective of those receiving direct care but is representative of the reality that caregivers are involved with online health care (Sarkar et al., 2014). Finally, comparisons with studies of secure electronic message are limited by the fact that non-secure e-mail was used in this study. Although e-mail and secure electronic messaging are distinct entities, e-mail is the best proxy for secure electronic messaging in resource-poor safety-net clinics that lack patient portals.

Conclusion

Patients, their caregivers, and providers in safety-net settings successfully use electronic messages to communicate between visits. We found this communication to be appropriately non-urgent; we did not identify risks to patient safety in the e-mails we reviewed. E-mails predominantly included requests for provider action or information, highlighting the constantly increasing utility of between-visit electronic communication.

Practical Implications

The findings from this study highlight the need for wider implementation of advances in electronic messaging, specifically secure electronic messaging through patient portals. This is especially true in California’s outpatient safety-net clinics, where 72% had not implemented EHR by 2012 (Lopez, Patterson, John, & Sarkar, 2012). Moreover, secure electronic messaging in this setting would benefit greatly from advances beyond the unstructured e-mail communication directly with providers presented here, which pose significant security and patient outcome liabilities. Some health systems route electronic messages to a triage nurse, in a similar fashion to commonly used telephone triage nurses, whose role decreases primary care visits (Bunn, Byrne, & Kendall, 2004). One such example of secure electronic message triaging, the EMAIL System, employs nurses to use subject headings and message content to appropriately route incoming messages, with only 43.4% of messages requiring follow-up by a physician (White et al., 2004). This system resulted in greater e-mail volume to physicians but no difference in phone volume or rates of patient no-shows, with more favorable attitudes from physicians (Katz, Moyer, Cox, & Stern, 2003). Our system could also benefit from an upstream decision component (similar to online customer service features of websites such as Amazon.com) that would alleviate some of the clinic-dependent triage decision-making. In such a system, similar to that used by the Department of Veterans Affairs “My HealtheVet (Haun et al., 2014),” free text would be replaced with drop-down menus to either select a desired recipient or specify “medical updates,” “requests for information,” or “requests for action.” A recent study demonstrates the benefits of patient self-triage in patient portals with improved medication adherence associated with an automated refill request function (Sarkar et al., 2014).

Future studies need to further explore the impact of secure electronic messaging on provider workload, which is one of the main concerns that providers express regarding electronic communication (Hobbs et al., 2003). This will be especially important in safety-net settings, where physician workload is already high (Linzer et al., 2005). Health systems must allot providers enough time to address patient needs that arise electronically and consider incorporating a team-based workflow into secure electronic messaging. It is also unclear how secure electronic messaging will affect face-to-face visits in a safety-net setting given that prior studies demonstrate mixed impacts on in-person utilization (Bergmo, Kummervold, Gammon, & Dahl, 2005; Liss et al., 2014; Palen, Ross, Powers, & Xu, 2012; Shimada et al., 2013; Zhou, Garrido, Chin, Wiesenthal, & Liang, 2007). Finally, given the increasingly broad adoption of this medium across the United States, more studies are needed in varying care settings to elucidate the factors influencing utilization and content of electronic communication.

Acknowledgments

Granting Institution: AHRQ. Grant Number: K99 HS022408-01 (PI: Lyles, Courtney). This project was supported by grant number P30HS023558 (PI Sarkar, Urmimala) from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. The sponsor had no role in study design, data collection, analysis and interpretation of data, in the writing of the report, or in the decision to submit the article for publication.

References

  1. Allen KD, Oddone EZ, Coffman CJ, Datta SK, Juntilla KA, Lindquist JH, … Bosworth HB. Telephone-based self-management of osteoarthritis: A randomized trial. Ann Intern Med. 2010;153(9):570–579. doi: 10.7326/0003-4819-153-9-201011020-00006. [DOI] [PubMed] [Google Scholar]
  2. America’s Safety Net Hospitals and Health Systems, 2010. Washington, D.C: National Association of Public Hospitals and Health Systems; 2012. [Google Scholar]
  3. Anand SG, Feldman MJ, Geller DS, Bisbee A, Bauchner H. A content analysis of e-mail communication between primary care providers and parents. Pediatrics. 2005;115(5):1283–1288. doi: 10.1542/peds.2004-1297. [DOI] [PubMed] [Google Scholar]
  4. Bergmo TS, Kummervold PE, Gammon D, Dahl LB. Electronic patient-provider communication: will it offset office visits and telephone consultations in primary care? Int J Med Inform. 2005;74(9):705–710. doi: 10.1016/j.ijmedinf.2005.06.002. [DOI] [PubMed] [Google Scholar]
  5. Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med. 2010;363(6):501–504. doi: 10.1056/NEJMp1006114. [DOI] [PubMed] [Google Scholar]
  6. Bunn F, Byrne G, Kendall S. Telephone consultation and triage: effects on health care use and patient satisfaction. Cochrane Database Syst Rev. 2004;(4):Cd004180. doi: 10.1002/14651858.CD004180.pub2. [DOI] [PubMed] [Google Scholar]
  7. Carrell D, Ralston J. Messages, strands and threads: measuring use of electronic patient-provider messaging. AMIA Annu Symp Proc. 2005:913. [PMC free article] [PubMed] [Google Scholar]
  8. Charmaz K. Constructig Grounded Theory: A Practical Guide through Qualitative Analysis. 1. Thousand Oaks, California: SAGE Publications; 2006. [Google Scholar]
  9. Crabtree BF, Miller WL. Doing Qualitative Research. 2. London, England: SAGE Publications; 1999. [Google Scholar]
  10. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: Institute of Medicine; 2001. [PubMed] [Google Scholar]
  11. Delbanco T, Sands DZ. Electrons in flight--e-mail between doctors and patients. N Engl J Med. 2004;350(17):1705–1707. doi: 10.1056/NEJMp038209. [DOI] [PubMed] [Google Scholar]
  12. Derose SF, Green K, Marrett E, Tunceli K, Cheetham TC, Chiu VY, … Scott RD. Automated outreach to increase primary adherence to cholesterol-lowering medications. JAMA Intern Med. 2013;173(1):38–43. doi: 10.1001/2013.jamainternmed.717. [DOI] [PubMed] [Google Scholar]
  13. Fox S. The Social Life of Health Information, 2011. Washington, D.C: 2011. [Google Scholar]
  14. Gage LS, Burch CC. Massachusetts Health Reform: Lessons Learned about the Critical Role of Safety Net Health Systems. Washington, D.C: National Association of Public Hospitals and Health Systems; 2009. [Google Scholar]
  15. Gaster B, Knight CL, DeWitt DE, Sheffield JV, Assefi NP, Buchwald D. Physicians’ use of and attitudes toward electronic mail for patient communication. J Gen Intern Med. 2003;18(5):385–389. doi: 10.1046/j.1525-1497.2003.20627.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Goel MS, Brown TL, Williams A, Cooper AJ, Hasnain-Wynia R, Baker DW. Patient reported barriers to enrolling in a patient portal. J Am Med Inform Assoc. 2011;18(Suppl 1):i8–12. doi: 10.1136/amiajnl-2011-000473. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Goel MS, Brown TL, Williams A, Hasnain-Wynia R, Thompson JA, Baker DW. Disparities in enrollment and use of an electronic patient portal. J Gen Intern Med. 2011;26(10):1112–1116. doi: 10.1007/s11606-011-1728-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Green BB, Anderson ML, Cook AJ, Catz S, Fishman PA, McClure JB, Reid RJ. e-Care for heart wellness: a feasibility trial to decrease blood pressure and cardiovascular risk. Am J Prev Med. 2014;46(4):368–377. doi: 10.1016/j.amepre.2013.11.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Green BB, Cook AJ, Ralston JD, Fishman PA, Catz SL, Carlson J, … Thompson RS. Effectiveness of home blood pressure monitoring, Web communication, and pharmacist care on hypertension control: a randomized controlled trial. Jama. 2008;299(24):2857–2867. doi: 10.1001/jama.299.24.2857. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Harris LT, Haneuse SJ, Martin DP, Ralston JD. Diabetes quality of care and outpatient utilization associated with electronic patient-provider messaging: a cross-sectional analysis. Diabetes Care. 2009;32(7):1182–1187. doi: 10.2337/dc08-1771. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Haun JN, Lind JD, Shimada SL, Martin TL, Gosline RM, Antinori N, … Simon SR. Evaluating user experiences of the secure messaging tool on the Veterans Affairs’ patient portal system. J Med Internet Res. 2014;16(3):e75. doi: 10.2196/jmir.2976. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Health Information Technology for Economic and Clinical Health (HITECH) Act, Title XIII of Division A and Title IV of Division B of the American Recovery and Reinvestment Act C.F.R. (2009).
  23. Health Information Technology. Final Rule. 2010. Initial Set of Standards, Implementation Specifications, and Certification Criteria for Electronic Health Record Technology. [PubMed] [Google Scholar]
  24. Hobbs J, Wald J, Jagannath YS, Kittler A, Pizziferri L, Volk LA, … Bates DW. Opportunities to enhance patient and physician e-mail contact. Int J Med Inform. 2003;70(1):1–9. doi: 10.1016/s1386-5056(03)00007-8. [DOI] [PubMed] [Google Scholar]
  25. Huffman JC, Mastromauro CA, Beach SR, Celano CM, DuBois CM, Healy BC, … Januzzi JL. Collaborative care for depression and anxiety disorders in patients with recent cardiac events: the Management of Sadness and Anxiety in Cardiology (MOSAIC) randomized clinical trial. JAMA Intern Med. 2014;174(6):927–935. doi: 10.1001/jamainternmed.2014.739. [DOI] [PubMed] [Google Scholar]
  26. Inglis SC, Clark RA, McAlister FA, Ball J, Lewinter C, Cullington D, … Cleland JG. Structured telephone support or telemonitoring programmes for patients with chronic heart failure. Cochrane Database Syst Rev. 2010;(8):Cd007228. doi: 10.1002/14651858.CD007228.pub2. [DOI] [PubMed] [Google Scholar]
  27. Katz SJ, Moyer CA, Cox DT, Stern DT. Effect of a triage-based E-mail system on clinic resource use and patient and physician satisfaction in primary care: a randomized controlled trial. J Gen Intern Med. 2003;18(9):736–744. doi: 10.1046/j.1525-1497.2003.20756.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Kravitz RL, Bell RA, Franz CE. A taxonomy of requests by patients (TORP): a new system for understanding clinical negotiation in office practice. J Fam Pract. 1999;48(11):872–878. [PubMed] [Google Scholar]
  29. Kroenke K, Krebs EE, Wu J, Yu Z, Chumbler NR, Bair MJ. Telecare collaborative management of chronic pain in primary care: a randomized clinical trial. Jama. 2014;312(3):240–248. doi: 10.1001/jama.2014.7689. [DOI] [PubMed] [Google Scholar]
  30. Liederman EM, Lee JC, Baquero VH, Seites PG. The impact of patient-physician Web messaging on provider productivity. J Healthc Inf Manag. 2005;19(2):81–86. [PubMed] [Google Scholar]
  31. Linzer M, Baier Manwell L, Mundt M, Williams E, Maguire A, McMurray J, Plane MB. Advances in Patient Safety Organizational Climate, Stress, and Error in Primary Care: The MEMO Study. In: Henriksen K, Battles JB, Marks ES, Lewin DI, editors. Advances in Patient Safety: From Research to Implementation (Volume 1: Research Findings) Rockville (MD): Agency for Healthcare Research and Quality (US); 2005. [PubMed] [Google Scholar]
  32. Liss DT, Reid RJ, Grembowski D, Rutter CM, Ross TR, Fishman PA. Changes in Office Visit Use Associated With Electronic Messaging and Telephone Encounters Among Patients With Diabetes in the PCMH. Ann Fam Med. 2014;12(4):338–343. doi: 10.1370/afm.1642. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Lopez A, Patterson AM, John V, Sarkar U. Electronic health record implementation in outpatient safety-net settings in California. J Health Care Poor Underserved. 2012;23(4):1421–1430. doi: 10.1353/hpu.2012.0154. [DOI] [PubMed] [Google Scholar]
  34. Lyles CR, Grothaus L, Reid RJ, Sarkar U, Ralston JD. Communication about diabetes risk factors during between-visit encounters. Am J Manag Care. 2012;18(12):807–815. [PubMed] [Google Scholar]
  35. Palen TE, Ross C, Powers JD, Xu S. Association of online patient access to clinicians and medical records with use of clinical services. Jama. 2012;308(19):2012–2019. doi: 10.1001/jama.2012.14126. [DOI] [PubMed] [Google Scholar]
  36. Pearl R. Kaiser Permanente Northern California: current experiences with internet, mobile, and video technologies. Health Aff (Millwood) 2014;33(2):251–257. doi: 10.1377/hlthaff.2013.1005. [DOI] [PubMed] [Google Scholar]
  37. Ralston JD, Hirsch IB, Hoath J, Mullen M, Cheadle A, Goldberg HI. Web-based collaborative care for type 2 diabetes: a pilot randomized trial. Diabetes Care. 2009;32(2):234–239. doi: 10.2337/dc08-1220. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Ralston JD, Rutter CM, Carrell D, Hecht J, Rubanowice D, Simon GE. Patient use of secure electronic messaging within a shared medical record: a cross-sectional study. J Gen Intern Med. 2009;24(3):349–355. doi: 10.1007/s11606-008-0899-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Rosen P, Kwoh CK. Patient-physician e-mail: an opportunity to transform pediatric health care delivery. Pediatrics. 2007;120(4):701–706. doi: 10.1542/peds.2007-1094. [DOI] [PubMed] [Google Scholar]
  40. Roter DL, Larson S, Sands DZ, Ford DE, Houston T. Can e-mail messages between patients and physicians be patient-centered? Health Commun. 2008;23(1):80–86. doi: 10.1080/10410230701807295. [DOI] [PubMed] [Google Scholar]
  41. Sarkar U, Lyles CR, Parker MM, Allen J, Nguyen R, Moffet HH, … Karter AJ. Use of the refill function through an online patient portal is associated with improved adherence to statins in an integrated health system. Med Care. 2014;52(3):194–201. doi: 10.1097/mlr.0000000000000069. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Schickedanz A, Huang D, Lopez A, Cheung E, Lyles CR, Bodenheimer T, Sarkar U. Access, interest, and attitudes toward electronic communication for health care among patients in the medical safety net. J Gen Intern Med. 2013;28(7):914–920. doi: 10.1007/s11606-012-2329-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Shimada SL, Hogan TP, Rao SR, Allison JJ, Quill AL, Feng H, … Houston TK. Patient-provider secure messaging in VA: variations in adoption and association with urgent care utilization. Med Care. 2013;51(3 Suppl 1):S21–28. doi: 10.1097/MLR.0b013e3182780917. [DOI] [PubMed] [Google Scholar]
  44. Simon GE, Ralston JD, Savarino J, Pabiniak C, Wentzel C, Operskalski BH. Randomized trial of depression follow-up care by online messaging. J Gen Intern Med. 2011;26(7):698–704. doi: 10.1007/s11606-011-1679-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Sittig DF. Results of a content analysis of electronic messages (email) sent between patients and their physicians. BMC Med Inform Decis Mak. 2003;3:11. doi: 10.1186/1472-6947-3-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. The Value of Personal Health Records and Web Portals to Engage Consumers and Improve Quality. Princeton, NJ: Robert Wood Johnson Foundation; 2012. [Google Scholar]
  47. White CB, Moyer CA, Stern DT, Katz SJ. A content analysis of e-mail communication between patients and their providers: patients get the message. J Am Med Inform Assoc. 2004;11(4):260–267. doi: 10.1197/jamia.M1445. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Zhou YY, Garrido T, Chin HL, Wiesenthal AM, Liang LL. Patient access to an electronic health record with secure messaging: impact on primary care utilization. Am J Manag Care. 2007;13(7):418–424. [PubMed] [Google Scholar]
  49. Zhou YY, Kanter MH, Wang JJ, Garrido T. Improved quality at Kaiser Permanente through e-mail between physicians and patients. Health Aff (Millwood) 2010;29(7):1370–1375. doi: 10.1377/hlthaff.2010.0048. [DOI] [PubMed] [Google Scholar]

RESOURCES