Abstract
Background
Previous studies of patient portals have found low rates of enrollment and significant disparities in enrollment by race and ethnicity. As the reasons for these findings are unclear, we sought to identify patient reported barriers to enrollment in a patient portal.
Methods
We conducted a telephone survey of patients in one urban general internal medicine clinic. Patients were eligible if they did not enroll within 30 days of receiving an electronic order inviting participation. Our primary outcomes were: (a) reasons for not enrolling in the patient portal; (b) reasons for not attempting enrollment; and (c) perceived benefits of the portal.
Results
Participants' (N=159) mean age was 51 years, 48% were black, 72% female, and 70% had a college degree or greater. 63% of respondents not enrolling reported never attempting enrollment despite remembering receiving an order. Most of these 63% did not attempt enrollment because of lack of information or motivation. Smaller proportions reported not attempting enrollment because of negative attitudes toward the portal (30%) or computer related obstacles (8%). Overall, respondents favorably viewed most patient portal features, however black respondents were less likely than white respondents to consider features assisting self-management such as getting test results (69% vs 86%; p<0.05) as important. Adjusting for age, gender, education, and chronic disease did not substantially change results.
Conclusion
Strategies to increase enrollment in patient portals need to ensure patients understand patient portal features and receive follow-up reminders. Interventions to reduce racial disparities in enrollment must address attitudinal barriers and not focus solely on improving access.
Keywords: Racial/ethnic disparities, electronic health record, patient portal, personal health record, disparities, patient portal, quality measure, clinical decision support, chronic disease management, patient engagement, health literacy, machine learning, predictive modeling, statistical learning, privacy technology
Introduction
The use of electronic health records (EHRs) is increasing over time1 and provides many opportunities for enhancing patient–provider communication. Patient portals, also known as tethered personal health records, are electronic entry points into the EHR and have many potential functions which can enhance communication. Many allow patients to send secure messages to providers, access test results, refill medications, schedule appointments, and even participate in unique disease management programs. As such, they have the potential to promote patient empowerment,2 increase satisfaction with care,3–5 and perhaps even increase the timeliness, quality, and patient-centeredness of healthcare delivery. With the use of federal subsidies incenting providers to achieve meaningful use of EHRs, patient portals are likely to play an increasing role in healthcare delivery.6 7
Despite their potential benefits, patient portals have generally low rates of enrollment.8 Furthermore, several studies have found significant disparities in enrollment by race and ethnicity.9–13 After patients enroll, however, disparities in the use of the patient portal are no longer clearly evident.8 12 14 While the reasons for low overall enrollment and disparities are not known, several possibilities have been proposed. One potential cause is lack of access to the internet (ie, the digital divide); however, disparities in portal use have been demonstrated even in geographic areas where disparities in access to the internet do not exist.12 15 Variations in computer literacy or technical skills, differences in patient attitudes toward communicating with providers electronically, and patient preferences for access to their individual health information may also be contributing factors.16–18 To date, studies have not been able to identify causes of non-enrollment because they have relied primarily on chart abstractions or on broad-based surveys examining general patient attitudes toward this technology. These studies have been valuable in identifying patterns of enrollment and factors associated with non-enrollment, but they have not been able to establish causation.
Therefore, we conducted telephone surveys with patients to identify patient reported barriers to enrollment among patients who did not enroll despite being directly offered this service by their providers.
Methods
Setting and participants
This study was conducted at the Northwestern Medical Faculty Foundation, the group practice for the full-time faculty of the Feinberg School of Medicine of Northwestern University. MyChart, the commercial patient portal used by the practice (EpicCare, version Spring 2007; Epic Systems, Verona, Wisconsin, USA), allows a patient to log-on to a secure portal to access personalized health information, including laboratory results and a medication list. Patients can also send secure electronic messages to physicians. In this clinic, physicians must place an electronic order inviting patient enrollment. After the order is placed, the process for enrollment requires that: (a) a paper print-out with instructions for enrollment and a unique access code be given to each patient by clinic staff; (b) patients have access to a computer with internet access; and (c) patients successfully complete the instructions for enrollment.
Participants were eligible if they had an attending physician within the general internal medicine clinic, two visits in the past 18 months, an electronic order to enroll placed between January 2009 and March 2010, and had not enrolled within 30 days after the order was placed. The 30-day enrollment period was established based on internal reports showing a decrement in enrollment rates after this period of time. Interviews were conducted between February and June 2010.
Based on findings from our previous study demonstrating racial and ethnic disparities in enrollment in this practice,12 we sought to ensure adequate numbers of interviews with minority patients. We generated a list of eligible white, black, and Latino patients, in random order. We initially aimed to sample 100 participants from each racial and ethnic group and interviewed all eligible black and Latino patients and a random sample of white patients; however, we were unable to achieve adequate sampling of Latino patients to provide meaningful evaluation. As a result, we present data only for white and black participants. We provided a $10 gift card incentive to individuals who completed the survey. This study protocol was approved by the Northwestern University Institutional Review Board. The survey response rate was 65%, inclusive of white, black, and Latino participants.19
Telephone survey
After providing respondents with general information about the patient portal at Northwestern, we first asked participants if they recalled discussing the patient portal with their provider. Those who recalled a discussion with their providers were asked if they attempted to enroll and reasons why they did not attempt to enroll or about experiences with attempting to enroll, as appropriate. In addition to reasons for non-enrollment, all participants were asked questions from the following domains: (a) personal access to and use of the internet; (b) perceived benefits of the patient portal; and (c) demographic characteristics. We developed this telephone survey based on earlier studies of barriers to enrollment,9 11–13 general attitudes toward the use of patient portals and the internet,20 as well as informal feedback from patients in our clinic.
The survey contained both closed (ie, discrete choice) and open-ended response options and required 15 min to administer. It incorporated a branching pattern dependent upon responses to individual items; therefore the final questionnaire ranged from 28 to 43 questions.
Study measures
The main study measures of interest were: (a) reasons for non-enrollment; (b) reasons for not attempting enrollment among those who remembered discussing the portal with their providers; and (c) perceived benefits of the patient portal.
Reasons for non-enrollment were coded based on responses to two questions: (a) participants recalling a discussion with their providers about the patient portal (Y/N); and (b) participants attempting enrollment among those who remembered a discussion with their provider (Y/N). The three categories of responses were mutually exclusive: (a) did not remember discussion with the provider; (b) remembered discussion but did not attempt to enroll; and (c) remembered discussion and attempted to enroll.
Primary reasons for not attempting enrollment were elicited only from those participants who recalled a discussion about the patient portal with their providers yet did not attempt to enroll. We asked the open-ended question, ‘What was the main reason you didn't try to activate your account?’ We initially developed response categories based on a priori hypotheses and pilot interviews. Free text responses were then reviewed by three members of the research team (TB, AW, MSG) to revise the coding scheme based on latent content analysis.21 Based on this review, we added five categories retrospectively: (a) did not know about enrollment requirements/did not have enrollment instructions; (b) forgot to enroll; (c) too sick at the time; (d) encountered internet or computer obstacles; and (e) lacked the computer skills needed to enroll. Two members of the research team (TB, AW) subsequently reviewed and coded all content independently according to the coding scheme; the κ statistic was 0.81. For responses with coding disagreement, the principal investigator also reviewed the responses (MSG) and assigned a final classification. Finally, we collapsed the main reasons participants did not enroll in the portal into three main categories: lack of information or motivation, negative attitudes, and connectivity obstacles. χ2 Tests were performed on these collapsed categories.
After ascertaining the main reason for not attempting to enroll, we asked participants in the subset who did not attempt enrollment, whether specific factors influenced their decision. We chose to ask about specific factors to examine the role of barriers proposed in extant literature. For this portion of the survey, we asked eight questions with multiple choice response categories.
We also asked all participants about the perceived benefits of the patient portal using a 3-point Likert scale. Responses were later dichotomized as very important/important versus unimportant.
Lastly, because the presence of chronic medical conditions might influence patients' barriers to enrollment as well as perceived benefits of the patient portal, we assessed comorbid conditions using a modified version of the 2008 Dartmouth Atlas of Health. In addition to eight conditions included in the original categories of chronic disease (ie, chronic pulmonary disease, coronary artery disease, congestive heart failure, peripheral vascular disease, severe chronic liver disease, diabetes with end-organ damage, renal failure, and dementia), we broadened the diabetes category to include anyone diagnosed with diabetes (not only those with end organ damage) and included hypertension. We included these last two categories because they are among the most common general medical conditions that may require self-management. Because of the distribution of chronic conditions, we dichotomized chronic conditions into zero and one or more chronic conditions for analyses.
Statistical analysis
Analyses were performed in SAS (V.9.2; SAS Institute). In addition to descriptive measures, we compared differences in responses for white and black respondents using χ2 statistics. If race data were missing or marked as a refusal, the participant was not included in the analyses. In order to analyze potential confounding, we performed multivariate logistic regression analyses adjusting for age (continuous), gender, education (dichotomous), and chronic disease (dichotomous yes/no) for all outcomes of interest. Differences were considered significant if the p value was <0.05 based on two-sided tests.
Results
Participants' (N=159) mean age was 51 years; 48% were black, 72% were female, and 70% had a college degree or greater (table 1).
Table 1.
Participants (N=159) | |
Mean age, years (SD) | 51 (14) |
Race/ethnicity, n (%) | |
African American | 76 (48) |
Caucasian | 83 (52) |
Female gender, n (%) | 114 (72) |
Educational attainment: college degree or above, n (%) | |
African American | 37 (49) |
Caucasian | 72 (88) |
Presence of chronic medical conditions,* n (%) | |
African American | 55 (62) |
Caucasian | 34 (38) |
Presence of chronic medical conditions is based on how subjects were classified in 10 chronic disease categories, eight of which were included in the 2008 Dartmouth Atlas of Health Care (cancer, chronic pulmonary disease, coronary artery disease, congestive heart failure, peripheral vascular disease, severe chronic liver disease, renal failure, and dementia) plus hypertension and all diabetes.
Reasons for not enrolling in the patient portal
Twenty-six percent of respondents did not remember discussing the patient portal with their providers, 63% did not attempt enrollment despite remembering a discussion with their providers, and 11% attempted to enroll but did not succeed. Although black respondents were twice as likely to report unsuccessfully attempting to enroll, differences by race were not significant (table 2).
Table 2.
Total (n=159) | Black (n=76) | White (n=83) | p Value | |
Doesn't recall provider offering portal, n (%) | 41 (26) | 19 (25) | 23 (28) | 0.35 |
Remembered provider order, but did not attempt enrollment, n (%) | 100 (63) | 46 (61) | 54 (65) | |
Attempted enrollment but did not succeed, n (%) | 18 (11) | 11 (15) | 6 (7) |
Primary reason for not attempting enrollment
Among the 63% of participants who did not attempt to enroll in the patient portal, 60% stated reasons related to lack of information or motivation, 30% reported negative attitudes toward the patient portal, and 8% reported connectivity obstacles. There were large, but non-significant differences in reasons for not attempting enrollment by race (table 3). Black respondents were more likely to cite negative attitudes as the primary reason for not attempting enrollment compared with white respondents (32% vs 28%). More specifically, 25% of black respondents did not feel that the patient portal would be useful, compared with 15% of white respondents. Furthermore, black respondents were more likely to cite connectivity obstacles as a barrier than white respondents (14% vs 4%, respectively). None of these differences were statistically significant. Although there were large differences in reasons for not attempting enrollment by presence of chronic disease (lack of information/motivation was cited by 55% with chronic disease vs 71% without chronic disease), the relationship between race and reasons for not attempting enrollment remained non-significant in analyses adjusting for chronic disease, age, gender, and education.
Table 3.
Total (n=98) | Black (n=44) | White (n=54) | |
Lack of information/motivation, n (%)† | 61 (62) | 24 (55) | 37 (69) |
Didn't know about or have enrollment instructions | 24 (24) | 11 (25) | 13 (24) |
Forgot/lazy/busy | 29 (30) | 8 (18) | 21 (39) |
Thought it would take too much time | 7 (7) | 4 (9) | 3 (6) |
Too sick at the time | 1 (1) | 1 (2) | 0 |
Negative attitudes, n (%)† | 29 (30) | 14 (32) | 15 (28) |
Didn't think it would be useful | 19 (19) | 11 (25) | 8 (15) |
Prefer phone over email | 7 (7) | 2 (5) | 5 (9) |
Thought it would be too complicated | 1 (1) | 0 | 1 (2) |
Internet is not a safe way to communicate | 2 (2) | 1 (2) | 1 (2) |
Connectivity obstacles, n (%)† | 8 (8) | 6 (14) | 2 (4) |
No computer/internet access | 5 (5) | 4 (9) | 1 (2) |
Lack of computer skills | 3 (3) | 2 (5) | 1 (2) |
Among respondents who remembered patient portal orders but did not attempt enrollment, missing data from two black respondents.
χ2 Test comparing collapsed categories (ie, Lack of information/motivation, Negative attitudes, and Connectivity obstacles) for not attempting to enroll by race, p=0.16.
Additional reasons for not attempting enrollment
We asked respondents who never attempted to enroll (N=100) about the contribution of eight specific factors to their decision not to enroll. Among the eight factors, preference and security concerns were the two most commonly cited barriers to enrollment. Thirty-seven percent said they prefer to call the provider's office to discuss health matters rather than communicate electronically and nearly 25% reported they did not feel the internet is a safe way to communicate sensitive health information. The prevalence of these factors did not vary by race. Importantly, obstacles related to technology were infrequently cited as contributing factors; only 4% of participants reported lack of access to the internet as a barrier, and only 10% thought the patient portal would be too complicated to use. Neither of these factors varied by race.
Perceived benefits of the patient portal
Respondents felt most features offered by a patient portal were important or very important to them, with the exception of sharing medical records with family members (table 4). There were important variations in the perceived benefits of the patient portal by race. Black respondents were less likely than white respondents to endorse as important or very important features that assisted self-management, including getting test results (69% vs 86%; p<0.05), managing medical problems (58% vs 82%; p<0.05), and seeing when you are due for screening tests (72% vs 88%; p<0.05). There were no significant differences in attitudes toward administrative tasks such as scheduling appointments or refilling medications. Analyses adjusting for age, gender, education, and chronic disease did not substantially change the results by race. For example, white patients remained significantly more likely to perceive features that assist self-management as important or very important in both unadjusted (OR 3.43; 95% CI 1.63 to 7.19) and adjusted analyses (OR 3.95; 95% CI 1.64 to 9.52).
Table 4.
Participants rating a feature as very important or important, % | Total (n=159) | Black (n=76) | White (n=83) | p Value |
Self-management features, n (%) | ||||
Manage your medical problems | 111 (70) | 44 (58) | 68 (82) | 0.00 |
See when due for screening tests | 127 (80) | 56 (72) | 73 (88) | 0.01 |
Get your test results | 122 (77) | 52 (69) | 71 (86) | 0.01 |
Ask your doctor about new medical problems | 112 (71) | 51 (67) | 63 (76) | 0.24 |
Ask questions about your medical problems | 129 (81) | 58 (76) | 71 (86) | 0.11 |
Get a list of medications and doses | 114 (72) | 56 (73) | 59 (71) | 0.81 |
Administrative features, n (%) | ||||
Schedule appointments online | 116 (73) | 52 (69) | 65 (79) | 0.15 |
Refill medications online | 118 (74) | 52 (69) | 66 (80) | 0.13 |
Share your medical record with other physicians | 134 (84) | 61 (80) | 73 (88) | 0.22 |
Share your medical record with family | 68 (43) | 32 (42) | 37 (45) | 0.83 |
Among all respondents, regardless of whether they remembered patient portal orders or attempted enrollment.
Discussion
Most participants who did not enroll in an electronic patient portal after receiving an electronic order from their providers never attempted to enroll despite rating many features of the patient portal as very important or important. Fewer participants reported not enrolling because they did not remember being offered enrollment or had attempted to enroll but were unable to successfully complete the required steps. Most participants who did not attempt to enroll cited factors related to lack of information or motivation as the main barrier to enrollment. Although there were no differences in barriers to enrollment by race, black participants were significantly less likely to perceive importance in patient portal features related to self-management. Importantly, lack of access to computers or the internet was rarely cited as a barrier to enrollment in this population, and did not differ by race.
To date, most studies have used EHR data to identify factors associated with low enrollment. These studies have cited lack of access to computers or the internet9 11 14 as major barriers to enrollment and hypothesized about the role of negative attitudes toward the patient portal. Other studies have examined general attitudes toward patient portals in broad patient populations and found concerns about privacy and security with the use of electronic communications22; however, it is not clear whether these concerns would prevent patients from enrolling in a real-world setting. To our knowledge, this is the first study to directly engage patients who had an opportunity to enroll in a patient portal and ask for their individual reasons for not enrolling.
We found several important results that may inform future efforts to expand and enhance patient portal use. First, nearly one-quarter of participants invited to enroll in the patient portal did not recall discussing the portal with their providers or receiving information about enrollment. This finding is consistent with prior studies showing that patients do not accurately recall verbal communications during office visits.23 24 As many different topics may be covered during a visit, patients may not recall a discussion about the patient portal. In order to enhance enrollment, patients must be made aware of what a patient portal is and how it could benefit them directly. Providing multiple opportunities to learn about the portal by giving written information at the time of a clinical visit and sending electronic information about the portal and enrollment at periodic intervals following a visit may help overcome this barrier.
Second, most participants never attempted to enroll, despite remembering a discussion with their providers. The most common reasons cited for not attempting enrollment were related to lack of information or motivation, particularly being ‘too busy’ or ‘forgetful’ or ‘lazy’ to enroll. Lack of motivation itself may arise for many different reasons including low perceived importance of the portal and inability to fit enrollment into one's daily workflow. Strategies to maximize enrollment need to increase the perceived importance of the portal to patients. Thus, clearly communicating the features and benefits of the patient portal within the context of a clinical visit (eg, if you sign up for the patient portal, you can send the physician an electronic message in the next week to update her on how you are feeling), may increase the perceived importance of patient portal enrollment. Furthermore, providing multiple opportunities for enrollment will allow for easier integration into a patient's home workflow. Allowing onsite enrollment or sending e-mails containing a link to the enrollment web page within 1 week of receiving a physician order might boost enrollment rates. Lastly, simplifying enrollment processes and instructions might further overcome barriers related to lack of information or motivation.
Third, although participants valued many features offered by the patient portal, one-third of participants who did not attempt to enroll reported negative attitudes toward the patient portal. Most of these participants felt that the patient portal would not be useful to them. The disconnect between this negative attitude and the overall perceived importance of many features of the patient portal highlights the importance of communicating the portal's features and potential benefits. Many patients may not understand the features available in the patient portal they are being offered. Providing standardized, relevant information at the time enrollment is offered may counter some negative attitudes about the portal and motivate enrollment. In addition, concerns about the safety and privacy of the internet for health communications negatively influenced enrollment in nearly 25% of the survey population. Specific messages conveying the rigorous safety measures taken to secure the privacy of the EHRs may help allay concerns.
Lastly, reasons for racial disparities in patient portal enrollment are likely multifactorial. Addressing the causes of these disparities is essential to preventing future disparities in health and healthcare as patient portals become a more important vehicle for the management of health and illness. In contrast to earlier studies hypothesizing that distrust and differential access to technology (ie, the digital divide) might be the primary drivers of disparities,9 11 13 14 few participants cited distrust or access as the primary barrier to enrollment. This may have been for a few reasons. The participants in this survey had already been offered access to the patient portal by their providers. Patients are more likely to change behavior if advised by their physician (eg, patients are more likely to quit smoking after receiving smoking cessation advice from their physicians25); therefore, they may be more amenable to enrolling in a technology endorsed by their physicians. Furthermore, because providers were tasked with writing the electronic orders for their patients, providers may have discussed the patient portal with patients and only written orders if patients reported having access to the internet. Respondents in our survey rarely cited lack of access to the internet or a computer as barriers to enrollment; this finding suggests that the digital divide does not fully explain racial/ethnic disparities in patient portal enrollment. The most striking racial/ethnic differences seen in this study were in the perceived importance of various patient portal features. White respondents were significantly more likely to perceive importance in patient portal features related to self-management than black respondents, despite the higher prevalence of chronic conditions among the black study patients. This finding is congruent with previous studies showing that African American patients are less likely to feel control over their health.26 It is unclear whether this finding explains racial differences in enrollment; however, efforts to market the patient portal to diverse populations should consider using messages that appeal broadly to patients, such as the importance of communicating healthcare information to other healthcare providers, rather than focusing on messages of self-efficacy and management that disproportionately appeal to only some patients.
Our study has several limitations; most notably, our survey was conducted at one primary care site. Consequently, our results may not be generalizable to other settings where patient portals are in use. Additionally, we do not know the content of communications between participants and their providers at the time the patient portal order was placed. It is likely that some physicians spent more time explaining and highlighting the benefits of portal use than others and this might have influenced patient reported barriers to enrollment. Lastly, participants may have felt compelled to provide desirable responses to survey questions because the survey was conducted by the same institution where they seek their care. We would expect that this would most strongly influence responses to questions eliciting primary reasons for non-enrollment and perceived benefits of the patient portal.
In summary, we found that among participants who were explicitly invited to enroll in a patient portal by their physicians, most report positive attitudes toward the benefits of patient portals but describe lack of awareness of the patient portal or lack of motivation as the primary barriers to enrollment. We also found significant differences in the perceived benefits of the patient portal by race. This suggests that addressing technical barriers alone would be insufficient to reduce disparities in enrollment. Further research is needed to develop interventions that address identified barriers and successfully motivate enrollment of diverse patient populations in patient portals.
Footnotes
Funding: This research was supported entirely by internal funds.
Competing interests: None.
Ethics approval: The Northwestern University Institutional Review Board approved the study protocol.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1.Hsiao CJ, Hing E, Socey TC, et al. Electronic Medical Record/Electronic Health Record Systems of Office-based Physicians: United States, 2009 and Preliminary 2010 State Estimates. NCHS Health E-Stat [Internet], 2010. Available from http://www.cdc.gov/nchs/data/hestat/emr_ehr_09/emr_ehr_09.htm [Google Scholar]
- 2.Hassol A, Walker JM, Kidder D, et al. Patient experiences and attitudes about access to a patient electronic health care record and linked web messaging. J Am Med Inform Assoc 2004;11:505–13 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Lin CT, Wittevrongel L, Moore L, et al. An Internet-based patient-provider communication system: randomized controlled trial. J Med Internet Res 2005;7:e47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Schnipper JL, Gandhi TK, Wald JS, et al. Design and implementation of a web-based patient portal linked to an electronic health record designed to improve medication safety: the Patient Gateway medications module. Inform Prim Care 2008;16:147–55 [DOI] [PubMed] [Google Scholar]
- 5.Detmer D, Bloomrosen M, Raymond B, et al. Integrated personal health records: transformative tools for consumer-centric care. BMC Med Inform Decis Mak 2008;8:45. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Halamka JD, Mandl KD, Tang PC. Early experiences with personal health records. J Am Med Inform Assoc 2008;15:1–7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Health IT Policy Committee Electronic health records and meaningful use. http://healthit.hhs.gov/portal/server.pt?CommunityID=1472&spaceID=14&parentname=&control=SetCommunity&parentid=&in_hi_userid=11673&PageID=0&space=CommunityPage (accessed 29 Jun 2011).
- 8.Ancker JS, Barron Y, Rockoff ML, et al. Use of an electronic patient portal among disadvantaged populations. J Gen Intern Med 2011;26:1117–23 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Hsu J, Huang J, Kinsman J, et al. Use of e-Health services between 1999 and 2002: a growing digital divide. J Am Med Inform Assoc 2005;12:164–71 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Miller H, Vandenbosch B, Ivanov D, et al. Determinants of personal health record use: a large population study at Cleveland Clinic. J Healthc Inf Manag 2007;21:44–8 [PubMed] [Google Scholar]
- 11.Roblin DW, Houston TK, 2nd, Allison JJ, et al. Disparities in use of a personal health record in a managed care organization. J Am Med Inform Assoc 2009;16:683–9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Goel MS, Brown TL, Williams A, et al. Disparities in enrollment and use of an electronic patient portal. J Gen Intern Med 2011;26:1112–16 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Sarkar U, Karter AJ, Liu JY, et al. The literacy divide: health literacy and the use of an internet-based patient portal in an integrated health system-results from the diabetes study of northern California (DISTANCE). J Health Commun 2010;15(Suppl 2):183–96 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Yamin CK, Emani S, Williams DH, et al. The digital divide in adoption and use of a personal health record. Arch Intern Med 2011;171:568–74 [DOI] [PubMed] [Google Scholar]
- 15.Mossberger K, Tolbert C. Digital Excellence in Chicago: A City Wide View of Technology Use. Chicago, 2009. Available from http://www.cityofchicago.org/dam/city/depts/doit/supp_info/DEI/Digital_Excellence_Study_2009.pdf [Google Scholar]
- 16.Greenhalgh T, Wood GW, Bratan T, et al. Patients' attitudes to the summary care record and Health Space: qualitative study. BMJ 2008;336:1290–5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Wilson EV, Lankton NK. Modeling patients' acceptance of provider-delivered e-health. J Am Med Inform Assoc 2004;11:241–8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Pyper C, Amery J, Watson M, et al. Patients' experiences when accessing their on-line electronic patient records in primary care. Br J Gen Pract 2004;54:38–43 [PMC free article] [PubMed] [Google Scholar]
- 19.The American Association for Public Opinion Research Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. AAPOR, 2011. Available from http://www.aapor.org/AM/Template.cfm?Section=Standard_Definitions2&Template=/CM/ContentDisplay.cfm&ContentID=3156 [Google Scholar]
- 20.Koch-Weser S, Bradshaw YS, Gualtieri L, et al. The Internet as a health information source: findings from the 2007 Health Information National Trends Survey and implications for health communication. J Health Commun 2010;15(Suppl 3):279–93 [DOI] [PubMed] [Google Scholar]
- 21.Neuendorf KA. The Content Analysis Guidebook. Thousand Oaks, CA: Sage, 2002 [Google Scholar]
- 22.Simon SR, Evans JS, Benjamin A, et al. Patients' attitudes toward electronic health information exchange: qualitative study. J Med Internet Res 2009;11:e30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Flocke SA, Stange KC. Direct observation and patient recall of health behavior advice. Prev Med 2004;38:343–9 [DOI] [PubMed] [Google Scholar]
- 24.Ley P. Memory for medical information. Br J Soc Clin Psychol 1979;18:245–55 [DOI] [PubMed] [Google Scholar]
- 25.Fiore M, Jaen CR, Baker TB, et al. Clinical Practice Guideline: Treating Tobacco Use and Dependence: 2008 Update. Rockville, MD: Department of Health and Human Services, 2008 [Google Scholar]
- 26.Shaw BA, Krause N. Exploring race variations in aging and personal control. J Gerontol B Psychol Sci Soc Sci 2001;56:S119–24 [DOI] [PubMed] [Google Scholar]