Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Nov 17.
Published in final edited form as: Health Promot Pract. 2016 Jul 9;18(1):140–149. doi: 10.1177/1524839916643703

Measuring Health Literacy Levels of a Patient Portal Using the CDC’s Clear Communication Index

Jordan M Alpert 1, Linda Desens 2, Alex H Krist 1, Rebecca A Aycock 1, Gary L Kreps 2
PMCID: PMC5114169  NIHMSID: NIHMS789949  PMID: 27188894

Abstract

Once promised to revolutionize health care, patient portals have yet to fully achieve their potential of improving communication between patients and clinicians. In fact, their use can be detrimental to many consumers due to their limited literacy and numeracy skills. This study demonstrates how applying the Centers for Disease Control and Prevention’s Clear Communication Index to a patient portal can be used to identify opportunities for better patient communication and engagement. The Clear Communication Index contains 20 scored items grounded in communication science to enhance patients’ understanding of health information. The Index was applied to one portal used by over 80,000 patients in 12 primary care practices: MyPreventiveCare. This portal was selected because of its ability to personalize preventive and chronic care information by internally using content featuring health literacy principles and linking patients’ externally to trusted materials. Thirty-seven frequently visited portal pages (17 internal and 20 external) were evaluated based on the Index’s four main variables. The overall score for the portal was 72%, which falls below the 90% threshold to be considered clear communication. Internal content scored higher than external (75% vs. 69%). Specific changes to improve the score include simpler language, more specific examples, and clearer numerical explanations.

Keywords: health communication, patient portal, health literacy, eHealth

INTRODUCTION

In spite of new technological developments, there is significant progress to be made in making digital communication a solution that contributes to improved health outcomes rather than a problem that complicates the health delivery process (Kreps, 2014). Patient portals, that is, secure online websites that give patients 24-hour access to personal health information (HealthIT. gov, 2014), are a strategy to improve quality of care. Benefits of portals include the ability for clinicians to document important aspects of the examination and make lab results immediately available to be reviewed privately by patients. Generally, portals are helpful, but their impact has not been as profound as anticipated (Clamp & Keen, 2007). Properly using electronic health tools requires that patients possess skills such as computer literacy, health literacy, and the ability to perform numerical tasks (Norman & Skinner, 2006b). As people increasingly gain access to health information through the Internet or patient portals, the complexity of this information is often such that it is inaccessible to those with inadequate literacy levels (Bodie & Dutta, 2008).

This study applied the Centers for Disease Control and Prevention’s (CDC) Clear Communication Index, a research-based tool that assesses communication materials, to a portal designed to help patients better understand their prevention and chronic care needs, called MyPreventiveCare (Krist et al., 2014). This innovative patient portal was developed to extend care beyond the clinical encounter by explaining technical information in understandable language and to help patients take action, especially for preventive screenings (Krist & Woolf, 2011). The portal is currently used by more than 80,000 patients in 12 primary care practices, and as it continues to demonstrate positive patient outcomes, it will be extended to an additional 300 practices in 15 states through a series of research grants.

Using the Clear Communication Index, the objective of this study was to measure the content of the test portal and its referenced websites’ use of clear communication. Findings will help determine best practices for creating content suitable for patients of all health literacy levels, inform modifications to improve the Index, and build on previous literature regarding the importance of using plain language in digital health environments.

LITERATURE REVIEW

Health Literacy

Communication can be thought of as “the central social process in the provision of health care delivery and the promotion of public health” (Kreps, Bonaguro, & Query, 2003, p. 12). Breakdowns in communication include difficulty understanding general explanations or instructions provided by a physician, which is experienced by one third of the American population (Polack, Richmond, & McCroskey, 2008). The issue, otherwise known as health literacy, is defined as “the degree to which an individual has the capacity to obtain, communicate, process, and understand health information and services in order to make appropriate health decisions” (Patient Protection and Affordable Care Act, 2010, p. 501).

Frequent use of complicated medical jargon makes it problematic for patients with low health literacy to comprehend physician instructions (Castro, Wilson, Wang, & Schillinger, 2007) and leads to patient confusion and disengagement (Kreps & Neuhauser, 2013). The inability to comprehend clinician instructions or interpret data is a significant issue because only 12% of adults are considered to have proficient health literacy (Kutner, Greenburg, Jin, & Paulsen, 2006). Low health literacy is not only associated with limited health vocabulary but also affects individuals’ understanding of health concepts and treatment options (Davis, Williams, Marin, Parker, & Glass, 2002). Furthermore, low health literacy is associated with increased hospitalizations, greater use of emergency care, lower receipt of screening and vaccination, and poorer overall health and higher mortality (Berkman, Sheridan, Donahue, Halpern, & Crotty, 2011).

eHealth and Health Literacy

A method for improving health literacy is eHealth, “the use of emerging information and communication technology, especially the Internet, to improve or enable health and health care” (Eng, 2001, p. 1). One particular eHealth platform is patient portals, which are an evolution of electronic health records that fulfill patient desires for health information, while still under the clinicians’ control (Simborg, Detmer, & Berner, 2013). Portals provide a continuous connection between patient and physician, which can expedite the communication process when problems arise (Tang, Ash, Bates, Overhage, & Sands, 2006). However, portals also require that users are able to understand complex information, have access to computers, and can perform the necessary functionalities to acquire and manipulate data (Sharit et al., 2014). Those issues relate to eHealth literacy, which is the ability to find, understand and apply knowledge from digital sources to address a health problem (Norman & Skinner, 2006b). As a result, patients with low health literacy are less likely to view laboratory results, send e-mails to providers, and make medical appointments using a portal (Sarkar et al., 2011).

Health literacy may be an obstacle to patient portals fully maximizing their potential (Monkman & Kushniruk, 2013). Similar to online health information seeking, in which individuals with low health literacy often have self-efficacy barriers (Shieh, Mays, McDaniel, & Yu, 2009), portals face several impediments. For example, one study that analyzed a particular portal, HealthSpace, found that it failed to empower patients, provide higher quality data, or improve health literacy due to poor ease of use and difficulty accessing data (Greenhalgh, Hinder, Stramer, Bratan, & Russell, 2010). Since the portal did not align with patients’ attitudes and informational needs, adoption was unlikely (Greenhalgh et al., 2010). Other studies have found that a major barrier for patients using portals was understanding and interpreting lab results (Cimino, Patel, & Kushniruk, 2002) and medical reports (Segall et al., 2011). However, the very same portal used in the current study helped improve rates of colorectal, breast, and cervical cancer screenings (Krist et al., 2012). Moreover, nearly double the amount of portal users were up-to-date on all preventive services compared with nonusers (Krist et al., 2012). Improving patient outcomes occurred by incorporating the portal’s design with patient-centered principles, such as consideration of the patient’s desire for information and shared decision making (Stewart, 2001). This was achieved through the use of lay language, tailored recommendations, and educational resources.

Plain Language

Another approach to addressing health literacy issues is the use of plain language, or “communications that engage and are accessible to the intended audience” (Stableford & Mettger, 2007, p. 75). For text-based information, plain language includes the use of evidence-based standards in writing and design to create user-friendly content (Stableford & Mettger, 2007). Plain language has been advocated in health (Stableford & Mettger, 2007); however, adoption has been slow since the health care industry is considered among the least effective communicators due to not using accessible and clear language (Watson Wyatt Worldwide, 2006). Incorporating plain language is extremely important because one out of five Americans reads at the fifth grade level or below, yet most health care materials are written at the 10th grade level (Point-of-Care-Partners, 2012). Implementing plain language consists of (1) organizing information so that the most important points are listed first, (2) breaking complex information into small chunks that can be easily understood, and (3) avoiding technical terms (Point-of-Care-Partners, 2012).

CDC’s Clear Communication Index

To assist in the development of clear communication that utilizes plain language, the CDC designed a tool that provides a set of evidence-based criteria to assess public communication for diverse audiences (Baur & Prue, 2014). The Index complies with the Plain Writing Act of 2010, requiring that governmental communication is clear to the public (Plain Writing Act, 2010), and supports the efforts of the National Action Plan to Improve Health Literacy (U.S. Department of Health and Human Services, 2010), which aims to improve health, longevity, and quality of life by delivering health services in ways that are easy to understand. Thus far there is limited application of the Index, but pilot studies have demonstrated its ability to enhance public health information (Atkins, 2015; Prue, 2015), and when materials were revised based on the Index, they were more easily understood (Baur & Prue, 2014). The Index is constructed of four main categories used for assessment: Core, Behavioral Recommendations, Numbers, and Risk (Baur & Prue, 2014).

Core (Part A): Questions are related to the materials’ use of visual cues, and gauges whether the main message can be immediately recognized. Language is also assessed by checking for familiar words and use of the active voice; the information design section examines whether there are bulleted lists and section headings; the last area, state of the science, asks if opinions of subject matter experts are included.

Behavioral Recommendations (Part B): This category assesses whether the material explains the importance of recommendations to the audience with specific directions. For example, step-by-step guidelines like “Look for cereal with 100% daily value of folic acid” or detailed instructions about when and how often to perform a specific behavior would be considered acceptable.

Numbers (Part C): The objective is to determine if numbers are presented in a manner that the primary audience would understand, free of mathematical calculations.

Risk (Part D): Establishes whether the nature of the risk is explained and if both benefits and risks of the behavior are addressed.

The test portal used in this study has previously demonstrated effectiveness in improving preventive care screenings, but using the CDC’s Clear Communication Index will gauge the clarity of its content. Since the portal uses information from widely used and respected noncommercial federal and organizational health entities, measuring the content it provides will test the Index as a viable tool for assessing the clarity of health information.

Research Questions

Research Question 1: Does the portal’s internal content meet the acceptable levels of the CDC’s Clear Communication Index?

Research Question 2: Does the portal’s links to external content meet the acceptable levels of the CDC’s Clear Communication Index?

METHOD

Instrumentation

The CDC’s Clear Communication Index consists of 20 questions, separated into four sections (Baur & Prue, 2014). Each question receives a score of 1 or 0, depending on if the criteria was or was not present. The highest score that can be received is 20/20. Total scores are then converted to a scale of 100. A score of 90% or higher is considered excellent. A score of 89% or below indicates that the material may need to be revised to make it clearer for the priority audience.

Analytical Process

The patient portal selected provides personalized prevention and educational materials, as well as tailored content from HealthFinder.org, a credible online health information resource guided by expert advisors. It also links patients, based on their profile, to a range of existing educational material from noncommercial federal (e.g., CDC, National Cancer Institute) and organizational entities (e.g., American Diabetes Association, Mayo Clinic). Many patient portals similarly use and link patients to existing resources, making MyPreventiveCare a good test case for this study.

A total of 37 internal and external portal web pages were selected from the portal’s three main sections (Preventive Care You Need Now, Your Labs, Other Preventive Care) based on patient analytics of the most frequently clicked pages from previous months. Content included a wide variety of medical topics, from the flu shot to cancer screenings. Each web page was evaluated using the CDC’s Clear Communication Index; 17 pages were internal content, written by providers and administrators of the portal, and 20 pages were recommended links to external content posted by providers to websites such as Cancer.gov, Heart.org, and CDC.gov. First, two coauthors discussed the way in which they perceived content as it related to the scoring index. For instance, the “Exercise” page on the portal was jointly reviewed and aspects such as use of the active voice were discussed, to ensure content was evaluated equally. Next, a third of the links were evaluated by both coders separately to establish intercoder reliability (Wimmer & Dominick, 2013). The coauthors then discussed the differences in scores and determined why variations were occurring. If there was more than a 2-point difference in any of the four variables, the entire score sheet was independently recoded by both researchers. Intercoder reliability was tested using ReCAl2 (Freelon, 2010) set for two coders. Krippendorff’s alpha for Variable 1 (Core) was .716, Variable 2 (Behavioral Recommendations) was .837, Variable 3 (Numbers) .926, and Variable 4 (Risk) was .807. The Krippendorff’s alpha for total scores was .873. Typically, a reliability coefficient of .70 or above is acceptable for establishing intercoder reliability (Keyton, 2006). The remaining links were split between the coauthors and individually scored.

RESULTS

The average score of all 37 internal and external web pages evaluated was 72%; internal portal pages scored 75% and external pages scored 69%. A full summary containing scores for each page can be found in Table 1.

Table 1.

clear communication Scores for Internal and External content

Internal Link Score, % External Link Score, %
Exercise 94 FDA Nutrition Label 95
Get a Pneumonia Shot 93 NIH Diabetes 94
Colon Cancer Testing 84 beTobaccoFree.hhs.gov 88
Quit Smoking 84 CDC Tetanus 88
Get a Tetanus Shot 82 Mayo Clinic Weight Loss 86
Take Aspirin 80 National Library Blood Pressure 85
Bone Density 80 AHA Medications 84
You Have High Blood Pressure 79 Weight Control Info Network 82
Cervical Cancer 75 Mayo Clinic Blood Pressure 75
Get Tested for Diabetes 74 CDC Vaccines 75
Watch Your Weight 74 CDC Diabetes 73
Weight (other preventive care) 74 Smokefree.gov 71
Mammogram 71 Mayo Clinic Aspirin 69
You Have Diabetes 68 CDC About Tetanus 65
Your Last Blood Pressure Was Too High 63 NIH Calculator 60
Diet 62 CDC About Pneumonia Disease 53
PSA Test 50 Smokefree.gov Medications 50
CDC Pneumonia Vaccine—Who Needs It? 40
U.S. Preventive Services Taskforce 35
CDC Pneumonia Vaccines Can Help 19
Open in a new tab

NOTE: FDA = Food and Drug Administration; NIH = National Institutes of Health; CDC = Centers for Disease Control and Prevention; AHA = American Heart Association; PSA = prostate-specific antigen.

Research Question 1: Does the Portal’s Internal Content Meet the Acceptable Levels of the CDC’s Clear Communication Index?

Two pages scored above 90%; the “Exercise” page (94%) and “Get a Pneumonia Shot” page (93%; Figure 1). The “Exercise” page lost only 1 point for not including the opinions of subject matter experts. It excelled in having the most important information at the top of the page, like examples of activities (gardening, jogging) and how long an individual should exercise. The page also had a very clear, universal message stating the importance of exercise. In addition, benefits were discussed, such as “Physical activity increases your chances of living longer and keeps you healthier.” An image of a man playing basketball reinforced the overall message. Similarly, the “Get a Pneumonia Shot” page lost only 1 point for not including authoritative sources or the opinions of subject matter experts. Like the “Exercise” page, information found on the “Get a Pneumonia Shot” page presented recommendations in plain language and used bulleted lists so as to not overwhelm the patient.

FIGURE 1.

FIGURE 1

Open in a new tab

MyPreventivecare “Get a Pneumonia Shot” Sample Page.

The other 15 portal pages scored under 90%, although only 4 pages scored below 70%. Overall, the Numbers category (Part C) was rated the lowest, receiving an average score of 1 out of 3 because numbers were not thoroughly explained. For instance, on the “Blood Pressure” page, an explanation of the measure 120/80 was not provided.

Core (Part A) scored favorably, averaging 9 out of 11 points, mostly due to the portal’s simple design layout. The portal frequently employed the use of bulleted lists instead of using large chunks of text. In addition, easily understandable language was used and almost every page had a friendly graphic associated with the content. However, this section consistently lost a point by not stating the source of the content provided or what was still unknown about a particular condition or recommendation.

Behavioral Recommendations (Part B) averaged 2 out of 3 points but unsuccessfully explained recommended behaviors. For example, the “Diabetes” page offered recommendations like “Eat healthy,” but that general statement can be interpreted differently according to different patients. In another instance, the “Blood Pressure” page recommended that patients limit salt and alcohol intake. This recommendation is not specific and leaves it to the patient to determine how much salt and alcohol is too much. Furthermore, on the “PSA Test” page, the recommendations state that the patient may not need further testing but the rationale is not explained.

The last category, Risk (Part D), emphasized the benefits of treatments, but potential drawbacks were mostly lacking. The closest the portal came to addressing risks was on the “Pneumonia Shot” page, where side effects were provided in the following statement: “The pneumonia shot is very safe although some people may have redness or pain where the shot is given.” More detailed information could be provided, given the public’s suspicion of vaccinations. Similarly, very little information about patient concerns and risks was provided on the “Mammogram” page. A summary of each category’s score can be found in Table 2.

Table 2.

Internal category Scores

Category Average Score
Part A (Core): call to action, visual cues, main statement, language 9/11
Part B (Behavioral Recommendations): importance explained, specific direction 2/3
Part C (Numbers): understandable and free of calculations 1/3
Part D (Risk): explanations, benefits and risks presented 2/3
Open in a new tab

Research Question 2: Does the Portal’s Links to External Content Meet the Acceptable Levels of the CDC’s Clear Communication Index?

The most popular noncommercial federal link within the portal was to the CDC, followed by the National Institutes of Health, National Cancer Institute, and smokefree.gov. The most frequently linked organizational website was the American Heart Association, followed by the U.S. Preventive Services Taskforce and Mayo Clinic. A summary of all external links can be seen in Table 3.

Table 3.

External Link Frequencies

Noncommercial Federal Link Frequency Organization Link Frequency
Centers for Disease Control and Prevention 18 American Heart Association 4
National Institutes of Health 11 U.S. Preventive Services Taskforce 4
National Cancer Institute 9 Mayo Clinic 3
Smokefree.gov 8 Familydoctor.org 1
beTobaccoFree.gov 2 American Diabetes Association 1
Medline Plus 2 Shape Up 1
Healthfinder 1 Health Crossroads 1
Nutrition.gov 1 American Urologic Association 1
Choosemyplate 1 American Cancer Society 1
SuperTracker (U.S. Department of Agriculture) 1
Food and Drug Administration 1
Open in a new tab

Out of the 20 external links analyzed, only two web pages received scores of 90% or greater: Food and Drug Administration’s “Nutrition Label” page and National Institutes of Health’s “Diabetes” page. All seven of the CDC’s pages analyzed were below 90%. The lowest scoring page, which received 19%, was the CDC’s “Pneumonia Vaccines” page (Figure 2).

FIGURE 2.

FIGURE 2

Open in a new tab

cdc’s “Pneumonia Vaccines” Page.

The Food and Drug Administration Nutrition Label page featured several images, including analysis of each section of the nutritional label with simple explanations. Also, numerical calculations about benefits and risks of calories, fat, and nutrients were all provided in plain language. Another top performer, beTobaccoFree, had a very clear primary message (e.g., [smoking] can harm nearly every organ of the body) and used the active voice (e.g., “Every cigarette you smoke damages your breathing and scars your lungs”).

Breaking down external links by category, Core (Part A) received an average of 7 out of 11 points because web pages rarely contained imagery. For the most part, paragraphs of text appeared that were not bulleted or separated with headings. In addition, a major flaw of commercial websites was requiring the user to click for more information instead of having the main message appear at once. Behavioral Recommendations (Part B) and Numbers (Part C) scored equally to the portal. Most external links contained general recommendations, and numbers were often not explained. Risk (Part D) rarely contained balanced information and instead focused solely on either benefits or risks. A summary of each category’s score can be found in Table 4.

Table 4.

External category Scores

Category Average Score
Part A (Core): call to action, visual cues, main statement, language 7/11
Part B (Behavioral Recommendations): importance explained, specific direction 2/3
Part C (Numbers): understandable and free of calculations 1/3
Part D (Risk): explanations, benefits and risks presented 1/3
Open in a new tab

Discussion

As patient portal utilization continues to increase by patients and clinicians, it is essential that they present reliable and comprehensive health information, in a clear and straightforward manner. Portals allow patients to view and interpret medical data in their homes, away from providers (Ralston, Revere, Robins, & Goldberg, 2004), increasing the importance of information conveyed at appropriate health literacy levels. A similar awareness is needed for health information on the Internet, resulting in modifications that make content more understandable.

Findings from this study revealed that the CDC’s Clear Communication Index may be a useful tool in identifying areas of unclear communication in a patient portal and referenced health websites. The Index revealed that communicating Numbers (Part C) and Risk (Part D) was most troublesome. Previous studies have indicated that even readers with strong literacy skills can have difficulty using quantitative information (Rothman et al., 2006). Therefore, results should be used to evaluate and revise existing health communication materials and develop new resources. However, formative research and retesting of materials should still be conducted to ensure that the right messaging and communication channels are applied (Baur & Prue, 2014). In addition to using the Index, content evaluation through focus groups, interviews, and surveys should be considered as well as the use of other literacy tools, like eHEALS (Norman & Skinner, 2006a) and Rapid Estimate of Adult Literacy in Medicine (Davis et al., 1991).

The Index was originally developed to address content for large priority populations. In contrast, patient portals are focused on one-to-one patient–clinician communication and follow algorithms to allow for personalization of materials. Considerations such as diverse cultural beliefs, health literacy, language barriers, and computer literacy are not addressed by the Index but need to be considered when developing content for a patient portal given the broad priority populations. Additionally, disparities in patient portal use due to factors such as educational attainment and race/ethnicity are still significant, increasing the importance of evaluation and participatory design (Kreps & Neuhauser, 2010).

One of the challenges of measuring a patient portal with a tool like the Index is that portals often link to external websites, as evaluated in this study. This is particularly important because most adults in the United States (72%) use the Internet for health information (Fox & Duggan, 2013), but little is known about the quality and readability of the health content found on commercial, nonprofit, and governmental websites. Of the few studies that have assessed health content, it was found that among 125 sampled websites, 92% of government websites, 92% of commercial websites, and 84% of nonprofit websites (.org) required reading comprehension capabilities higher than the eighth grade and nearly 30% required a college education to comprehend health-related information (Becker, 2004). Similarly, websites containing cancer information were considered “difficult” and had readability scores higher than grade 12 (Friedman, Hoffman-Goetz, & Arocha, 2006). Findings from the current study demonstrate that freely available health information disseminated by well-known and respected organizations should be further scrutinized for health literacy considerations.

Implications

To support eHealth literacy and tailoring eHealth applications to appropriate health literacy levels, reliable, valid, and convenient instruments are needed (Miller & West, 2009). The CDC’s Clear Communication Index has the potential to improve health communication materials, but greater focus needs to be placed on the manner in which materials are presented by clinicians and content creators. Patient portals allow the opportunity for innovative methods of information transfer, such as the use of graphics and video and audio clips to enhance the understandability and impact of health messages (Kreps & Neuhauser, 2010). For instance, when medical reading materials used instructional graphics, comprehension improved among poor readers (Davis et al., 2002). New technologies, like patient portals, should be evaluated using the CDC’s Clear Communication Index and take the lead in prioritizing health literacy to ensure that health information is presented clearly.

Conclusion

The CDC’s Clear Communication Index is sufficient for developing and revising health information found on portals and websites to ensure that simplified language and limited text is used to make interpretation clearer, contributing to improved health literacy. With portals becoming more widely implemented by medical practices, it is necessary that they are designed to function as valuable health communication tools appropriate for all patients.

Limitations

A limitation to this study was the lack of comparisons across other patient portals. MyPreventiveCare may have distinct advantages with regard to health literacy compared to other platforms. Contrasting the differences between systems may have distinguished the various ways that systems are used depending on the context. In addition, scoring was guided by criteria outlined by the CDC’s Clear Communication Index. Several categories were left to the interpretation of the researchers since the Index did not explicitly state the nuances found when reviewing the website.

Direction for Future Studies

Future research should follow up the current study to determine differences in scores between the original portal and the revised portal. Additionally, further research should analyze perceptions of the content developed for patient portals based on the Index and whether it has the ability to enhance patient–clinician communication and positively affect health behaviors.

Acknowledgments

This research was supported by grants from the National Cancer Institute (R01CA166375-01A1) and the National Center for Advancing Translational Sciences (UL1TR000058).

References

  1. Atkins B. A smooth takeoff: Results of a pilot project to introduce the Clear Communication Index to Federal public health staff. Paper presented at the 2015 National Conference on Health Communication, Marketing, and Media; Atlanta, GA. 2015. Aug, [Google Scholar]
  2. Baur C, Prue C. The CDC Clear Communication Index is a new evidence-based tool to prepare and review health information. Health Promotion Practice. 2014;15:629–637. doi: 10.1177/1524839914538969. [DOI] [PubMed] [Google Scholar]
  3. Becker SA. A study of web usability for older adults seeking online health resources. ACM Transactions on Computer-Human Interaction. 2004;11:387–406. [Google Scholar]
  4. Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: An updated systematic review. Annals of Internal Medicine. 2011;155:97–107. doi: 10.7326/0003-4819-155-2-201107190-00005. [DOI] [PubMed] [Google Scholar]
  5. Bodie GD, Dutta MJ. Understanding health literacy for strategic health marketing: eHealth literacy, health disparities, and the digital divide. Health Marketing Quarterly. 2008;25:175–203. doi: 10.1080/07359680802126301. [DOI] [PubMed] [Google Scholar]
  6. Castro CM, Wilson C, Wang F, Schillinger D. Babel babble: Physicians’ use of unclarified medical jargon with patients. American Journal of Health Behavior. 2007;31(Suppl 1):S85–S95. doi: 10.5555/ajhb.2007.31.supp.S85. [DOI] [PubMed] [Google Scholar]
  7. Cimino JJ, Patel VL, Kushniruk AW. The patient clinical information system (PatCIS): Technical solutions for and experience with giving patients access to their electronic medical records. International Journal of Medical Informatics. 2002;68:113–127. doi: 10.1016/s1386-5056(02)00070-9. [DOI] [PubMed] [Google Scholar]
  8. Clamp S, Keen J. Electronic health records: Is the evidence base any use? Informatics for Health and Social Care. 2007;32:5–10. doi: 10.1080/14639230601097903. [DOI] [PubMed] [Google Scholar]
  9. Davis TC, Crouch MA, Long S, Jackson HR, Bates P, George RB, Bairnsfather LE. Rapid estimate of literacy levels of adult primary care patients. Family Medicine. 1991;23:433–435. [PubMed] [Google Scholar]
  10. Davis TC, Williams MV, Marin E, Parker RM, Glass J. Health literacy and cancer communication. CA: A Cancer Journal for Clinicians. 2002;52:134–149. doi: 10.3322/canjclin.52.3.134. [DOI] [PubMed] [Google Scholar]
  11. Eng TR. The eHealth landscape: A terrain map of emerging information and communication technologies in health and health care. Princeton, NJ: Robert Wood Johnson Foundation; 2001. [Google Scholar]
  12. Fox S, Duggan M. Health online 2013. 2013 Retrieved from http://bibliobase.sermais.pt:8008/biblionet/upload/pdf5/003820.pdf.
  13. Freelon DG. ReCal: Intercoder reliability calculation as a web service. International Journal of Internet Science. 2010;5:20–33. [Google Scholar]
  14. Friedman DB, Hoffman-Goetz L, Arocha JF. Health literacy and the World Wide Web: Comparing the readability of leading incident cancers on the Internet. Informatics for Health & Social Care. 2006;31:67–87. doi: 10.1080/14639230600628427. [DOI] [PubMed] [Google Scholar]
  15. Greenhalgh T, Hinder S, Stramer K, Bratan T, Russell J. Adoption, non-adoption, and abandonment of a personal electronic health record: Case study of HealthSpace. British Medical Journal. 2010;2010:341. doi: 10.1136/bmj.c5814. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. HealthIT.gov. What is a patient portal? 2014 Retrieved from https://www.healthit.gov/providers-professionals/faqs/what-patient-portal.
  17. Keyton J. Communication research: Asking questions, finding answers. New York, NY: McGraw-Hill; 2006. [Google Scholar]
  18. Kreps GL. Achieving the promise of digital health information systems. Journal of Public Health Research. 2014;3:471. doi: 10.4081/jphr.2014.471. doi:10.4081%2Fjphr.2014.471. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kreps GL, Bonaguro EW, Query JL., Jr The history and development of the field of health communication. Russian Journal of Communication. 2003;10:12–20. [Google Scholar]
  20. Kreps GL, Neuhauser L. New directions in eHealth communication: Opportunities and challenges. Patient Education and Counseling. 2010;78:329–336. doi: 10.1016/j.pec.2010.01.013. [DOI] [PubMed] [Google Scholar]
  21. Kreps GL, Neuhauser L. Artificial intelligence and immediacy: Designing health communication to personally engage consumers and providers. Patient Education and Counseling. 2013;92:205–210. doi: 10.1016/j.pec.2013.04.014. [DOI] [PubMed] [Google Scholar]
  22. Krist AH, Aycock RA, Etz RS, Devoe JE, Sabo RT, Williams R, … Deshazo J. MyPreventiveCare: Implementation and dissemination of an interactive preventive health record in three practice-based research networks serving disadvantaged patients—a randomized clustertrial. Implementation Science. 2014;9:181. doi: 10.1186/s13012-014-0181-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Krist AH, Woolf SH. A vision for patient-centered health information systems. Journal of the American Medical Association. 2011;305:300–301. doi: 10.1001/jama.2010.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Krist AH, Woolf SH, Rothemich SF, Johnson RE, Peele JE, Cunningham TD, … Matzke GR. Interactive preventive health record to enhance delivery of recommended care: A randomized trial. Annals of Family Medicine. 2012;10:312–319. doi: 10.1370/afm.1383. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kutner M, Greenburg E, Jin Y, Paulsen C. The health literacy of America’s adults: Results from the 2003 National Assessment of Adult Literacy. Washington, DC: National Center for Education Statistics; 2006. NCES 2006–483. [Google Scholar]
  26. Miller EA, West DM. Where’s the revolution? Digital technology and health care in the internet age. Journal of Health Politics, Policy and Law. 2009;34:261–284. doi: 10.1215/03616878-2008-046. [DOI] [PubMed] [Google Scholar]
  27. Monkman H, Kushniruk A. Applying usability methods to identify health literacy issues: An example using a personal health record. In: Courtney KL, Shabestari O, editors. Enabling health and healthcare through ICT. Vol. 15. Amsterdam, Netherlands: IOS Press BV; 2013. pp. 179–185. [PubMed] [Google Scholar]
  28. Norman CD, Skinner HA. eHEALS: The eHealth Literacy Scale. Journal of Medical Internet Research. 2006a;8(4):e27. doi: 10.2196/jmir.8.4.e27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Norman CD, Skinner HA. eHealth literacy: Essential skills for consumer health in a networked world. Journal of Medical Internet Research. 2006b;8(2):e9. doi: 10.2196/jmir.8.2.e9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Patient Protection and Affordable Care Act, 111th Cong. (2010).
  31. Plain Writing Act of 2010, Pub. L. No. 111–274 (2010). Retrieved from http://www.gpo.gov/fdsys/pkg/PLAW-111publ274/pdf
  32. Point-of-Care-Partners. Six areas of opportunity under the Affordable Care Act. 2012 Retrieved from http://www.pocp.com/PDF/health-plans-hit-opportunities.pdf.
  33. Polack EP, Richmond VP, McCroskey JC. Applied communication for health professionals. Kendall/Hunt; 2008. [Google Scholar]
  34. Prue C. Using the Clear Communication Index to review a web-based toolkit. Paper presented at the 2015 National Conference on Health Communication, Marketing, and Media; Atlanta, GA. 2015. Aug, [Google Scholar]
  35. Ralston JD, Revere D, Robins LS, Goldberg HI. Patients’ experience with a diabetes support programme based on an interactive electronic medical record: qualitative study. British Medical Journal. 2004;328(7449):1159. doi: 10.1136/bmj.328.7449.1159. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Rothman RL, Housam R, Weiss H, Davis D, Gregory R, Gebretsadik T, … Elasy TA. Patient understanding of food labels: the role of literacy and numeracy. American Journal of Preventive Medicine. 2006;31:391–398. doi: 10.1016/j.amepre.2006.07.025. [DOI] [PubMed] [Google Scholar]
  37. Sarkar U, Karter AJ, Liu JY, Adler NE, Nguyen R, López A, Schillinger D. Social disparities in internet patient portal use in diabetes: evidence that the digital divide extends beyond access. Journal of the American Medical Informatics Association. 2011;18:318–321. doi: 10.1136/jamia.2010.006015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Segall N, Saville JG, L’Engle P, Carlson B, Wright MC, Schulman K, Tcheng JE. Usability evaluation of a personal health record. Paper presented at the AMIA annual symposium; Washington, DC. 2011. Oct, [PMC free article] [PubMed] [Google Scholar]
  39. Sharit J, Lisigurski M, Andrade AD, Karanam C, Nazi KM, Lewis JR, Ruiz JG. The roles of health literacy, numeracy, and graph literacy on the usability of the VA’s personal health record by veterans. Journal of Usability Studies. 2014;9:173–193. [Google Scholar]
  40. Shieh C, Mays R, McDaniel A, Yu J. Health literacy and its association with the use of information sources and with barriers to information seeking in clinic-based pregnant women. Health Care for Women International. 2009;30:971–988. doi: 10.1080/07399330903052152. [DOI] [PubMed] [Google Scholar]
  41. Simborg DW, Detmer DE, Berner ES. The wave has finally broken: Now what? Journal of the American Medical Informatics Association. 2013;20(e1):e21–e25. doi: 10.1136/amiajnl-2012-001508. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Stableford S, Mettger W. Plain language: A strategic response to the health literacy challenge. Journal of Public Health Policy. 2007;28:71–93. doi: 10.1057/palgrave.jphp.3200102. [DOI] [PubMed] [Google Scholar]
  43. Stewart M. Towards a global definition of patient centred care: The patient should be the judge of patient centred care. British Medical Journal. 2001;322(7284):444. doi: 10.1136/bmj.322.7284.444. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Tang PC, Ash JS, Bates DW, Overhage JM, Sands DZ. Personal health records: Definitions, benefits, and strategies for overcoming barriers to adoption. Journal of the American Medical Informatics Association. 2006;13:121–126. doi: 10.1197/jamia.M2025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. U.S. Department of Health and Human Services. National Action Plan to Improve Health Literacy. Washington, DC: Author; 2010. [Google Scholar]
  46. Watson Wyatt Worldwide. Effective communication: A leading indicator of financial performance, 2005/2006 communication ROI study. Arlington, VA: Author; 2006. [Google Scholar]
  47. Wimmer R, Dominick J. Mass media research. Boston, MA: Cengage Learning; 2013. [Google Scholar]

RESOURCES