Assessing Readability of Patient Education Materials: Current Role in Orthopaedics

Abstract

Background

Health literacy is the single best predictor of an individual’s health status. It is important to customize health-related education material to the individual patient’s level of reading skills. Readability of a given text is the objective measurement of the reading skills one should possess to understand the written material.

Questions/purposes

In this article, some of the commonly used readability assessment tools are discussed and guidelines to improve the comprehension of patient education handouts are provided.

Where are we now? Several healthcare organizations have recommended the readability of patient education materials be no higher than sixth- to eighth-grade level. However, most of the patient education materials currently available on major orthopaedic Web sites are written at a reading level that may be too advanced for comprehension by a substantial proportion of the population.

Where do we need to go?

There are several readily available and validated tools for assessing the readability of written materials. While use of audiovisual aids such as video clips, line drawings, models, and charts can enhance the comprehension of a health-related topic, standard readability tools cannot construe such enhancements.

How do we get there?

Given the variability in the capacity to comprehend health-related materials among individuals seeking orthopaedic care, stratifying the contents of patient education materials at different levels of complexity will likely improve health literacy and enhance patient-centered communication.

Introduction

An individual’s health literacy is considered the single best predictor of their health status [5, 27, 51]. Health literacy is defined as the “capacity to obtain, interpret, and understand basic health information and services and the competence to use such information and services to enhance health” [22, 44]. Poor health literacy is estimated to contribute more than 73 billion dollars of additional burden to the US healthcare system [19, 51]. The healthcare cost of Medicaid patients with limited literacy is about four times that of those who have adequate health literacy [52]. Poor communication between physician and patients predisposes to medical malpractice cases, while effective communication lowers patient’s anxiety, improves compliance and clinical outcome [51].

Providing health education through written materials is a time-tested method. Readability, an important attribute of written material, can affect the reader’s ability to comprehend. Several healthcare agencies have recommended readability of patient education materials should not be higher than sixth- to eighth-grade level [11, 44, 52]. However, most of the available patient education materials in orthopaedics are written at a reading level that does not meet these standards and may be too complex for comprehension by a substantial proportion of the population [4, 47, 48, 50].

In this article, we explore the current state of health literacy and patient education materials (“Where are we now?”), the goals of patient education with regard to readability (“Where do we need to go?”), and potential solutions to enhance suitability of patient education materials (“How do we get there?”).

Where Are We Now? The Patient and Society

Reading skills or the ability to read and understand written text is an important element of literacy. While reading skills are measured in terms of “grade level” achieved in school, an individual’s highest grade level attained in school is not a reliable indicator of their reading skills or literacy status. The terms “functional illiteracy” and “marginal literacy” can be equated to zero- to fifth-grade reading skills and sixth- to eighth-grade reading skills, respectively [34]. Based on National Adult Literacy Surveys nearly half of the US population is either “functionally illiterate’ or “marginally literate” [41, 42]. About one in five (21%) adults in the US reads at or below the fifth-grade level [31] and 25% of functionally illiterate individuals have graduated from high school [27, 51]. Over 90 million adult Americans are affected by low literacy and have difficulty in comprehending long passages or understanding information from a chart [1]. Nearly 50% of the Hispanic and African American population in the United States is functionally illiterate [41, 42]. Others, including the elderly, unemployed, those from low socioeconomic status and school dropouts are also likely to have limited health literacy [51].

An individual’s health literacy is influenced by their general literacy, familiarity with the healthcare system and how the health information is presented and can predict their health status better than factors such as their age, economic status, educational level, or occupation [39]. Those with poor health literacy are at higher risk for seeking emergency care, have more frequent hospital admissions with longer stays in the hospital [21], are prone to missing medical appointments, have poor compliance with treatment recommendations [1, 5, 18, 39, 44, 51], and greater disease progression [28]. Patients typically have reading skills that are about five grades lower than the highest attained educational grade [26, 31]. Consequently, a major portion of the population may have difficulty comprehending some of the available patient education materials.

Where Are We Now? The Provider and Patient Educational Tools

Even in the era of mass media such as television, radio, and the Internet, written health information serves many important functions. Giving patients and caretakers written information is vital in reinforcing verbal communication [2, 12]. In an attempt to contain healthcare costs, patients and their caregivers are being given more responsibility in their disease management, making the role of patient education including written information even more critical [12].

Readability of a written text is an objective measure of the reading skills an individual must possess to understand that material [2]. Readability is measured in terms of “grade levels.” For instance, a text material with a readability of ninth-grade level can be read and comprehended by someone who has reading skills of ninth grade or higher, but it will be too difficult to read for a person with reading skills of a lower grade. Even though readability of patient education materials has a considerable influence on comprehension by the patient [12, 18], the reading and comprehension skills of patients are often overlooked when developing health education materials [10]. Investigators from various medical fields have consistently demonstrated patient education materials are often written at a level too difficult to read and be comprehended by a substantial portion of the adult US population [3, 8, 10, 15, 17, 18, 31, 35, 36].

Where Do We Need To Go?

The Joint Commission on Accreditation of Healthcare Organization (JCAHO) mandates the availability of easily understandable patient information materials as part of the accreditation process for healthcare facilities. Healthcare organizations such as the National Work Group on Cancer and Health [13], American Medical Association [51], and National Institutes of Health [43] recommend the readability of patient information material should be no higher than sixth-grade level, while the Centers for Disease Control and Prevention (CDC) recommends the readability to be lower than eighth-grade level [11]. Given the variability in the capacity to comprehend health-related materials among individuals seeking orthopaedic care, stratifying the contents of patient education materials at different levels of complexity will be required to improve health literacy and enhance patient-centered communication.

How Do We Get There? Assessment of Readability

There are a variety of methods available to assess the readability of written materials. Ever since the introduction of various readability formulas during the early half of the twentieth century [31], these diagnostic tools have been widely adopted in areas ranging from insurance and legal documents to military manuals. However, there is no consensus as to which readability formula is best suited for assessing patient education materials. Some of the readability assessment tools used in the healthcare setting include the Flesch Reading Ease scale [22], Flesch-Kincaid Grade [22], Fry Readability Graph [11], McLaughlin’s SMOG grading [40], Gunning Fog Index [22], Dale-Chall Readability formula [19], and Suitability Assessment of Materials [18] (Table 1). In general, it is preferable to use more than one readability method to improve the validity of the results. While some readability formulas are validated against various tests of comprehension, the most common being McCall-Crabbs criterion [31], there is no gold standard readability test. Furthermore, readability formulas use different comprehension levels to compute grade level. The Flesch Reading Ease (FRE) score, the earliest of the commonly used tools to assess readability, gives a score on a scale ranging from 0–100, with 0 being unreadable and 100 being most readable [22]. It is based on the average number of syllables per word and the average number of words per sentence. The Flesch-Kincaid grade is the most widely used tool to assess readability [2, 12]. It is a modified version of FRE scale. The Fry Readability Graph is favored by some experts, including the CDC, since it requires only three 100-word samples from different parts of the text instead of the entire document and the average number of syllables is noted. The results are then plotted on a graph, generating the grade level of readability [11]. This readability assessment tool is simple and can be applied manually without the need for computers and software. It is particularly well suited for lengthy patient education materials and can also be applied to Spanish text. Originally developed by McLaughlin in 1969, the SMOG formula is recommended by the National Cancer Institute [40]. This readability assessment tool is based on the number of polysyllabic (containing more than two syllables) words in a sample of 30 consecutive sentences. SMOG scores are one to two grades higher than results attained using some of the other readability formulas since it is based on 100% comprehension ability compared to a lower percentage. For instance, if a particular material has SMOG readability grade of 6, it means it will be comprehensible to all individuals with sixth-grade reading skills [31]. The Gunning Fog index is based on the average words per sentence and the percentage of polysyllable words [22]. The New Dale-Chall Readability formula was developed specifically for evaluating health education materials and has the highest validity when tested for reader comprehension [19, 50]. This formula calculates readability using not only the sentence structure but also the vocabulary noting the percentage of unfamiliar words in a written passage based on 3000 commonly used words typically known by at least 80% of fourth graders [50].

Table 1.

Summary of commonly used readability formulas [22, 31]

Instrument	Variables	Advantages	Disadvantages	Comments
Flesch Reading Ease score	Average number of syllables per word and average number of words per sentence. Flesch Reading Ease score = 206.835 − 0.846 × number of syllables per 100 words − 1.015 × average number of words per sentence	Widely available through Microsoft® Word® Office package	Interpretation is relatively difficult as score ranges from 0–100 and negatively correlates with other readability formulas	As FRE scores increase, the FK Grade level decreases and vice versa
Flesch Kincaid Grade	Same as FRE score. Flesch-Kincaid Grade = 0.39 × average number of words per sentence + 11.8 × number of syllables per word – 15.59	Wide availability, extensively validated, quick and easy to administer, correlates highly with other readability formulas	Solely based on polysyllable words and long sentences, thus may underestimate reading difficulty of medical jargon that may contain short but unfamiliar words	Most commonly used formula; available only for English language
Gunning Fog Index	Average number of words per sentence, percentage of words with more than 2 syllables. Gunning Fog index = 0.4 × (average number of words per sentence + percentage of words with more than two syllables)	Needs less time to administer if done manually compared to FRE and FK Grade formula	Computer software measuring index is not as widely available as FRE and FK Grade	Like FRE, FK grade, SMOG and Fry, Fog Index also cannot measure reading difficulty of text arranged in tables and rows
SMOG Readability formula	Number of words with more than 2 syllables. SMOG score = 3 + square root of polysyllable count (number of words with more than two syllables) in 30 sentences.	Compared to other formulas, SMOG uses a stricter criterion, ie, it aims for 100% comprehension; thus, the grade level readability will be usually about 1–2 grades higher than FK Grade	Not as widely available as FRE and FK Grade	Recommended by National Cancer Institute for cancer pamphlets
Fry Readability Graph	Number of sentences and syllables per 100 words.	Validated in both Spanish and English; correlates highly with other readability formulas	Software tools that can estimate graph not as widely available as FRE and FK Grade	Recommended by some because of the ease of manual administration compared to other readability tools
The New Dale-Chall readability formula	Number of syllables per sentence and percentage of difficult words. raw score = 0.0496 × average words per sentence + 0.1579 × % unfamiliar words + 3.6365	Validated for comprehension; takes into consideration the vocabulary and sentence structure	Not available in commonly available word processing software	Developed primarily for health education materials
Suitability Assessment of Materials	22 different variables	Can be used for both print media and audiovisual tools; only tool that can assess the influence of illustrations on comprehension	Time consuming, steep learning curve; not as extensively validated as other readability tools, subjective; no software tools available and has to be administered manually

FRE = Flesch Reading Ease score; FK Grade = Flesch-Kincaid Grade; SAM = Suitability Assessment of Materials.

Adapted from Friedman DB, Hoffman-Goetz L. A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav. 2006;33:352–373 ©2006 Sage Publications [22] and reprinted by permission of SAGE Publications.

Currently, many software packages are available that can reliably assess the readability level of written text [33]. Some of the these programs include Microsoft^® Word^® Office Package (Microsoft Corp, Redmond, WA), Corel^® WordPerfect^® Office X3 (Corel Corp, Ottawa, ON, Canada), Readability Calculations (Micro Power & Light Co, Dallas, TX), Readability Studio 1.1 (Oleander Solutions, Vandalia, OH), and InText (Social Science Consulting, Rudolstadt, Germany). The latter three packages allow assessment of readability using multiple readability formulas simultaneously. The ubiquitous Microsoft^® Word^® Office package can measure readability using Flesch-Kincaid Grade and Flesch Reading Ease alone (Table 2). Additionally, online word processors such as Google Docs (Google Inc, Mountain View, CA) have built-in support for readability formulas.

Table 2.

Step-by-step approach to assess readability of a given text with Flesch Reading Ease and Flesch-Kincaid Grade using Microsoft^® Word^® Office software [4, 47]

Step	Instructions
1	Make a digital copy of the written material as a word file
2	For articles on a Web page, copy the entire article and paste as “plain text” to avoid HTML tags
3	Delete all information that is irrelevant to patient education context such as date stamps, copyright notice, citations, postal address, etc.
4	Keep only running text and delete all headings and subheadings, tables, and charts
5	Delete colons and semicolons within a sentence; also remove decimal points within numbers
6	Select “Options” under “Tools” menu in Microsoft® Word® Office interface
7	Click on “Spelling and Grammar” tab and select the check-box “Show readability statistics”
8	Then click “OK” to save the changes made
9	Perform a “Spelling and Grammar check” on the document and when it is complete a pop-up screen will appear which gives the FK-grade and FRE score of the document.

None of the formulas mentioned thus far have the ability to assess the effect of audiovisuals in aiding the comprehension of written materials. In addition to illustrations, factors such as font size, use of color, content, and vocabulary used in the text can also influence comprehension. The Suitability Assessment of Materials (SAM) is a tool that can be used to assess the suitability of print materials including assessment of illustrations, video and audio materials [7, 18, 23, 49]. Various factors are evaluated including content, literacy demand, graphics, layout, typography, learning stimulation, motivation and cultural appropriateness. However, SAM is relatively time consuming, inherently subjective and has not been validated as extensively as some of the other readability formulas [18, 49].

How Do We Get There? Assessment of Patient’s Reading Skills and Literacy

To reliably match patient education tools with individuals, it is imperative to objectively measure the reading and literacy skills of the target population. Use of readability formulas alone may not suffice as such methods are only an indirect measurement of literacy skills required to read particular patient education material. Certain behavioral patterns such as incorrectly or incompletely filled intake forms and noncompliance in keeping medical appointments and taking prescribed medications can help identify patients at-risk of poor health literacy [51].

The mismatch between the patient reading skills and the readability of patient education has led to the development of several tools for assessment of health literacy among patients and their caretakers. Rapid Estimate of Adult Literacy in Medicine (REALM) is a reading recognition test of medical terminology [16, 25]. Literacy skill is measured in terms of grade level based on the individual’s ability to read aloud 66 common medical terms presented in the order of increasing number of syllables and difficulty in pronunciation. This tool can be completed in a few minutes and has been validated against other instruments measuring health literacy. However, REALM is not applicable for individuals who lack proficiency in English and does not assess comprehension, but only pronunciation of medical information. Other tools such as Test of Functional Health Literacy in Adults (TOFHLA) [45] and Shortened–Test of Functional Health Literacy in Adults (S-TOFHLA) can be used to measure comprehension [6, 25]. While these instruments are available in both English and Spanish [51], they are more time-consuming [34]. Typically, REALM requires approximately 3 minutes while TOFHLA and S-TOFHLA require 20 minutes and 10 minutes, respectively [27]. Newer tools such as Newest Vital Sign (NVS) are faster, can be completed in under 3 minutes and are available in both Spanish and English [27]. Another means of assessing an individual’s comprehension is the Cloze method, where the individual is given a text passage with a certain number of words deleted (usually every fifth word) and they are asked to select the correct word from a list to replace the deleted words [1]. The Cloze score is the percentage of words correctly identified. The rationale for such an assessment is, as an individual’s reading skills improve, their ability to choose the correct missing words should also advance. For a frustration-free reading, it is recommended the Cloze score of the patient education material should be at least 50% [19]. Although there are some studies where the researchers have used Wide Range Achievement Test (WRAT) [1, 12], experts believe it should not be used as most of the vocabulary used in the test is nonmedical and thus less relevant in a patient education setting [22].

How Do We Get There? Improving Patient Education Materials

It is imperative for healthcare providers to communicate a broad range of health messages to a wide variety of audiences [11]. Several health organizations have made recommendations on various ways to improve the delivery of health-related information to the general population. It is important to substitute complex medical terms with simpler terms that can be found at Web sites such as http://plainlanguage.gov/howto/wordsuggestions/index.cfm [13]. The “Living Word Vocabulary” [14] contains about 43,000 words arranged in various grade levels of complexity and can also be used to substitute difficult words [31]. The National Cancer Institute has promoted a step-by-step guide to improve the comprehension of patient education materials by a wider segment of the population (http://www.cancer.gov/aboutnci/oc/clear-and-simple/page1). The American Medical Association has published a manual for clinicians on health literacy and offers methods to improve readability of patient education materials [51]. In general, long sentences should be avoided and typically contain no more than eight to ten words [26]. Pamphlets and articles lacking illustrations should be supplemented with an adequate number of illustrations, font size should be between 12 to 14 points, and use of all capital letters, italics, nontraditional fonts and sans-serif fonts should be avoided [2, 11]. Other recommendations to improve communication include making patients repeat instructions and “teach-back,” and limiting the amount of new information provided in each office visit [29, 30]. The CDC has published guidelines on ways to improve readability and comprehension of patient education materials [11] (Table 3). Several reports suggest guidelines regarding the appropriate use of diagrams and illustrations in patient education materials [11, 18, 51] (Table 4). We have established recommendations for adoption by the wider orthopaedic community (Table 5).

Table 3.

Guidelines for improving readability and comprehension of written text in patient education materials [11]

Limit the number of messages

Use short sentences

Use conversational style as if you are talking to someone

Limit medical jargon

Avoid ambiguous words

Select familiar words and use them consistently

Use analogies that are familiar and culturally appropriate for the target audience

Instead of real numbers, when conveying statistics use words a like “half,” “one-third”

Avoid symbols and quotation marks

Table 4.

Guidelines for improving visuals and illustrations to enhance comprehension of patient education materials [11, 51]

Limit the number of messages portrayed in a single image to one

Limit details and keep the illustrations relevant

Line drawings are better than real photographs

If real-life photographs are used, care should be taken to make sure the background does not contain any distractions

Images should be culturally appropriate

Images should be place right next to the relevant text

Unless relevant, use back and white drawings as opposed to color drawings

Images of internal body parts should be accompanied by an image that shows where in the body that particular body part is located

Provide appropriate captions and highlight area of interest in the image with the use of circles and arrows

Table 5.

How do we get there?

1. Increase awareness of the issue of readability and how to enhance readability of patient education materials among orthopaedic surgeons

a. Through CME’s, Instructional Course lectures and Workshops

b. Inclusion in residency/medical school curriculum

2. Creation of easily readable patient education materials

a. AAOS and subspecialty organizations should lead the effort through collaboration

b. Patient education materials should clearly identify the readability level at which it is prepared for.

3. Feedback from patients

a. Direct testing of patient education material on patients for comprehension.

b. Online patient education materials should have a brief “post-test” survey to obtain feedback from patients directly related to that material’s ease of readability and comprehension.

4. Accessibility of patient education materials

a. AAOS and subspeciality organizations should further enhance their efforts to make available all patient education material though Internet

b. Avoid fragmentation and make available all patient education material in one location preferably “Your Orthopaedic Connection” website (http://orthoinfo.aaos.org/) instead of different subspecialty websites.

c. Websites should make available “printer-friendly” versions of patient education materials

5. Promote further research into

a. Reading skills of orthopaedic patient population

b. Impact of improving readability of orthopaedic patient education materials.

c. Reading skills of Internet users

d. Creation of better standards and tools for measuring comprehensibility and appropriateness of multimedia rich patient education materials for patients with limited literacy skills.

Discussion

Effective communication of health-related information in print and Web-based formats requires providers to tailor the education materials to the appropriate levels of readability and health literacy of the target population. Pretesting the education material for the target population may allow clinicians to make appropriate modifications to the text and ultimately enhance comprehension by the intended consumers. Given the variability in the capacity to comprehend health-related materials among individuals who seek orthopaedic care, stratifying the contents of patient education materials at different levels of complexity will likely improve health literacy and enhance patient-centered communication.

One of the main criticisms against the use of readability tools is that improving readability by simplifying text may make information too simple and difficult to convey accurate information [51]. Although there have been many studies suggesting improving reading ease of patient education articles enhances health outcomes [1, 9, 20, 24, 32, 44], two studies show improving reading ease failed to improve patients’ comprehension of health-related information [34, 38]. However, this failure may be related to overestimation of the patient’s health literacy, resulting in inadequate education programs and thus warranting more customized solutions [38]. Another drawback of most readability formulas is that they are solely based on the length and structure of a sentence and ignore factors that influence comprehension such as illustrations, layout, and, most importantly, motivation of the reader [22, 34, 37]. Similar concerns have been raised against tools used to measure literacy skills and reading skills of patients. Tools such as REALM are purely based on reading skills and do not necessarily equate to comprehension ability [51]. In addition, limited literacy may manifest not only as poor reading skills but also as poor oral communication skills [51], for which tools such as REALM and TOFHLA may not suffice. Furthermore, administration of such methods may intimidate patients and raise their anxiety in an already overwhelming healthcare setting [22]. However, a recent study found 99% of patients readily participated when offered literacy screening with no decline in patient satisfaction at centers offering such screening [46].

It is well known that health literacy positively influences patient outcomes [1, 9, 20, 24, 32, 44]. Use of written patient education materials is a time-tested method of educating the patient and their caregivers about various ailments and their management [2, 11, 12, 44]. Though the availability of written patient education articles has increased over the years, there is a wide gap between the reading skills of the patient population and the readability of patient education materials. One of the ways to improve the comprehension of such material is to enhance its readability. The advent of computers has made the application of readability formulas within the reach of most orthopaedic surgeons.

Effective patient centered communication is a challenging goal. Given the abundance of existing patient education materials, it is virtually impossible to test all documents using currently available readability assessment tools. However, by raising the awareness among healthcare providers regarding the importance of matching the complexity of health education material to the patient’s level of health literacy, substantial gains in effective patient-centered communication can be made. Government sponsored organizations such as the CDC have played a pivotal role in bringing such issues to the forefront and can hopefully continue to spearhead such efforts. Organizations such as the American Academy of Orthopaedic Surgeons (AAOS) and various specialty organizations already have patient education materials available on their websites and can champion such cause using above noted guidelines. Similarly, editors of medical journals could require authors to submit an abbreviated online version with easier readability to be available for online access to all. Thus, by using a multifaceted approach with judicious use of the available tools to enhance the readability and comprehension of health-related materials, clinicians can further improve communication with their patients and hopefully enhance clinical outcome.