Most American Academy of Orthopaedic Surgeons’ Online Patient Education Material Exceeds Average Patient Reading Level

Abstract

Background

Advancing health literacy has the potential to improve patient outcomes. The American Academy of Orthopaedic Surgeons’ (AAOS) online patient education materials serve as a tool to improve health literacy for orthopaedic patients; however, it is unknown whether the materials currently meet the National Institutes of Health/American Medical Association’s recommended sixth grade readability guidelines for health information or the mean US adult reading level of eighth grade.

Questions/purposes

The purposes of this study were (1) to evaluate the mean grade level readability of online AAOS patient education materials; and (2) to determine what proportion of the online materials exceeded recommended (sixth grade) and mean US (eighth grade) reading level.

Methods

Reading grade levels for 99.6% (260 of 261) of the online patient education entries from the AAOS were analyzed using the Flesch-Kincaid formula built into Microsoft Word software.

Results

Mean grade level readability of the AAOS patient education materials was 9.2 (SD ± 1.6). Two hundred fifty-one of the 260 articles (97%) had a readability score above the sixth grade level. The readability of the AAOS articles exceeded the sixth grade level by an average of 3.2 grade levels. Of the 260 articles, 210 (81%) had a readability score above the eighth grade level, which is the average reading level of US adults.

Conclusions

Most of the online patient education materials from the AAOS had readability levels that are far too advanced for many patients to comprehend. Efforts to adjust the readability of online education materials to the needs of the audience may improve the health literacy of orthopaedic patients. Patient education materials can be made more comprehensible through use of simpler terms, shorter sentences, and the addition of pictures. More broadly, all health websites, not just those of the AAOS, should aspire to be comprehensible to the typical reader.

Introduction

In the United States, the mean adult reading level is an eighth grade level [21, 22, 43, 51]. Fourth grade text is difficult to comprehend for one of five US adults [22], and nearly 50% of adults “experience considerable difficulty in performing tasks that required them to integrate or synthesize information from complex or lengthy texts” [30]. The readability of education materials is measured by the grade level at which the written material is comprehended and educational materials should be written at the appropriate grade level for individuals to understand the content.

The “capacity to obtain, interpret, and understand basic health information and services and the competence to use such information and services to enhance health” defines a person’s health literacy [52]. Health-related quality of life is independently predicted by the level of one’s health literacy [49, 55, 57]. Moreover, lower health literacy correlates with higher rates of complication [48], hospitalization [6, 7, 48], increased costs [27, 53, 62], and overall worse health [8, 11, 29, 48, 50, 61].

Given adult reading skills in the United States and the important effects of health literacy on actual health, the National Institutes of Health (NIH) and American Medical Association (AMA) officially recommend that patient education materials not exceed the sixth grade reading level [18, 22, 36, 59, 60]. As suggested by several past studies, patient education materials may be too complex for some individuals to comprehend [2, 5, 17, 22, 38, 47, 54, 58].

The Internet has become a major source of patient education. Eight million individuals in the United States go online for health information everyday [24]. Studies have demonstrated orthopaedic patients use the Internet extensively for health education materials [9, 10, 15, 31]. Patients likely also use online education materials without being able to discuss its content with an orthopaedic surgeon.

The American Academy of Orthopedic Surgeons (AAOS) is an important purveyor of online health information about musculoskeletal conditions for patients. The AAOS is “the world’s largest medical association of musculoskeletal specialists” and “the preeminent provider of musculoskeletal education” [4]. The AAOS is a particularly high-traffic educational platform with more than 35,000 people visiting the website daily [47] and 6361 other websites linking directly to the website [3]. Surgeons often print online patient education handouts for their patients [26] and commonly refer patients specifically to the AAOS for reliable, peer-reviewed, up-to-date, and accurate content [47].

The goal of our study was to analyze if the readability of the AAOS online patient education materials are in accordance with AMA/NIH readability recommendations [18, 22, 36, 59, 60]. Specifically, we sought (1) to evaluate the mean grade level readability of online AAOS patient education materials; and (2) to determine what proportion of the online materials exceed recommended (sixth grade) and mean US (eighth grade) reading levels.

Materials and Methods

Our study analyzed the AAOS online patient educational materials found at www.orthoinfo.org/. The website was accessed between January 1 and February 2, 2014, and again on July 31, 2014. All 261 articles under “Parts of the Body” were evaluated, except the one article “Treatment Options for Low Back Pain” (video), which was excluded from analysis because it consisted predominantly of a video and contained minimal text.

For each of the 260 articles, text was copied into separate Microsoft Word 2010 (Microsoft Corporation, Redmond, WA, USA) documents in plain text format. Following the recommendation of Flesch [23] and others [25], all numbers, abbreviations, decimals, bullets, semicolons, paragraph breaks, colons, and dashes within a sentence were removed to avoid underestimating readability level. Any text not directly related to patient education was deleted, including author information, hyperlinks, copyright notes, date stamps, tables, and citations.

For each document, the well-validated Flesch-Kincaid Grade Level (FKGL) formula was used to analyze written material readability in terms of academic grade levels [2, 5, 13, 17, 44, 47, 54, 58, 63]. Although there are various readability assessment tools used in the healthcare setting, the FK formula is the most widely validated method of analyzing readability [2, 5, 13, 17, 44, 47, 54, 58, 63]. The FKGL was obtained using the Microsoft Word built-in readability calculator. Sequentially, selection of document tools “Review,” “Spelling & Grammar,” “Options,” and “Show Readability Statistics” enabled the FKGL calculator to automatically display a given document’s FKGL after the “Spelling & Grammar” function was used. The FKGL formula is: 0.39 × [average number of words per sentence] + 11.8 × [average number of syllables per word] − 15.59.

The FKGLs of the AAOS articles were compared with the AMA/NIH-recommended readability level (sixth grade) [18, 22, 36, 59, 60] as well as the mean US adult reading level (eighth grade) [21, 43, 51] with data throughout the article being presented as mean ± SD except where otherwise specified and comparisons being performed using unpaired t-tests. The cutoff for statistical significance was p < 0.05.

Results

For all AAOS patient education articles, mean grade level readability was 9.2 ± 1.6. The articles under “Shoulder & Elbow” patient education materials had a mean FKGL of 9.9 ± 1.4; “Hand & Wrist” articles had a mean FKGL of 8.7 ± 0.9; “Hip & Thigh” articles had a mean FKGL of 9.6 ± 1.5; “Knee & Lower Leg” articles had a mean FKGL of 9.2 ± 1.8; “Foot & Ankle” articles had a mean FKGL of 8.4 ± 1.1; and “Neck & Back” articles had a mean FKGL of 9.2 ± 1.7. Grade level readability for all sections was greater (p < 0.05) than the recommended sixth grade reading level (Table 1).

Table 1.

Flesch-Kincaid Grade Level readability by article section

Section (number of articles)	FKGL*	Grade levels above recommendation (FKGL 6.0)†
Shoulder & Elbow (46)	9.9 ± 1.4	3.9
Hand & Wrist (34)	8.7 ± 0.9	2.7
Hip & Thigh (35)	9.6 ± 1.5	3.6
Knee & Lower Leg (57)	9.2 ± 1.8	3.2
Foot & Ankle (40)	8.4 ± 1.1	2.4
Neck & Back (48)	9.2 ± 1.7	3.2
Total articles (260)	9.2 ± 1.6	3.2

* Mean ± SD; ^†all values are significant (p < 0.05); FKGL = Flesch-Kincaid Grade Level.

Of the total 260 articles we evaluated, 251 (97%) had a readability score above the sixth grade level. Readability of the articles exceeded the sixth grade level by an average of 3.2 grade levels (95% confidence interval [CI], 3.0–3.4; p < 0.001). The readability score for 210 (81%) of the 260 articles was above the eighth grade level. Readability of the articles exceeded the eighth grade level by an average of 1.2 grade levels (95% CI, 1.0–1.4; p < 0.001; Fig. 1).

Fig. 1.

The distribution of AAOS patient education articles by Flesch-Kincaid grade level.

Discussion

Advancing health literacy has the potential to improve patient outcomes. It is therefore important that trusted and much-used sources of health information like the AAOS be comprehensible to most users, but it is unknown whether they are. Readability analysis of the AAOS online patient education materials has broad applicability and relevance. Because the AAOS is an expert website that is visited daily by more than 35,000 individuals [47], linked to 6361 other websites [3], and a commonly recommended and used patient education resource of orthopaedic surgeons [26, 28, 31, 42, 47], it serves as an important example bringing attention to the effects of readability of patient education materials on health literacy and health outcomes. This analysis highlights the need for orthopaedic surgeons and societies to appreciate the concept of readability when creating any patient materials, including educational materials and consent forms.

Despite the heavy use of web-based health information [9, 10, 15, 24, 31], only 12% of adults have the health literacy skills to navigate their health issues. According to the National Assessment of Adult Literacy [45], the mean reading level for adults in the United States is of an eighth grade level [21, 22, 43, 51]. Moreover, even patients with stronger literacy skills prefer lower grade level online content in simpler formats [19, 56]. Taken together, the AMA/NIH recommend that patient education materials not exceed a sixth grade reading level [18, 22, 36, 59, 60].

In this study, we found that the average readability of AAOS online patient education materials exceeds the AMA/NIH-recommended health information readability level and the average US adult reading level. A substantial portion of the intended patient audience may not effectively comprehend the content, which can negatively affect health literacy, contributing to poorer health outcomes, increased hospitalizations, and higher healthcare costs [12].

Our study had several limitations. Although the FKGL scoring system is well validated [2, 5, 13, 17, 44, 47, 54, 58, 63], it only factors in written text—not illustrated, video, or audio content, which may improve understanding of content. An important additional potential study limitation is the assumption that the reading skills of those reading the AAOS patient education materials are equivalent to the general US adult population. However, improved Internet access makes the online population increasingly similar to the general population [2]. Online readers may have reading skills comparable to the general population; one-third of those seeking health information online have a high school diploma or less education [2]. Of the adults with “below basic health literacy,” 20% get their health information online [33]. Consequently, most studies assess online material readability compared with mean adult reading skill level [12, 14, 19, 25, 26, 34, 56]. Because www.orthoinfo.org is publically accessible, the reading level of these materials should match the reading skills of any potential orthopaedic patient or the general population.

Orthopaedic surgeons in practice may not depend on written AAOS patient teaching information alone to prevent postsurgical disasters related to uninformed patient behavior. Any written patient education material should take an immediate backseat to the critical process and content of the dialogue between the treating surgeon and the patient. Ostensibly, physicians should be able to judge the general reading comprehension level of each patient. Subsequently, the informing surgeon should adjust his or her teaching points based on an eye-to-eye assessment of the patient’s capacity to absorb and fully comprehend the specifics of the informed consent dialogue.

Sabharwal et al. [47] analyzed AAOS patient education materials produced between September 1999 and July 2006 and found the average FKGL was 10.4. Our findings suggest that the AAOS has improved the readability of its patient education materials by 1.2 grade levels over the last 8 years; however, further improvement is still needed. Such minimal improvement over time may reflect a lack of awareness in the orthopaedic community regarding the concept of readability.

Patient education material readability can be analyzed simply using widely available software as demonstrated by the methods of this study. Patient education materials can then be made more comprehensible through use of simpler terms, shorter sentences, and the addition of pictures [2, 18]. Additional media formats such as brief YouTube videos gauged to a sixth grade level of science comprehension may improve communication. Multiple other health information websites (eg, [1, 32, 35, 37, 39–41, 46]) have proven it is actually possible to achieve appropriate level patient education materials. Making the material more readable must and can be done without losing important content. There are multiple publically accessible tools (eg, [16, 20, 52]) that can be used to substitute complex medical jargon with simpler lay terms and improve the readability of patient educations materials. If leading expert websites, like the AAOS, improve readability of their patient education materials, other health information websites could follow suit, starting a trend toward optimizing all patient education materials.

With increased use of online health information, the study of health education is a particularly relevant subject. The full value of online patient education materials cannot be realized unless readers can comprehend the content. To improve health literacy and, in turn, health outcomes, modifying readability of patient education materials to levels comprehendible by the average reader should be emphasized.