Readability of Trauma-Related Patient Education Materials From the American Academy of Orthopaedic Surgeons

Abstract

Context:

According to the american medical association (AMA) and the national institutes of health (NIH), the recommended readability of patient education materials should be no greater than a sixth-grade reading level. The online patient education information produced by the american academy of orthopaedic surgeons (AAOS) may be too complicated for some patients to understand. This study evaluated whether the AAOS’s online trauma-related patient education materials meet recommended readability guidelines for medical information.

Evidence Acquisition:

Ninety-nine articles from the “Broken Bones and Injuries” section of the AAOS-produced patient education website, orthoinfo.org, were analyzed for grade level readability using the Flesch-Kincaid formula, a widely-used and validated tool to evaluate the text reading level. Results for each webpage were compared to the AMA/NIH recommended sixth-grade reading level and the average reading level of U.S. adults (eighth-grade).

Results:

The mean (SD) grade level readability for all patient education articles was 8.8 (1.1). All but three of the articles had a readability score above the sixth-grade level. The readability of the articles exceeded this level by an average of 2.8 grade levels (95% confidence interval, 2.6 - 3.0; P < 0.0001). Furthermore, the average readability of the articles exceeded the average reading skill level of U.S. adults (eighth grade) by nearly an entire grade level (95% confidence interval, 0.6-1.0; P < 0.0001).

Conclusions:

The majority of the trauma-related articles from the AAOS patient education website have readability levels that may make comprehension difficult for a substantial portion of the patient population.

1. Context

Patients commonly utilize the Internet to access health information (1-7). Every day, an estimated eight million Americans seek health information online (5). In order for patients to use the health information online to inform healthcare decisions, they must first be able to comprehend the material (7). The requisite reading comprehension level a person must have to understand written material is determined by the readability of the text (8). The flesch-kincaid grade level (FKGL) formula is a commonly utilized and validated instrument for determining the readability of written materials in terms of U.S. academic grade levels (9-17). Higher FKGL texts require more advanced reading skills.

Approximately 20% of adults in the U.S. cannot comprehend above fourth-grade-level texts (18). Furthermore, nearly half of American adults experience considerable difficulty in synthesizing information from complex or lengthy texts (19), and the average patient reads five grade levels below their reported graduation grade (20). The average American adult reads at an eighth-grade level (18).

Health literacy is defined as the “capacity to obtain, interpret, and understand basic health information and services and the competence to use such information and services to enhance health” (21). Health literacy has been shown to predict health-related quality of life (22-24). Lower health literacy correlates with overall increase in health-care costs (25-28), worse general health (29-34), poor understanding of one’s disease (35-41), increased complications (30), and increased hospitalizations (42, 43).

The national institutes of health (NIH) and the american medical association (AMA) recommend patient education materials should be no greater than a sixth-grade reading level (44-48). Other studies have suggested that currently utilized patient education materials may be too complex for most patients to comprehend (9, 10, 12, 13, 15, 17, 18, 49). To our knowledge, no one has specifically assessed the readability of the trauma-related patient information produced by the american academy of orthopaedic surgeons (AAOS). As noted on the website, materials “were developed in partnership with the orthopaedic trauma association” (OTA).

Given that the AAOS is an expert source of information, the prevalence of trauma-related orthopedic problems, and the effect of health literacy on patient outcomes, the goal of our study was to evaluate the readability of trauma-related patient education materials produced by the AAOS to assess if it meets recommended medical information readability guidelines.

2. Evidence Acquisition

The analyzed trauma-related articles from the AAOS patient education website were found under the “Broken Bones and Injuries” section (www.orthoinfo.org/menus/injury.cfm). On the website, articles are further categorized into “Shoulder and Elbow,” “Hand and Wrist,” “Hip and Thigh,” “Knee and Lower Leg,” “Foot and Ankle,” “Neck and Back,” and “General.” The average reading levels for these subsections were analyzed. Our study was exempt from IRB review. The website was accessed between November 3 and 15, 2013. All patient education articles were assessed, excluding those written in non-English, with content predominately in graphic/pictorial form, or table/list format. We hypothesized that the readability of these materials would have a mean FKGL of > 6.

Each article’s text was copied in plain text to individual Microsoft Office Word 2010 (Microsoft Corporation, Redmond, WA) documents. As recommended by Flesch and others, all numbers, decimals, bullets, abbreviations, paragraph breaks, colons, semicolons, and dashes within a sentence were removed in order to avoid underestimating the readability level (50, 51). Any text not directly related to patient education was deleted, including copyright notes, date stamps, author information, hyperlinks, citations, and tables.

For each document, FKGLs were obtained using the readability calculator built into the Word software. FKGL calculator feature was enabled by sequentially selecting “Review,” “Spelling and Grammar,” “Options,” and “Show Readability Statistics.” FKGL is calculated by: 0.39 × (average number of words per sentence) + 11.8 × (average number of syllables per word) - 15.59. After grammar and spelling was checked for each document, FKGL was automatically displayed. Each FKGL was calculated and recorded by the same reviewer.

Unpaired t-tests were done in Microsoft Office Excel 2010 (Microsoft Corporation, Redmond, WA) to compare the mean FKGL with the recommended sixth-grade readability level and the average American adult reading level. A P-value of < 0.05 was used to determine significance.

3. Results

One hundred and seven patient education articles were evaluated for inclusion. Four articles could not be analyzed because they contained predominately video and photo content. Four other articles were used two times in different sections of the website. These articles were only analyzed once. Therefore, 99 articles met the inclusion criteria and were evaluated for FKGL.

The average (SD) FKGL for all the patient education materials was 8.8 (1.1). The readability of the articles exceeded the sixth grade level by an average of 2.8 grade levels (95% confidence interval, 2.6-3.0; P < 0.0001). Furthermore, the average readability of the articles exceeded the average reading skill level of U.S. adults (eighth grade) by nearly an entire grade level (95% confidence interval, 0.6-1.0; P < 0.0001).

Only three articles “Fracture of the Finger,” “Frostbite,” and “Wrist Sprains” were at or below a sixth-grade reading level, thus 97.0% of the articles were above the recommended reading level (Figure 1).

Figure 1. Patient Education Articles on AAOS-Produced Orthoinfo.org, Sorted by Grade Level Readability.

The articles under “General” patient education materials had a mean FKGL of 9.1, “Neck and Back” articles had a mean FKGL of 9.5, “Shoulder and Elbow” articles had a mean FKGL of 9.1, “Hip and Thigh” articles had a mean FKGL of 8.7, “Knee and Lower Leg” articles had a mean FKGL of 8.4, “Foot and Ankle” articles had a mean FKGL of 8.4, and “Hand and Wrist” articles had a mean FKGL of 7.9 (Table 1).

Table 1. Distribution of Articles by Grade Level and Sub-Section Average.

Subsection	Average FKGL	Grade 4 - 6	Grade 7 - 8	Grade 9 - 10	Grade 11 - 12	Total Articles
General	9.1 ± 1.3	2	8	6	4	20
Shoulder and Elbow	9.1 ± 1.2	0	14	10	2	26
Hand and Wrist	7.9 ± 0.6	1	9	0	0	10
Hip and Thigh	8.7 ± 1.0	0	4	4	0	8
Knee and Lower Leg	8.5 ± 0.7	0	11	3	0	14
Foot and Ankle	8.4 ± 0.6	0	11	0	0	11
Neck and Back	9.5 ± 1.1	0	3	7	0	10
Total	8.8	3	60	30	6	99

4. Conclusions

The readability level of trauma-related patient education materials on AAOS-produced orthoinfo.org may be too advanced for many patients. The mean reading grade level of the website’s patient education materials was 8.8, which was nearly 3 full grade levels beyond the recommended grade level recommended by the NIH and AMA.

In 2007, Sabharwal et al. analyzed all online patient education articles created by AAOS between 1999 and 2006 and found a mean reading level of 10.4 (13). Although we are unable to directly compare individual articles, our findings suggest that the AAOS may have taken notice and modified the readability of its patient education materials.

When developing patient education materials, the need to provide complete and accurate medical information should be balanced with the reading skills of the intended audience. The readability of patient education materials can be enhanced by using simpler terms, shorter sentences, and illustrations (10, 47, 52, 53). The findings from this study may be of particular interest to the OTA, as the patient education section of the OTA website consists entirely of an external link to orthoinfo.org.

Many medical subspecialty organizations have created patient education materials in an attempt improve the health literacy and outcomes of their patients. This may be due to evidence indicating that low levels of health literacy correlates with lower health-related quality of life, worse general health, and more hospitalizations and complications. Fewer than 20% of adults have the health literacy skills needed to properly manage their health; according to the National Assessment of Adult Literacy (54), improving health literacy may, in turn, improve patient outcomes. More than just patient outcomes, literacy has been shown to have a significant impact on healthcare utilization and costs. Health literacy has been shown to be highly correlated with follow-up rates and compliance with, often complicated, written discharge instructions (25). Thus these organizations, such as the AAOS, may consider lowering the readability level of patient education materials. A recent survey has found that “Fifteen percent of adults with below basic health literacy used the Internet ‘some’ or ‘a lot’ for information on health topics, compared with 31 percent of those with basic health literacy, 49 percent with intermediate health literacy, and 62 percent of those with proficient health literacy” indicating that there is a mismatch between the general population of patients, to which the documents are designed and the population reading these educational articles (55). Whether decreasing the FKGL of patient education materials actually improves health literacy is a hypothesis which will require examination.

We acknowledge several potential limitations. FKGL evaluates text only (i.e., not diagrams), and does not directly measure comprehensibility. Additionally, we did not assess the reading skills of the website’s visitors, which may differ from the general population. Although we limited our investigation to only the trauma-related patient education materials, orthopaedic surgeons increasingly refer their patients to such professional websites, making this investigation relevant (2, 56-58). Despite these limitations, this study hopefully provides important data regarding the readability of the AAOS’s publically available patient education materials.

For many patients, the AAOS-produced trauma-related patient education materials are too complex to understand. Optimizing patient education materials may enhance patient comprehension, and in turn improve health outcomes.