Evaluation of readability of patient education materials on lateral epicondylitis (tennis elbow) from the top 25 orthopedic institutions

Abstract

Hypothesis

Lateral epicondylitis (LE), also known as “tennis elbow,” is a common orthopedic tendinosis of the elbow that affects between 1% and 3% of the overall population. LE occurs due to overuse of the extensor mechanism or muscles of supination in the forearm. The National Institute of Health and the Center for Disease Control suggest that patient education materials (PEMs) be written at a 6th or 8th grade reading level, respectively; however, PEMs are often written far beyond these recommended reading levels. The goal of this study was to assess the readability level of PEMs published by some of the top orthopedic institutions throughout the United States.

Methods

A list of the top 25 ranking orthopedic hospitals in the country was compiled using the 2022 U.S. News and World Report Best Hospitals Specialty Ranking. PEMs related to LE were cataloged from each institution’s website, and readability levels for each PEM were measured using the http://www.readabilityformulas.com website. This software analyzes readability using the formulas listed in Table 1. While the Flesch-Kincaid (FK) Reading Ease Score formula outputs a number from 0 to 100, in which larger numbers indicate easier reading, the remaining formulas demonstrate a text’s readability through assigning a grade-appropriate reading level. A Spearman regression was used to evaluate correlation between institutional ranking and FK Reading Ease Scores.

Results

Of the 25 PEM texts analyzed during this study, none were written at or below the sixth grade reading level, as recommended by the National Institutes of Health. These results suggest that the most prestigious orthopedic hospitals provide online informational resources that are unable to be read or understood by a large portion of the institution’s intended audience. Additionally, there was no correlation found between institutional ranking and FK Reading Ease Score.

Conclusion

Internet-based health information has conveniently allowed patients to educate themselves on their health care. In accordance with National Institutes of Health and Centers for Disease Control and Prevention guidelines, orthopedic institutions should strive to publish PEMs at or below an eighth grade reading level.

Patients are increasingly utilizing internet-based resources and patient education materials (PEMs) to learn about common medical conditions and treatments. Convenient access to accurate online information can be a positive influence in the patient-doctor relationship; however, this is dependent on the patient’s ability to read, interpret, and understand the information presented. Physicians are trained to decipher and understand the multifaceted nuances of scientific literature with complex statistics and multilayered analyses. Translating these highly detailed papers for the general public to understand can be quite challenging. Patient health literacy illustrates a major challenge to health care providers in terms of effectively communicating with patients through written PEMs.

Reports by the National Center for Education Statistics indicate that Americans read at roughly an eighth-grade reading level.⁹ The National Institute of Health and the Center for Disease Control suggest PEMs be written at a level of complexity that a theoretical sixth and eighth grader should appropriately be able to understand (reading-grade level), respectively.¹² Furthermore, the correlation between poor health literacy and worse clinical outcomes has been well established in the literature.^{4,10,13,14,19} The need for providers to appropriately communicate with their target audience cannot be overstated.

Lateral epicondylitis (LE), also known as “tennis elbow,” is a common orthopedic tendinosis of the elbow that affects between 1% and 3% of the overall population.¹ LE poses a unique obstacle with respect to the readability of PEMs, as individuals who hold blue-collar professions that are physical-labor intensive (ie, factory worker, industrial laborer, plumber, landscaper, etc.) are often at an increased risk for overuse injuries such as LE.^3,8 Additionally, these individuals are often associated with lower reading-grade levels on average.¹⁸ Our goal was to assess the readability level of the texts included in PEMs for LE published by some of the top national orthopedic institutions, and to determine if PEMs on LE have been written with this target audience in mind.

Methods

A list of the top 25 ranking orthopedic hospitals in the country was compiled using the 2022 U.S. News and World Report Best Hospitals Specialty Ranking. PEMs related to LE were cataloged from each institution’s website. Several, but not all, of the institutions included in this study specified on their website that these PEMs were written or reviewed by physicians affiliated with these institutions. Institutions were excluded from the study if no relevant PEMs existed. PEMs were then converted to text-only format, and readability levels were measured using the http://www.readabilityformulas.com website software. This software analyzes readability using the following tests: Flesch-Kincaid (FK) Reading Ease Score, Gunning Fog, The FK Grade Level, The Coleman-Liau Index, Simple Measure of Gobbledygook (SMOG) Index, Automated Readability Index, and Linsear Write Formula. The calculations performed for each test are displayed in Table I. The EK Reading Ease Score formula outputs a value from 0 to100, in which a larger number indicates an easier reading level. The remaining formulas demonstrate a text’s readability through assignment of a corresponding grade-appropriate reading level.

Table I.

Description of readability tools and their corresponding formulas.

Readability tool	Formula
Flesch-Kincaid Reading Ease Score	Readability Ease = 206.835−(1.015 × Average Sentence Length)−(84.6 × Average Number of Syllables per Word)
Gunning Fog	Grade Level = 0.4 × (Average Sentence Length/Percentage of Hard Words)
Flesch-Kincaid Grade Level	Grade Level = (0.39 × Average Sentence Length) + (11.8 × Average # Syllable per Word)−15.59
The Coleman-Liau Index	Grade Level = 0.0855 × (Average # of letters per 100 words)−0.296 × (Average # of sentences per 100 words)−15.8
SMOG Index	Grade Level = 3 × Square Root of Polysyllable Count
Automated Readability Index	Grade Level = 4.71 × (characters/words) + 0.5 × (words/sentences)−21.43
Linsear Write Formula	n = [(2 Syllables words ∗ 1) + (3 or more syllable words ∗ 3)]/Number of Sentences
	If n < 20, Grade Level = n/20
	If n > 20, Grade Level = n−2/20

SMOG, Simple Measure of Gobbledygook.

The mean and standard deviation were calculated for the results of each readability test. To assess collinearity between the readability tests, variance inflation factors (VIFs) were calculated. VIF values of greater than or equal to ten suggested sufficient collinearity between tests and readability scores.¹¹ Finally, Spearman Regression modeling was used to evaluate a correlation between FK Reading Ease Scores and institution ranking.

Results

All 25 orthopedic institutions had PEMs related to LE that could be analyzed for this study. The mean readability score out of 100 for the FK Reading Ease Score was 51.6 ± 10.8. The remaining scores were as follows: Gunning Fog, 12.7 ± 2.3; FK Grade Level, 10.7 ± 2.3; The Coleman Liau Index, 10.8 ± 1.6; SMOG Index, 9.3 ± 1.7; Automated Readability Index, 10.8 ± 2.9; Linsear Write Formula, 11.6 ± 3.4. Individual readability scores and institution rankings are summarized in Table II. Interestingly, only 28% of the institutions’ PEMs were written at or below the eighth-grade reading level, as recommended by the Centers for Disease Control and Prevention (CDC) (Fig. 1). No PEMs were written at or below the sixth-grade reading level, as recommended by the National Institutes of Health (NIH).

Table II.

Individual and mean grade-levels and readability ease for the top 25 orthopedic institutions.

Institution ranking	Flesch-Kincaid Reading Ease Score∗	Gunning Fog	Flesch-Kincaid Grade Level	The Coleman-Liau Index	SMOG Index	Automated Readability Index	Linsear Write Formula
1	44.3	14.9	12.1	12.0	11.0	12.6	13.7
2	48.1	13.0	12.2	11.0	9.1	12.6	13.7
3	54.6	10.7	8.8	11.0	8.0	8.2	7.2
4	58.7	12.6	9.8	9.0	9.2	10.0	12.1
5	56.2	11.1	9.2	10.0	8.4	8.3	8.6
	64.6	10.6	8.6	9.0	7.6	9.0	9.4
7	44.6	12.1	11.0	13.0	9.8	11.2	10.8
8	68.5	10.2	8.0	8.0	7.2	7.8	9.1
9	43.3	13.0	11.0	12.0	9.8	10.2	9.5
10	69.1	8.4	6.7	8.0	6.1	5.3	6.3
11	63.7	11.5	8.9	9.0	8.5	9.0	11.3
12	68.5	9.4	7.2	8.0	6.9	6.5	7.4
13	52.7	13.3	10.8	11.0	9.7	11.0	12.5
14	53.6	10.7	8.6	12.0	8.0	8.4	6.6
15	44.2	13.7	11.7	12.0	10.5	12.0	12.6
16	40.5	12.6	11.0	13.0	9.5	10.1	8.4
17	39.5	16.1	14.3	12.0	11.6	15.6	17.2
18	41.8	15.1	12.4	12.0	11.4	12.3	13.8
19	41.1	15.3	13.3	12.0	11.7	14.1	15.8
20	48.1	13.0	12.2	11.0	9.1	12.6	13.7
21	39.1	16.0	13.5	12.0	11.6	14.2	15.6
	35.8	16.8	15.7	12.0	11.4	17.3	18.9
23	62.9	11.2	9.1	9.0	8.2	9.2	11.2
24	64.6	10.5	8.6	9.0	7.6	9.0	9.4
	42.2	15.5	13.1	12.0	11.0	13.9	15.1
Average (SD)	51.6 (10.8)	12.7 (2.3)	10.7 (2.3)	10.8 (1.6)	9.3 (1.7)	10.8 (2.9)	11.6 (3.4)

SD, standard deviation; SMOG, Simple Measure of Gobbledygook.

^∗Flesch Reading Ease Score is out of 100. All remaining scores are grade-levels.

Figure 1.

Flesch-Kincaid grade readability scores on patient education materials on lateral epicondylitis from some of the top orthopedic institutions in relation to the average American reading grade level. () denotes the average American reading-grade level of eighth grade, per NCES. () denotes the individual Flesch-Kincaid grade readability scores for each of the top 25 orthopaedic institutions included in this study. NCES, National Center for Education Statistics; SMOG, Simple Measure of Gobbledygook.

Our results of collinearity analysis demonstrated a high degree of collinearity for each test relative to the FK Reading Ease Score. The VIFs for individual tests all exceeded 10 and are as follows: Gunning Fog = 86.6, FK Grade Level = 79.5, The Coleman Liau Index = 24.6, SMOG Index = 64.2, Automated Readability Index = 117.2, and Linsear Write Formula = 75.0. Furthermore, Spearman Regression revealed no significant correlation between institution ranking and FK Reading Ease Score (ρ = −0.25; P = .259).

Discussion

The results of this study found that PEMs related to LE, published by some of the nation’s most prestigious orthopedic institutions, have poor readability. The term ‘poor readability’ referring to PEMs may be defined as being incongruent with CDC and NIH recommended guidelines, which currently suggest PEMs be written at or below an eighth and sixth-grade reading level, respectively. These results indicate that PEMs are often written at a reading-grade level that is too complex for the majority of the intended patient audience to accurately read and understand. Furthermore, institutional ranking is not significantly correlated with readability scores. The individual readability tests display a high degree of collinearity.

The finding of poor readability of PEMs related to LE is not an isolated singular issue, as prior studies have demonstrated similar results during analysis of PEMs in numerous other orthopedic and nonorthopedic subspecialties. Stelzer et al demonstrated poor readability for PEMs related to shoulder arthroplasty as well as no significant correlation between institutional rank and readability.¹⁶ Additionally, Hartnett et al reported similar results within the area of foot and ankle surgery.⁷ Williams et al demonstrated poor readability with respect to PEMs for ophthalmology.²⁰ Hence, poor readability is not an unique issue limited to online orthopedic PEMs, but it is one that should be remediated, nonetheless.

Numerous articles have proposed guidelines for health care professionals to revise written PEMs whose readability levels do not meet NIH, CDC, or similar organizational recommendations.^2,4,5,17 Such recommendations include: substituting complex medical jargon with simple vernacular, limiting sentence length to 8-10 words, limiting paragraphs to 3-5 sentences, using diagrams/audiovisual multimedia in place of complex explanations, limiting the number of messages or main ideas per document, and limiting vocabulary to 1 or 2 syllable words when possible. In summary, the CDC suggests to “write as if you were talking to a friend.”⁶

When put into practice, these recommendations have been shown to drastically improve the readability of PEMs by patients. Sheppard et al was able to achieve an average reduction of 1.67 reading-grade levels for PEMs related to foot and ankle surgery published on the American Academy of Orthopaedic Surgeons, American Orthopaedic Foot and Ankle Society, and individual academic institution websites.¹⁵ Furthermore, by soliciting patient feedback, Williams et al qualitatively analyzed patient perceptions and approval of revised PEMs.²⁰ These studies reinforce the notion that reducing the reading-grade level of PEMs is not only relatively straight-forward, but widely beneficial with respect to promoting health literacy in patients who wish to educate themselves using these resources.

Our study has several notable limitations. First, readability scores are only one component used to describe the complexity of PEMs. Other factors such as the use of audiovisual multimedia or general website layout and design were unable to be fully analyzed during this study. Figures, charts, and other graphics were not analyzed during this study. Furthermore, several institutions did not specify who wrote or reviewed the PEMs published on their website, while some institutions did clarify they were written by physicians. It is unclear if this may have confounded our results. Additionally, this study’s analysis focused on PEMs from the top 25 orthopedic institutions, as reported by the 2022 U.S. News and World Report Best Hospitals Specialty Ranking. These PEMs and corresponding readability levels may not be fully representative of PEMs offered by other orthopedic institutions throughout the country. We encourage administrative faculty throughout the country to critically evaluate their own institutions PEMs, and to modify when appropriate. Finally, commercial aspects of the contents of the PEMs used to promote the concerned institutions could have contributed to low readability. This was beyond the scope of our study, but may be a topic of further research.

Conclusion

This study has demonstrated that PEMs related to LE (tennis elbow) from the highest ranking orthopedic instructions are too often written at a reading level that is too advanced for the majority of Americans to adequately comprehend. In accordance with NIH and CDC guidelines, orthopedic institutions should strive to publish PEMs at or below an eighth-grade reading level. Through improving the readability of PEMs, orthopedic institutions will expand outreach, promote health care literacy, and ultimately provide better patient outcomes.

Disclaimers

Funding: No funding was disclosed by the authors.

Conflicts of interest: The authors, their immediate families, and any research foundations with which they are affiliated have not received any financial payments or other benefits from any commercial entity related to the subject of this article.