Readability of American Online Patient Education Materials in Urologic Oncology: a Need for Simple Communication

Abstract

Objectives

To determine readability levels of reputable cancer and urologic websites addressing bladder, prostate, kidney and testicular cancers.

Methods

Online patient education materials (PEMs) for bladder, prostate, kidney and testicular malignancies were evaluated from the American Cancer Society, American Society of Clinical Oncology (ASCO), National Cancer Institute (NCI), Urology Care Foundation (AUA-UCF), Bladder Cancer Advocacy Network (BCAN), Prostate Cancer Foundation (PCF), Kidney Cancer Association (KCA), and Testicular Cancer Resource Center (TCRC). Grade level was determined using several readability indices, and analyses were performed based on cancer type, website, and content area (general, causes, risk factors and prevention, diagnosis and staging, treatment, and post-treatment).

Results

Estimated grade level of online PEMs ranged from 9.2 to 14.2 with an overall mean of 11.7. Websites for kidney cancer had the least difficult readability (11.3) and prostate cancer had the most difficult readability (12.1). Among specific websites, the most difficult readability levels were noted for the AUA-UCF website for bladder and prostate cancer and the KCA and TCRC for kidney and testes cancer. Readability levels within content areas varied based on disease and website.

Conclusion

Online PEMs in urologic oncology are written at a level above the average American reader. Simplification of these resources are necessary to improve patient understanding of urologic malignancy.

Introduction

The use of online patient education materials (PEMs) as a source of health information is rising. From online websites to social media, PEMs represent an important adjunct for patient information beyond that obtained from clinic visits. National surveys estimate that 67% and 54% of adults aged 30–49 years and 50–64 years, respectively, have looked online for health information in the past year.¹ While PEMs are becoming increasingly available, this online information may exceed the literacy abilities of the typical American patient.

Health literacy is defined by the Institute of Medicine as “The degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions.”² Patient understanding of online PEMs represent one aspect of the larger definition of health literacy. An evaluation by the 2003 National Assessment of Adult Literacy assessed the English literacy of adults in the United States, and found that more than one third have limitations in health literacy (defined as a reading level below that of the 7^th grade).³

Patients’ understanding of disease may be strengthened by health information resources such as online PEMs. However, online PEMs are often beyond the reading level recommended for comprehension.^4–8 Limitations in health literacy are a significant issue among patients facing a cancer diagnosis.⁹ Cancer patients are often older and may face cognitive deficits, contributing to difficulty in understanding complex diagnostic and treatment decisions.

We assessed the readability of online PEMs from major cancer and urologic organizations for four urologic oncology cancer types (bladder, prostate, kidney and testes). We also evaluated differences in readability for specific content areas, including general information, causes, risk factors and prevention, diagnosis and staging, treatment, and post-treatment. We hypothesized that readability levels of online PEMs in urologic oncology would be, on average, higher than the nationally recommended basic reading level (6^th grade).¹⁰

Materials and Methods

Urologic oncology PEMs were obtained from a selection of commonly accessed oncology and urology websites, including the American Cancer Society (ACS),¹¹ American Society of Clinical Oncology (ASCO or Cancer.net),¹² National Cancer Institute (NCI),¹³ and the Urology Care Foundation of the American Urologic Association (AUA-UCF).¹⁴ Additional cancer type-specific websites were included: Bladder Cancer Advocacy Network (BCAN),¹⁵ Prostate Care Foundation (PCF),¹⁶ Kidney Cancer Association (KCA),¹⁷ and the Testicular Cancer Resource Center (TCRC).¹⁸ The websites selected were chosen to represent a sample of commonly accessed, reliable websites based on highest results from a Google search. Online PEMs for bladder, prostate, kidney and testicular malignancies were evaluated.

Content for each malignancy from each website was categorized into five groups: general information; causes, risk factors, and prevention; diagnosis and staging; treatment; and post-treatment. Each category’s text was then copied into a Microsoft Word document and analyzed using Readability Studio Professional Edition, Version 2012 for OS X (Oleander Software, Vandalia, Ohio). Grade level scores were determined using 7 readability assessment indices as described below. These readability assessments can also be calculated using other resources, but for the purpose of this study, we chose to use a comprehensive program that allows assessment of a variety of tools at once.

FORCAST was devised to assess U.S. Army technical manuals and forms and assesses the number of single-syllable words, and can be used for text without a running narrative (i.e. sentence fragments). The Fry Graph Readability Formula was designed for most text, including literature and technical documents, and calculates the average number of sentences and syllables per 100 word passages. The Gunning Fog Index is often used for business publications and journals, and is a weighted average of the number of words per sentence and the number of long words per word. The New Dale-Chall (NDC) is a formula generally used for primary and secondary age readers to classify school textbooks and literature. The NDC calculates the US grade level based on sentence length and the number of “hard” words (words that do not appear on a specially designed list of 3000 common words familiar to 4^th grade students). The Simple Measure of Gobbledygook (SMOG) readability formula is appropriate for secondary age (4^th grade to college level) readers and tests for 100% comprehension (whereas other formulas test for 50–75% comprehension). The SMOG formula estimates the years of education needed to understand a piece of writing, using the number of sentences and polysyllables (words of 3 or more syllables). The New Fog Count (Kincaid) is a modified version of the Gunning Fog Index created for the US Navy and was designed for technical documents and manuals. The Raygor Estimate was designed for most text, including literature and technical documents. The Raygor estimates US grade level by plotting the average number of sentences and letters per one hundred words. Results were also examined using the Flesch Readability Chart, which graphs words per sentence and syllables per word to determine a Readability Score that ranges from “very easy” to “very difficult.”

Results

Estimated overall readability of online PEMs across cancer types and websites averaged from grade levels 9.2 to 14.2 (Table 1). Evaluating each cancer type individually, the overall mean grade level was 11.7 across all cancer types with readability ranging from 11.3 for kidney cancer to 12.1 for prostate cancer. The most difficult readability grade levels were noted among the AUA-UCF for bladder and prostate cancer and the KCA and TCRC for kidney and testes cancer, respectively.

Table 1.

Estimated grade level readability of online PEMs from cancer and urology websites for bladder, kidney, prostate, and testicular cancer

Website for PEMs	Mean	FORCAST	Fry	Gunning Fog	NDC	New Fog Count	Raygor Estimate	SMOG
Bladder Cancer
American Cancer Society	10.3	10.7	10.0	10.6	10.3	8.4	10.8	11.3
American Society of Clinical Oncology	12.1	11.5	13.0	12.4	13.5	9.5	12.3	12.6
National Cancer Institute	11.2	11.1	12.4	10.5	10.9	7.9	13.4	11.9
Urology Care Foundation	13.8	11.7	16.0	14.1	14.3	10.8	15.7	13.9
Bladder Cancer Advocacy Network	11.1	11.3	12.0	11.1	12.6	7.8	11.0	12.2
Overall	11.7	11.2	12.7	11.7	12.3	8.9	12.6	12.4
Prostate Cancer
American Cancer Society	10.1	10.5	9.4	10.8	10.3	8.8	10.0	11.2
American Society of Clinical Oncology	12.1	11.3	12.4	12.6	12.1	10.2	13.4	12.6
National Cancer Institute	11.1	11.3	12.2	10.1	11.3	8.3	13.2	11.4
Urology Care Foundation	14.2	11.1	14.4	15.1	14.3	13.5	15.2	15.6
Prostate Cancer Foundation	12.8	11.1	13.4	13.4	13.0	10.9	13.4	14.1
Overall	12.1	11.1	12.4	12.4	12.2	10.3	13.0	13.0
Kidney Cancer
American Cancer Society	10.1	10.6	9.6	10.5	10.4	8.6	10.2	11.1
American Society of Clinical Oncology	11.4	11.2	11.5	11.6	12.4	8.6	12.5	12.1
National Cancer Institute	9.8	11.2	9.8	8.6	10.9	6.4	11.5	10.3
Urology Care Foundation	12.2	11.1	13.0	13.0	11.7	10.2	12.8	13.3
Kidney Cancer Association	13.0	12.4	16.3	12.1	14.3	7.3	15.7	12.9
Overall	11.3	11.3	12.0	11.2	11.9	8.2	12.5	11.9
Testicular Cancer
American Cancer Society	9.2	10.1	8.8	9.7	9.1	7.5	8.6	10.6
American Society of Clinical Oncology	12.2	11.0	13.4	13.4	12.0	10.1	11.8	13.5
National Cancer Institute	11.4	10.9	12.0	11.9	11.7	8.9	11.8	12.7
Urology Care Foundation	11.6	10.6	12.4	12.6	11.6	9.7	10.8	13.5
Testicular Cancer Resource Center	13.9	11.5	15.8	15.2	14.7	11.2	14.2	15.0
Overall	11.7	10.8	12.5	12.6	11.8	9.5	11.4	13.1

Readability based on content area was also assessed by cancer type and website. For bladder cancer, readability was more difficult for “Causes, Risk Factors, Prevention” and “Treatment” for most websites whereas “Post-Treatment” had the most difficult readability for AUA and ASCO (Figure 1a). Findings were similar for prostate (Figure 1b) and kidney cancer (Figure 1c). For testicular cancer, “Causes, Risk Factors, Prevention” was most difficult for the majority of websites, followed by “Treatment” and “Diagnosis & Staging” (Figure 1d).

Figure 1.

A : Mean grade level estimates for bladder cancer by content area and website

B : Mean grade level estimates for prostate cancer by content area and website

C : Mean grade level estimates for kidney cancer by content area and website

D : Mean grade level estimates for testicular cancer by content area and website

When comparing overall readability between each genitourinary cancer website, we found that all text passages were classified as fairly difficult or difficult based on the Flesch Readability Chart (Figure 2). The ACS received the highest score (59), representing the least difficult readability, while the AUA-UCF received the lowest score (39), representing the most difficult readability.

Figure 2.

Flesch Readability Chart Demonstrating Readability Score for Overall Urologic Oncology Information among PEM Websites*

*ACS, ASCO, NCI, AUA, BCAN, PCF, KCA, TCRC

Commonly used difficult words, defined as those with 3 or more syllables, were also evaluated. Difficult words were consistent across cancer types and cancer websites. These are detailed in Table 2.

Table 2.

List of Commonly Used Difficult Words in Online PEMs and Simplified Word Suggestions

Difficult Word	Simplified Word
Uncertainty	Doubt
Anxiety	Fear
Developing	Making, Growing
Cessation	Stop
Rapidly	Quickly
Variations	Changes, Differences
Recommend	Suggest
Investigate	Review
Advocate	Speak for
Evaluating	Checking
Eliminated	Cut, Dropped
Monitoring	Checking, Waiting
Typically	Often
Determine	Decide
Regimen	Routine
Benefit	Help
Incision	Cut
Abdomen	Belly, Stomach
Internal	Inner, Inside

Comment

Online PEMs represent an important source of health information among urologic oncology patients. Over the past decade, patients are increasingly likely to gain health information from Internet sources. Approximately 87% of US adults use the internet, and 72% of internet users say they looked online for health information within the past year.¹⁹ Even elderly Americans have demonstrated an increase in online use, with six in ten seniors now using the Internet compared to only 14% in 2000.²⁰ Given the rise of PEMs available to patients, readability assessment of these resources is a critical intermediate step to ensure the use of these materials matches their implicit intent of enhancing patient understanding and communication. Our study evaluated readability among well-recognized urologic oncology cancer websites for four genitourinary malignancies and found that mean grade level for urologic oncology PEMs was 11.7, well above the recommended 6^th grade level.¹⁰

The urologic oncology provider and patient advocacy community must critically evaluate the resources available, ensuring that readability is appropriate. Difficult readability may have the unintended consequence of leading patients to search other websites, which may be easier to understand but contain inaccurate information. Providing accurate yet readable online PEMs in urologic oncology will become increasingly important as the population ages, increasing the burden of these diseases, and an increasing proportion of patients look to the Internet for health information. A simple method of evaluating and altering online PEMs is identification and simplification of “difficult” words or words with 3 or more syllables.

Our study found several commonly used “difficult” words. While these words are commonplace in the medical field, lay patients may have difficulty understanding. Aside from difficult words due to medical jargon (e.g.“non-seminomatous” or “prostatic adenocarcinoma”), non-medical difficult words are also used when simpler words are possible. For example, “cessation” could be replaced by “stop” or “stopping.” “Variations” could be replaced by “changes.” There are a number of other commonly used words that could be easily simplified as listed in Table 2.

While word substitution is one specific method to increase understanding, a broad approach toward improving the readability of patient educational materials begins with a basic understanding of health literacy and steps to engage readers of all literacy levels. The National Institute of Health has published a simple guide “How to Write Easy-to-Read Health Materials” which outlines a set of steps geared toward creating a simple educational website.²¹ This was expanded by the National Cancer Institute with a “Clear and Simple” guide that outlines a step-wise process for developing health literature for persons with limited literacy skills.²² Five standard steps in developing educational material include defining the target audience, conducting audience research, developing a concept, developing content and visuals, and pretesting and revising draft materials.

Determining the target audience involves an assessment of reading level, cultural background, and age group. Once the audience is determined, target audience research in the form of surveys, focus groups, and interviews can assist with developing culturally relevant materials, establishing the baseline knowledge of your audience, and isolating existing information gaps. Using the information gathered from audience research, the concept and materials can be defined and developed. PEMs should include careful use of language (at a reading level appropriate to the target audience), concise writing style, simple layouts, and visual presentation when possible. In general, the recommended grade level should be within a range of a 6^th or 7^th grade, and the first few lines of text are the most important, as these are most critical for the reader to continue reading the material. Other general recommendations include use of examples and stories to engage the reader, logical structure with the most important points at the beginning of each paragraph, and use of lay language throughout.²²

Once the draft educational material is created, evaluation is imperative. Testing the materials on a few individuals or sample group from your target audience is a simple and easy way to ensure understanding. Furthermore, readability could easily be assessed using a variety of readability assessment tools presented in our results. A quick readability score can be calculated by pasting text into a variety of free online calculators, including www.readability-score.com.

Our study does have some limitations. First, we did not assess the content of each website, but rather the readability. However, all included websites were from reputable sources and presumed to be factually correct. Furthermore, we only sampled a small number of websites. However, based on prior studies in similar fields,^4–8 we hypothesize that much of the online PEMs in urologic oncology have similar readability scores. Finally, readability indices do not capture other significant characteristics that aid patients’ understanding of reading materials, including font size, text organization, context, and approach. Future studies could incorporate these items by enrolling patients in evaluating readability based on font size, number of images, and other characteristics (while also incorporating readability indices).

Conclusion

Readability of urologic oncology PEMs is clearly above national recommendations given the health literacy of US adults. Our study suggests that online urologic oncology PEMs should be refined to reach a wider audience, improving communication and understanding for cancer patients and their support network. Urologic oncology includes multiple complex diseases, and patients may not acquire the information needed for full understanding in a single clinic visit. Online PEMs represent an increasingly ubiquitous resource to fill this informational void. Our study reveals that readability of existing online PEMs from reputable organizations in urologic oncology are well above the average US reader, and simplification of these resources is needed to improve patient and caregiver understanding of these increasingly common genitourinary malignant conditions.