Readability assessment of concussion and traumatic brain injury publications by Centers for Disease Control and Prevention

Abstract

Health literacy is associated with a person’s capacity to find, access, contextualize, and understand information needed for health care-related decisions. The level of health literacy thus has an influence on an individual’s health status. It can be argued that low health literacy is associated with poor health status. Health care literature (eg, pamphlets, brochures, postcards, posters, forms) are published by public and private organizations worldwide to provide information to the general public. The ability to read, use, and understand is critical to the successful application of knowledge disseminated by this literature. This study assessed the readability, suitability, and usability of health care literature associated with concussion and traumatic brain injury published by the United States Centers for Disease Control and Prevention. The Flesch–Kincaid Grade Level, Flesch Reading Ease, Gunning Fog, Simple Measure of Gobbledygook, and Suitability Assessment of Materials indices were used to assess 40 documents obtained from the Centers for Disease Control and Prevention website. The documents analyzed were targeted towards the general public. It was found that in order to be read properly, on average, these documents needed more than an eleventh grade/high school level education. This was consistent with the findings of other similar studies. However, the qualitative Suitability Assessment of Materials index showed that, on average, usability and suitability of these documents was superior. Hence, it was concluded that formatting, illustrations, layout, and graphics play a pivotal role in improving health care-related literature and, in turn, promoting health literacy. Based on the comprehensive literature review and assessment of the 40 documents associated with concussion and traumatic brain injury, recommendations have been made for improving the readability, suitability, and usability of health care-related documents. The recommendations are presented in the form of an incremental improvement process cycle and a list of dos and don’ts.

Introduction

The American Medical Association reported that one in every three patients in the United States (US) has basic or below basic health literacy.¹ Inefficiencies and ineffectiveness in delivering health care due to poor health literacy cost US$58 billion per year.¹ Within the context of this study, health literacy can be best defined as the degree to which individuals have the capacity to access, process, and understand basic health information and obtain services needed to make appropriate health-related choices.² MedLinePlus identified that when a person’s health literacy is limited, (s)he has difficulty doing things such as filling out forms, finding providers, and understanding the specifics of medication.³ The objectives of the Healthy People 2020 project include the improvement of health literacy amongst the population.⁴ Specifically the project aims to increase the proportion of persons who report that their health care provider always gave them easy-to-understand instructions related to their illness or health condition.

Naidu showed that on an international level, health literacy has an influence on an individual’s health status and that low health literacy is related to low health status.⁵ Further studies by Lauder et al on patients aged ≥ 65 years determined that low health literacy had a strong relationship with mortality.⁶ They reported that within 67.8 months of a treatment, 39.4% of patients with low health literacy died as opposed to 18.9% with high health literacy. Osborn et al further clarified the relationship between health literacy and mortality.⁷ They indicated that literacy was a significant predictor of human immunodeficiency virus patients’ nonadherence to a medicine regimen. Solutions to the problem of low health literacy being prevalent in society are varied in that they include both putting the onus on the patient and the health care provider to ensure effective health understanding. Research by Hay concerning the use of the Internet by patients showed that while health literacy has improved, so has the need for patients to empower themselves to make use of this new technology in a more effective manner and to take on their own health.⁸ These materials were designed for the general public. Davis et al identified that the average reading ability in the US is equal to that of an eighth grader in the US school system.⁹ A survey conducted by the National Assessment of Adult Literacy identified that about 40% of the 19,000 subjects surveyed were below the basic level of proficiency in prose, document, and qualitative literacy skills.¹⁰ The study aims to add to the growing body of literature about the importance of improving health literacy, which in turn aids better patient care. This is achieved by examining the readability of pamphlets and brochures related to concussion and traumatic brain injury available on the Centers for Disease Control and Prevention (CDC) website for the general public.

The paper is structured to present a review of existing studies on the readability of health care materials. The focus is then limited to concussion and traumatic brain injury publications from CDC. This is followed by a presentation of the analyses of the documents collected. The results of the analyses provide the basis for recommendations on improving the readability of health care materials.

Readability and health literacy

Parker et al specified that multiple governmental organizations such as the Joint Commission on Accreditation of Health Care Organizations and the National Committee for Quality Assurance are focused on creating guidelines for patient readability of materials.¹¹ Williams et al put forth that the reason health literacy is a problem worthy of research is due to the gap that exists between the literacy skills of the average American and the readability level of patient education materials published for public consumption.¹² McCray put forth that the use of grade level of reading ability has emerged as a standard in understanding health care document readability.¹³ Doak et al suggested modifying terminology to use common words and phrases, larger font sizes, and larger space between lines in text.¹⁴ They felt that this would make readability easier for people who fall below this mean.

A standard instrument to measure health literacy is the Test of Functional Health Literacy in Adults. In a study of urban public hospitals, the Test of Functional Health Literacy in Adults determined that health literacy was significantly higher in patients aged ≥ 60 years than in younger patients.¹⁵ While age influences readability, Gazmararian et al constructed a study to understand readability’s influence.¹⁶ They showed that discharge instructions, consent forms, and medical education brochures often are written at levels exceeding patients’ reading skills.

Readability studies of health care literature

Several studies that examined the readability of health care-related materials targeted at the general public were reviewed. These studies indicated that health care-related materials are not as readable as desired. That is, they do not cater to the reading and literacy skills of their target audience. Hendrickson et al examined 27 pediatric oral health pamphlets or brochures.¹⁷ These materials were meant for parents of young children. The indices used for assessment were: (1) readability: using the Flesch–Kincaid Grade Level, Flesch Reading Ease, and Simple Measure of Gobbledygook (SMOG) indices; (2) thoroughness: inclusion of topics important to young children’s oral health; (3) textual framework: frequency of complex phrases and use of pictures, diagrams, and bulleted text within materials; and (4) terminology: frequency of difficult words and dental jargon. These materials were from commercial, government, industry, and private nonprofit sources within the US. Hendrickson et al noticed that the materials they analyzed had varied levels of readability. The materials were not always well suited for their target patient population. They also noticed that the government materials were more readable as compared to those published by commercial and industry entities.¹⁷

There are several studies that evaluated the readability of health care-related material by comparing it with the readability of local newspapers. Nicoll and Harrison compared the readability of health-related pamphlets with English national newspapers.¹⁸ They found that the pamphlets were not as readable as desired for the targeted audience. The authors felt that some of the specialist and commercially produced pamphlets were difficult to read. The authors suggested that readability formulae could be useful in the initial assessment of health care-related literature. In a similar study, Oates and Oates analyzed health care pamphlets intended for parents.¹⁹ They used the Flesch Reading Ease and Flesch–Kincaid Grade Level indices for their analysis. The parameters examined were (1) readability and (2) human interest. Further, Herreid described that in fast-paced novels, where characters’ names abound, the human interest score could be around 20.²⁰ In readable nonfiction magazines, such as Time or Newsweek, it ranges between six and eight. Oates and Oates compared the readability scores of the health care pamphlets with those for daily Sydney newspapers.¹⁹ They observed that (1) pamphlets produced by government agencies were more readable when compared to pamphlets produced by nongovernment agencies, (2) pamphlets produced with input from the parents (target audience) were the most readable of all, (3) pamphlets produced by government agencies had higher human interest scores when compared to pamphlets produced by nongovernment agencies, (4) pamphlets produced with input from the parents had the highest human interest scores of all, and (5) the readability and human interest scores of government-produced pamphlets were similar to those of the daily Sydney newspapers. Oates and Oates put forth that health care literature for parents needs to be easy to understand. They suggested involving parents in the production of the pamphlets to help improve their readability, human interest scores, and thus effectiveness.¹⁹

Murphy et al conducted a study to compare the readability of 99 adult-subject information leaflets for medical research projects with the readability of nine New Zealand newspaper editorials and nine popular magazine articles.²¹ They calculated the Flesch Reading Ease, Flesch–Kincaid Grade Level, and Gunning Fog indices using a computerized grammar check. They found that (1) the subject information leaflets were easier to read as compared to newspaper editorials, but more difficult to read when compared to popular magazine articles, and (2) the level of secondary school education required to read the leaflets averaged at 2 years, which is equivalent to a form four rating. Newspaper editorials had a form four rating, while magazine articles had a form one rating. They concluded that readability scores are useful in assessing the ease with which information can be read. Along the same lines, Wong audited the readability of patient information leaflets intended for marketing proprietary antiepileptic drugs in United Kingdom.²² The study compared twelve leaflets with six antiepileptic drug articles from medical journals and six headline articles from United Kingdom newspapers. The readability was judged through the scores obtained on the Gunning Fog and Flesch Reading Ease indices. Through these readability indices, this study established that the leaflets were suitable for the reading age of the general adult population. The Audit Commission in the United Kingdom recommends that patient information leaflets should be audited by health professionals using a formal readability test.²²

Amini et al assessed the readability of American Academy of Pediatric Dentistry patient education brochures.²³ They compared the readability of these brochures with the readability recommended by health education experts. This assessment used the Gunning Fog and Flesch Reading Ease indices to measure the readability of the association’s brochures. They found that the association’s brochures were above the sixth grade reading level recommended by the health experts. Thus the authors put forth that readability formulae could be used to help improve the reading ease of health education materials.²³ In another study, Grossman et al assessed the readability of consent forms used to describe clinical oncology protocols.²⁴ They analyzed 137 consent forms at the Johns Hopkins Oncology Center. The indices used to measure readability were the Flesch Reading Ease, Flesch–Kincaid Grade Level, and Gunning Fog indices. The authors concluded that from a patient point-of-view, the consent forms were more difficult to read than desired. They put forth that the consent process could be improved by increasing the readability of the consent forms. Mumford examined the readability of 24 health care-related information leaflets, designed by nurses.²⁵ These leaflets were meant for patients. He used the Flesch Reading Ease index, Gunning Fog index, and SMOG readability formula for his assessment. Through the scores obtained, he established that patients could not comprehend the information on the leaflets easily. Mumford mentioned that the results of his study were similar to the results of other related studies he had reviewed.

Clauson et al assessed the readability of dietary supplement leaflets used for diabetes mellitus and chronic fatigue syndrome.²⁶ These leaflets were meant for patients. They used the Flesch–Kincaid Grade Level and health information Readability Analyzer to evaluate readability. They recorded that these leaflets were higher than the desired sixth grade reading level – the level recommended by experts – and thus were more difficult to read. They also put forth that the health information Readability Analyzer was a better tool for measuring readability as compared to Flesh–Kincaid Grade Level.

Collins et al identified that according to Health Information Portability and Accountability Act (HIPAA) regulations, agencies must furnish patients with authorizations that they can read and understand.²⁷ They put forth that half of the American population reads at an eighth grade level. The authors studied HIPAA templates provided by 51 institutional review boards to judge if they catered to this reading level. They used the Flesch Reading Ease index, the Dale–Chall formula, and the Fry Readability Graph method to measure readability. Their results showed that the mean readability of the templates, estimated with the Flesch Reading Ease index and the Fry Readability Graph method, was at high school reading level or above. Similarly, the readability using the Dale–Chall formula was at a ninth grade level. They concluded that the HIPAA authorization forms were difficult to read for the targeted population and patients were being asked to read health care-related material that was higher than their reading level. Osborne et al identified that there was only one study that evaluated the readability of advance directives.²⁸ Advance directives help people convey how they wish to receive end-of-life care.²⁹ This study evaluated ten advance directive documents using the Flesch–Kincaid Grade Level and Gunning Fog indices. The average reading level of these documents was found to be equivalent to the reading level of an adult, someone who has completed 14 grade levels of education. Osborne et al asserted that often people are known to read two to five levels below their highest level of education. Thus the advance directives were not ideal for the general public.²⁸ Breese and Burman evaluated the readability of privacy practices forms from the top 185 health care institutions in the US.³⁰ HIPAA articulates that these forms must be in plain language. SMOG, Flesch–Kincaid Grade Level, and Flesch Reading Ease indices were used to measure readability. They concluded that (1) the privacy notifications were lengthier than desired, (2) the font size used was smaller than desired, and (3) these forms shared characteristics similar to major medical literature journals, which highlighted higher reading level requirement and language complexity.

Using the aforementioned studies as examples, it is plausible to assume that most health care-related literature is less readable than desired. In other words, health care-related literature is difficult for the average patient to read and comprehend. The studies indicate that, on average, the general public finds it difficult to read and understand health care-related brochures and pamphlets. Agencies designing these pamphlets and brochures do not seem to be evaluating their material with respect to the target audience. Furthermore, readability assessment can be performed using well-established indices similar to the one listed earlier.

Sampling of studies in health care

Badarudeen and Sabharwal presented the importance and relevance of health literacy as the single best predicator of an individual’s health.³¹ They put forth that Medicaid, the US health care program meant for people with low income and resources, cost about four times more for those who do not have adequate health literacy. The authors also established that adequate health literacy facilitates improved communication between the health care provider and the patient, which in turn improves health care quality. They suggested raising awareness among health care providers about the importance of designing health care-related material for the general public.

Banasiak and Meadows-Oliver studied the readability of six websites that provided information on asthma.³² They chose websites that had previously been evaluated for quality and accuracy of information. The authors used the Flesch–Kincaid Grade Level to evaluate readability. Through their study they found that the average readability of the websites was for a 9.73 grade level, which is high for an average consumer. They put forth that the desired reading level should not be higher than the eighth grade level. Paasche-Orlow et al conducted a study to test the readability of informed consent forms provided by institutional review boards.³³ They surveyed 114 US medical school websites to evaluate institutional review board readability standards and the informed consent form templates using Flesch–Kincaid Grade Level. The average readability of the templates was at the tenth grade level, two grades higher than the desired level. Paasche-Orlow et al concluded that the institutional review boards’ informed consent forms fall short of their own readability standard, and are not designed based on the local literacy rate. Berland et al evaluated health information on breast cancer, obesity, and childhood asthma available on the internet.³⁴ They focused on English and Spanish language search engines and websites. The authors evaluated the accessibility of 14 websites, quality of 25 websites, and content provided by one search engine. Thirty-four physicians were on the review board and followed a prestructured review list. They evaluated quality, coverage, and accuracy of key clinical elements. The Fry Readability Graph method was used for evaluation. They determined that (1) using search engines to search simple terms was not efficient, (2) the coverage of key information on English and Spanish language websites was poor and inconsistent, (3) the accuracy of key clinical elements was good, and (4) a high reading level was required to comprehend web-based health-related information.

Walsh and Volsko studied the readability of Internet-based health care information about the top five medical related causes of death in the US.³⁵ They studied the websites of the American Heart Association, American Cancer Society, American Lung Association, American Diabetes Association, and American Stroke Association. The authors gathered 100 consumer health information articles and assessed their readability using the SMOG, Gunning Fog, and Flesch–Kincaid Grade Level indices. They determined that Internet-based information was written above the level recommended by the US Department of Health and Human Services. They recommended that health care providers adhere to these standards, which could help improve patient comprehension of health care-related information.

Jukkala et al evaluated the readability of the Clinical Microsystem Assessment Tool.³⁶ The Clinical Microsystem Assessment Tool is used to assess the quality of high-performance health care tools. They used the Flesch Reading Ease index for their evaluation and determined that the tool was very difficult to read. They also conducted an analysis using SMOG and found that 14.71 years of education was required to understand the content. Considering the diversity within the health care staff, they concluded that the Clinical Microsystem Assessment Tool had its limitations and did not necessarily cater to the whole workforce. Pizur-Barnekow et al examined the readability of 94 documents provided by nine agencies that participate in the Birth to Three program.³⁷ The documents used for the study provide information about the nature of the program, financials, intervention planning, and programmatic consent of the families. The Flesch–Kincaid Grade Level, Flesch Reading Ease, and SMOG indices were used for this analysis. The authors determined that a majority of the documents were above the required fifth grade reading level and did not take into account the users’ purpose, interests, and cultural background, which can have an influence on reading ability.

Sampling of readability studies performed for the US CDC

The preceding section presented the work done to evaluate the readability of health care information materials targeted at the general public. In the current section, the focus is on similar readability analyses conducted on health care-related documents published by CDC. Melman et al conducted a study involving 150 parents/caretakers.³⁸ They reported that a majority of the subjects had an inadequate reading level and were not able to comprehend vaccine information pamphlets issued by CDC. Davis et al conducted a study with 522 subjects on the polio vaccine information pamphlets issued by CDC in 1994.³⁹ They reported that the readability level of the pamphlets was higher than desired. It was suggested that the information material be modified to a level equivalent to a third or fourth grade level as opposed to the eighth grade level available then. In a similar study, Davis et al established that it is important to include instructional graphics in addition to simplifying text to make information pamphlets more understandable.⁴⁰

Lagasse et al assessed the readability of the online information CDC had issued on H1N1/09 influenza.⁴¹ They concluded that it was crucial to modify the reading level of information based on the technical know-how of the target population. They also underlined that factors like formatting, font, and layout should be looked into to enhance the readability and accessibility of such material. Sugerman et al conducted a study to assess the readability of material used to communicate emergency health risk information.⁴² They had 1802 persons respond to a survey. They realized that a majority of the respondents found it difficult to recall, comprehend, and comply with the technical messages presented in the survey as opposed to the nontechnical messages. This suggested that using simple language and words could help improve the readability of health care documents.

Through two trials at an emergency department, Merchant et al compared patient comprehension of rapid human immunodeficiency virus pretest information.⁴³ They compared two methods of information delivery: in-person discussion and a tablet computer-based discussion. “Do you know about rapid human immunodeficiency virus testing?” was the topic of discussion for this test. The content for the discussion was based on material for pretest information as suggested by CDC. Participants were asked to complete a questionnaire. The Wilcoxon rank-sum test was used to score the questionnaires, and scores from both types of pretesting were compared. After analyzing the scores, the authors found that the video discussion could substitute an in-person discussion. Therefore, it is plausible to assume that people find it easier to understand and assess visual information.

CDC’s concussion and traumatic brain injury related publications

CDC is a US federal agency. It is an integral component of the US Department of Health and Human Services.⁴⁴ On its website, CDC states that it is dedicated to protecting health and promoting quality of life through the prevention and control of disease, injury, and disability. The CDC website is one of the primary ways in which it communicates with the general public. On average, the website gets about 41 million hits monthly. This number goes up to more than a hundred million annually.⁴⁵ A wide spectrum of users, ranging from health care providers, scientists, researchers, public health professionals, and students may refer to this resource for health-related information. CDC’s efforts are focused on the following five areas:⁴⁶

1. Supporting state and local health departments,

2. Improving global health,

3. Implementing measures to decrease leading causes of death,

4. Strengthening surveillance and epidemiology, and

5. Reforming health policies.

According to the website, users can find health information on a wide variety of topics. These may include but are not limited to the following:

• Data and statistics

• Emergencies and disasters

• Healthy living

• Life stages and populations

• Workplace safety and health

• Diseases and conditions

• Environmental health

• Injury, violence, and safety

• Travelers’ health.

The topics related to injury, safety, and violence could be further divided into the subtopics listed in Table 1. This paper focuses on the subtopics of concussion and traumatic brain injury.

Table 1.

Subtopics covered under the topic of injury, safety, and violence on the Centers for Disease Control and Prevention website

Subtopics of injury, safety, and violence
Burns/fire safety	Poisoning
Falls/drowning	School health/safety/bullying
Hospital visit/field triage	Sexual violence
Injury/safety – multitopic	Sports health/safety
Intimate partner relationship/violence	Stress management
Maltreatment	Suicide
Motor vehicle safety	Concussion/traumatic brain injury
Online violence	Youth violence/risks

Research questions

The readability of concussion and traumatic brain injury literature published by CDC has not been assessed adequately. In order to fill in this gap, this research effort tried to answer the following research questions:

1. What is the mean and standard deviation of the Flesch–Kincaid Grade Level, Flesch Reading Ease, Gunning Fog, Suitability Assessment of Materials (SAM), and SMOG indices for 40 selected concussion and traumatic brain injury publications from the US CDC?

2. What is the median, mode, and range of the Flesch–Kincaid Grade Level, Flesch Reading Ease, Gunning Fog, SAM, and SMOG indices for 40 selected concussion and traumatic brain injury from the US CDC?

Data collection

The health care literature associated with concussion and traumatic brain injury was collected from the CDC website ( http://www.cdc.gov/injuryviolencesafety/ ) on June 10, 2012, starting with the category titled injury safety and violence. This topic consisted of 16 subtopics (Table 1). The study was concentrated on the subtopics of concussion and traumatic brain injury. A total of 40 documents associated with this subtopic were downloaded in Adobe^® Acrobat™ (.pdf; Adobe Systems Inc, San Jose, CA) format. The count of documents by specific subject is shown in Table 2. The titles of the documents assessed are listed in Appendix A. These documents were then converted into plain text files. Sentences that were fragmented due to the conversion process were realigned by removing extra line breaks and/or white space. No other changes were made to the text.

Table 2.

The number of documents in the health care literature associated with concussion and traumatic brain injury

Subject	n
Concussion	22
Traumatic brain injury	18
Total	40

Data analysis

Consistent with studies included in previous sections, and in particular based on the recommendations of the National Institutes of Health at the US Department of Health and Human Services,⁴⁷ the following indices were used to evaluate the 40 documents collected: Flesch–Kincaid Grade Level, Flesch Reading Ease, Gunning Fog, and SMOG.⁴⁶ The indices were calculated using the Readability Studio™ software (version 2012.0.1) (Oleander Software, Vandalia, OH) in accordance with the recommendations from the National Institutes of Health at the US Department of Health and Human Services.⁴⁷ However, SAM was calculated manually.

Flesch–Kincaid Grade Level and Flesch Reading Ease index

Calderon et al maintained that Flesch–Kincaid Grade Level and Flesch Reading Ease formulas are most commonly used to assess readability.⁴⁸ The Flesch–Kincaid Grade Level score rates text on a US school grade level. The Flesch Reading Ease index rates text on a 100-point scale. The higher the score, the easier it is to understand the document. The Flesch–Kincaid Grade Level score is given by (0.39 × ASL) + (11.8 × ASW) − 15.59.⁴⁹ The Flesch Reading Ease index score is given by 206.835 − (1.015 × ASL) − (84.6 × ASW).⁴⁹ In the preceding formulas, ASL denotes the number of words divided by the number of sentences and ASW denotes the number of syllables divided by the number of words. The results of the descriptive analyses associated with Flesch–Kincaid Grade Level and Flesch Reading Ease index ratings of the 40 selected documents are presented in Table 3.

Table 3.

Descriptive analysis associated with the Flesch–Kincaid Grade Level and Flesch Reading Ease indices

	Flesch–Kincaid	Flesch Reading Ease
Mean	11.1	49.5
Median	11.6	48.0
Mode	7.4	35.0
Standard deviation	3.9	17.4
Range	17.4	70.0
Minimum	2.6	20.0
Maximum	20.0	90.0

It was found that the mean Flesch–Kincaid Grade Level for the 40 documents assessed was 11.1, standard deviation was 3.9, median was 11.6, mode was 7.4, and range was 17.4. It was found that the average Flesch Reading Ease index for the 40 documents was 49.5, standard deviation was 17.4, median was 48.0, mode was 35.0, and range was 70.0.

Gunning Fog index

The Gunning Fog index was the first, and now well-known, readability indicator.⁵⁰ The index correlates certain quantifiable aspects of a text with its grade level.⁵⁰ Most people are comfortable reading two grades below their highest attained grade.⁵⁰ The Gunning Fog index grade level is given by:⁵¹

$$0.4\left(\frac{w}{s}+100\times \frac{w_3}{w}\right)$$

In this equation, w is the number of words, w₃ is the number of words with at least three syllables, and s is the number of sentences. The results of the descriptive analysis associated with the Gunning Fog index are presented in Table 4.

Table 4.

Descriptive analysis associated with the Gunning Fog index

Gunning Fog
Mean	11.3
Median	11.0
Mode	9.6
Standard deviation	3.1
Range	12.3
Minimum	5.0
Maximum	17.3

It was found that the mean Gunning Fog index for the 40 documents assessed was 11.3, standard deviation was 3.1, median was 11.0, mode was 9.6, and range was 12.3.

SAM

The SAM index assesses not only the readability, but also the usability and suitability of written materials. In order to evaluate the SAM index, a written material is evaluated based on 22 factors on a scale ranging from zero to three. The overall rating based on the 22 factors is calculated in percent. Materials with an overall rating of 70%–100% are deemed superior. Materials with an overall rating of 40%–69% are deemed adequate. Materials with an overall rating of 0%–39% percent are deemed not suitable. The 22 factors and the associated scoring sheet are copyrighted material. However, additional details associated with calculating the SAM index can be found in the work of Doak et al.¹⁴ The results of the descriptive analysis associated with the SAM index are presented in Table 5.

Table 5.

Descriptive analysis associated with the Suitability of Assessment Materials index

Suitability of Assessment Materials
Mean	72.2%
Median	72.0%
Mode	72.0%
Standard deviation	6.0%
Range	23.0%
Minimum	61.0%
Maximum	84.0%

It was found that the mean SAM index for the 40 documents assessed was 72.2%, standard deviation was 6.0%, median was 72.0%, mode was 72.0%, and range was 23.0%.

SMOG

This index estimates the years of education needed to fully understand a piece of writing.⁵² SMOG is widely used, particularly for checking health messages. In order to calculate SMOG from a document with more than 30 sentences, count the words of three or more syllables (polysyllables) in three ten-sentence samples from the text. SMOG grade level is given by:⁵³

$$1.043\sqrt{p_s\times \left(\frac{30}{s}\right)}+3.1291$$

In cases where there are less than 30 sentences, SMOG grade level is given by:⁵³

$$\sqrt{p+p_a\times (30-s)}+3$$

In these equations, p_s is the number of polysyllables in a sample of 30 sentences, p_a is the average number of polysyllabic words per sentence, p is the total number of polysyllables, and s is the total number of sentences. It must be noted that in order to calculate SMOG, all abbreviations must be unabbreviated and numbers pronounced to determine whether they are polysyllabic. The results of the descriptive analysis associated with the SMOG index are presented in Table 6.

Table 6.

Descriptive analysis associated with the Simplified Measure of Gobbledygook index

Simplified Measure of Gobbledygook
Mean	12.8
Median	13.0
Mode	14.4
Standard deviation	2.5
Range	10.4
Minimum	7.6
Maximum	18.0

It was found that the mean SMOG index for the 40 documents assessed was 12.8, standard deviation was 2.5, median was 13.0, mode was 14.4, and range was 10.4.

Discussion

The mean ratings of the Flesch–Kincaid Grade Level, Gunning Fog, and SMOG indices indicate that the documents assessed were readable by people with an education level higher than the eleventh grade. The mean ratings of the Flesch Reading Ease index showed that the documents assessed were difficult to read. The median and mode of the ratings associated with the indices also confirmed these appraisals. These findings are consistent with the findings of many studies included in the “Readability studies of health care literature” section above. Furthermore, the wide ranges for these indices showed that there were some documents that required an advanced reader while there were others that could be read by rudimentary readers. Wilson, Kasabwala et al, and Wilson state that the average American reads at an eighth or ninth grade level.^54–56 Furthermore, Badarudeen and Sabharwal reported that several health care organizations have recommended the readability of patient education materials be no higher than sixth to eighth grade level.³¹ Hence, it could be concluded that the readability of the documents assessed is not adequate and needs to be improved.

The mean, median, and mode associated with the SAM index show that the documents assessed were superior in terms of usability and suitability. The lowest SAM rating was 61%. Hence, it can be concluded that the documents assessed were at least adequate in terms of usability and suitability. This conclusion presents a paradox: although the documents assessed were found not to be very readable, they were very usable. A plausible explanation for this paradox could be that while the text included in these documents was relatively hard to read, the layout, graphics/illustrations, and format made them very usable. In order to test this hypothesis, future research should try to evaluate and compare readability and usability for other types of documents from various sources. Lastly, the SAM index should be used in conjunction with the various other indices to assess both usability and readability of health care-related documents.

Based on the literature review and analyses of the 40 documents assessed for this study, recommendations were formulated to improve the readability, usability, and suitability of health care-related documents geared towards the general public. These recommendations have been supplemented by guidelines from the National Institutes of Health, Doak et al, Reinhard et al, and McKinney et al.^14,47,57,58 Figure 1 presents these recommendations in the form of an incremental improvement process cycle. The diagram can be used as a step-by-step checklist to design health care information materials targeted at the general public with varied readability levels. The process cycle checklist also provides a methodology to improve the readability of health care-related documents geared towards the general public. Additionally, the recommendations were created into a dos and don’ts list. This list is illustrated in Table 7.

Figure 1.

Recommendations to improve the readability, usability, and suitability of health care-related documents geared towards the general public.

Table 7.

Dos and don’ts list of improving the readability, usability, and suitability of health care-related documents geared towards the general public

Dos	Don’ts
✓ Assess target readers	✘ Technical and profession-specific terms
✓ Think and write like the target readers	✘ Images that are unclear and will not photocopy or print well
✓ Convey central focus of the document to the reader	✘ Complicated illustrations
✓ Adopt methods to effectively provide concise information, eg, summaries and lists	✘ Fancy fonts
✓ Use examples	✘ Cramping information with no white space
✓ Include graphics: diagrams, graphs, and illustrations pertinent to the audience	✘ Capitalize all the information
✓ Use techniques to grab the readers’ attention, eg, images, diagrams, text emphasis, headers
✓ Use techniques to emphasize important information
✓ Provide relevant definitions
✓ Expand on abbreviations
✓ Design a well-articulated cover/title page
✓ Provide methodical explanations
✓ Make the document easy to navigate through
✓ Make the document easy to read by using simple terms, short sentences, legible font, appropriate font size, adequate white space, and margins
✓ Always field test the document

Conclusion

This study has attempted to highlight the importance of evaluating the readability of health care-related literature geared towards the general public. The varying levels of health literacy and a significant portion of the general population at or under the basic literacy level reinforce the need for such studies. The recommendations based on the literature review and data analyses can be used to improve the readability, usability, and suitability of health care-related literature.

Appendix A.

Titles of documents assessed

1. Heads Up: Concussion in Youth Sports – A Fact Sheet for Athletes

2. Heads Up: Concussion in High School Sports – A Fact Sheet for Athletes

3. Heads Up: Concussion in Youth Sports – Signs and Symptoms and Action Plan

4. Heads Up: Concussion in High School Sports – Signs and Symptoms and Action Plan

5. Heads Up: Concussion in Youth Sports – A Fact Sheet for Coaches

6. Concussion in Sports

7. Heads Up to Schools: Know Your Concussion ABCs

8. Heads Up: Preventing Concussion

9. Heads Up: Concussion in Youth Sports – What Should You Do

10. Concussion – Main Message Poster

11. Parent/Athlete Concussion Information Sheet

12. Heads Up: Concussion in Youth Sports – A Fact Sheet for Parents

13. Heads Up: Concussion in High School Sports – A Fact Sheet for Parents

14. Heads Up: Concussion in Youth Sports – Signs and Symptoms

15. A Quiz For Coaches, Athletes, and Parents

16. Concussion – Signs Symptoms Poster

17. Heads Up: Concussion in High School Sports – Wallet Card

18. Check for Safety – A Home Fall Prevention Checklist for Older Adults

19. What You Can Do to Prevent Falls

20. Protect the Ones You Love – Falls

21. What You Can Do To Prevent Falls: Have Your Vision Checked

22. Home and Community – Fall Prevention Strategies

23. Sports, Recreation, and Exercise

24. Get the Stats on Traumatic Brain Injury in the United States

25. Facts about Concussion and Brain Injury

26. Preventing Traumatic Brain Injury

27. Preventing Traumatic Brain Injury in Older Adults – Information for Family Members and Other Caregivers

28. Preventing Traumatic Brain Injury in Older Adults

29. Help Seniors Live Better, Longer: Prevent Brain Injury

30. Help Seniors Live Better, Longer: Prevent Brain Injury – Signs and Symptoms

31. Signs and Symptoms of Traumatic Brain Injury

32. Help Seniors Live Better, Longer: Prevent Brain Injury – Key Facts

33. What is a Concussion?

34. Concussion: A Fact Sheet for Teachers, Counselors, and School Professionals

35. Concussion: A Fact Sheet for Parents

36. Returning to School After a Concussion

37. Concussion Signs and Symptoms Checklist

38. Signs and Symptoms of a Concussion

39. Traumatic Brain Injury

40. Victimization of Persons with Traumatic Brain Injury or Other Disabilities