5-Step Methodology for Evaluation and Adaptation of Print Patient Health Information to Meet the <5^th Grade Readability Criterion

Abstract

Background

In the setting of declining U.S. literacy, new policies include use of clear communication and low literacy accessibility practices with all patients. Reliable methods for adapting health information to meet such criteria remain a pressing need.

Objectives

To report method validation (Study 1) and method replication (Study 2) procedures and outcomes for a 5-step method for evaluating and adapting print health information to meet the current low literacy criterion of <5^th grade readability.

Materials

Sets of 18 and 11 publicly-disseminated patient education documents developed by a university-affiliated medical center.

Measures

Three low-literacy criteria were strategically targeted for efficient, systematic evaluation and text modification to meet a <5^th grade reading level: sentence length <15 words, writing in active voice, and use of common words with multisyllabic words (>2–3 syllables) minimized or avoided. Inter-rater reliability for the document evaluations was determined.

Results

Training in the methodology resulted in inter-rater reliability of 0.99–1.00 in Study 1 and 0.98–1.00 in Study 2. Original documents met none of the targeted low literacy criteria. In Study 1, following low-literacy adaptation, mean reading grade level decreased from 10.4±1.8 to 3.8±0.6 (p<0.0001), with consistent achievement of criteria for words per sentence, passive voice, and syllables per word. Study 2 demonstrated similar achievement of all target criteria, with a resulting decrease in mean reading grade level from 11.0±1.8 to 4.6±0.3 (p < 0.0001).

Conclusions

The 5-step methodology proved teachable and efficient. Targeting a limited set of modifiable criteria was effective and reliable in achieving <5^th grade readability.

Low health literacy, a persisting public health challenge in the U.S., is linked to poorer health status, increased morbidity and mortality, and higher healthcare costs.^1,2 The most recent National Assessment of Adult Literacy (NAAL) revealed that only 12% of American adults demonstrate proficient literacy with written materials.^3,4 Although lower literacy among racial and ethnic minorities has been found to contribute to health disparities,^5,6 the NAAL revealed a population decline in literacy over the past ten years, with greatest decline among White adults and persons with a high school diploma, GED, or some vocational education.^3,4

These findings suggest that lower literacy impacts the U.S. adult population broadly, warranting population-level intervention. A universal precautions approach has been recommended;⁷ recent federal legislative action mandates use of clear, plain language in written documents for the public, across government sectors.⁸

The current, low-literacy criterion for readability of health information is <5^th grade,^9–11 yet most patient education materials and websites remain at 12^th grade or college level readability.^12,13 Resources are available to assist with awareness of health literacy recommendations and materials adaptation (See Appendix A). However, an efficient, systematic approach that consistently and reliably results in <5^th grade readability, especially while preserving essential medical terms, remains a need for healthcare organizations and professionals.^14,15

Appendix A.

Agency Health Literacy Resources

Agency for Health Care Quality and Research (AHRQ) Health Literacy Universal Precautions Toolkit http://www.ahrq.gov/qual/literacy/healthliteracytoolkit.pdf

Centers for Disease Control and Prevention Guidance and Standards (Health Literacy Resource List) http://www.cdc.gov/healthliteracy/DevelopMaterials/GuidanceStandards.html

Centers for Disease Control and Prevention Simply Put: A Guide for Creating Easy-to-Understand Materials http://www.cdc.gov/healthliteracy/pdf/Simply_Put.pdf

Centers for Medicare & Medicaid Services (CMS) Toolkit for Making Written Material Clear and Effective https://www.cms.gov/WrittenMaterialsToolkit/

National Institutes of Health Clear Communication: An NIH Health Literacy Initiative http://www.nih.gov/clearcommunication/healthliteracy.htm

U.S. Department of Health and Human Services Health Resources and Services Administration (HRSA) Health Literacy http://www.hrsa.gov/publichealth/healthliteracy/index.html

This paper describes a 5-step approach for essential evaluation and efficient adaption of print patient education text to meet <5^th grade readability. We report: a) a method validation study (Study 1) to test effectiveness of the 5-step approach in achieving targeted low-literacy criteria in a set of 18 publicly-disseminated patient education documents, and b) a method replication study (Study 2) to test reliability of the personnel training and consistency of the 5-step approach in achieving targeted literacy criteria in an additional set of 11 patient education documents.

METHODS

Development of the Literacy Evaluation and Adaptation Methodology

Low-Literacy Criteria

Criteria for low-literacy patient education materials were derived from the Centers for Disease Control and Prevention (CDC),⁹ National Institutes of Health/National Cancer Institute (NIH/NCI),¹⁰ and Doak and colleagues.¹¹ Of 32 consensus criteria available on literacy demand, formatting, and behavioral activation characteristics¹⁴ (See Appendix B), the following literacy-demand criteria were selected for efficiency and effectiveness in adapting text to meet a <5^th grade reading level: 1) sentence length <15 words; 2) writing in active voice (<5% passive voice); and 3) use of common words, with multisyllabic words (>2–3 syllables) minimized or avoided. These criteria were deemed modifiable, and reading grade level, sentence length and passive voice can be readily evaluated using available electronic resources. (See Hill-Briggs and Smith¹⁴ for descriptions of modifiable low-literacy criteria, operational definitions, and procedures.)

Appendix B.

Consensus Criteria for Evaluation of Literacy-Demand (n = 23) and Behavioral Activation (n = 9) Characteristics of Print Patient Education Materials

Criteria	Specifications	Sources (refs.)
Literacy Demand
Word usage, reading level, and sentence length	Scientific jargon avoided; technical, concept, category, and value judgment terms introduced with understandable explanation or example	9–11
	Vocabulary uses common words; multisyllabic words (>2–3 syllables) avoided	9–11
	Sentence length <15 words	9, 11
	Writing in active (vs. passive) voice	9, 10
	Reading grade level <5th grade*	9, 10
Typography	Text in uppercase and lowercase serif (best) or sans-serif	9–11
	Type size ≥12 points (including text, tables, and captions)	9–11
	Typographic cues (bolding, bullets, and size) emphasize key points	9–11
	Subheaders used; complex topics subdivided into smaller parts of ≤5 main points, ≤5 items per list†	9, 11
	Line length ≤30–50 characters and spaces	9, 10
	Using all capital letters for long headers or running text avoided	9, 10
Graphics, illustrations, and tables	Cover graphic shows purpose of brochure, attracts attention, and is friendly	9–11
	Graphics designed to be simple, age appropriate, and familiar to readers	9–11
	Explanatory captions included with each graphic	9–11
	Illustrations on page adjacent to related text	9,11
	Illustrations present key messages so that reader can grasp key idea from illustration alone	9,11
	Illustrations not distracting	9, 10
Layout, space, and paper	Layout and organization enable predicable sequence/flow of information	9–11
	Visual cuing devices (e.g. shading, boxes, arrows) used to direct attention to specific points or key content	9–11
	Adequate white space and line spacing used to reduce appearance of clutter	9–11
	Contrast between type and paper is high	9–11
	Color use supports and does not distract from message. Readers need not learn color codes to understand and use message	9, 10
	Topics preceded by advanced organizers or headers>50% of the time	9, 10
Behavioral activation
Content, scope, and organization	Scope limited to information directly related to purpose‡	9–11
	Content ≥50% behaviorally focused	9, 10
	Summary, information overview, or information review included	9, 10
Engagement, interaction, and action facilitation	Recommended behaviors are modeled and specific (e.g. step by step)	9–11
	Questions or activities with records presented for reader response	9, 10
	“How to” information provided	9–11
Audience relevance and appropriateness	Recommendations sensible in the context of the audience’s culture, values, and beliefs	9–11
	Language and experience(s) used match those of the intended audience	9–11
	Cultural images and examples presented in a positive manner	9–11

Reprinted and adapted with permission from Hill-Briggs F, Smith AS. Evaluation of diabetes and cardiovascular disease print patient education materials for use with low-health literate populations. Diabetes Care. 2008 Apr;31(4):667–71.¹⁴ Operational definitions and techniques for implementing each criterion are available in the original article’s Online Appendix.

^*5th-grade reading level or less is the superior criterion according to Doak et al.¹¹ and the recommended criterion for low literacy according tothe National Cancer Institute.¹⁰

^†No more than three to four main ideas and five to six items per list according to Centers for Disease Control ⁹ criteria; less than seven main ideas and three to five items per list for low literacy according to Doak et al. criteria.¹¹

^‡A less stringent, adequate criterion according to Doak et al. is that <40% of information be nonessential to purpose.

5-Step Methodology

The 5-steps in the approach are shown in Table 1. Importantly, the approach begins with comprehensive evaluation to determine adaptation needs at the sentence level of analysis. For the two studies reported herein, readability statistics available in Microsoft Word, 2003/2007 were used to evaluate and adapt the documents. The Flesch-Kincaid Grade Level test rates text on a U.S. school grade level, with a score of 5.0 indicating that an average fifth grader can understand the document.¹⁶

Table 1.

5-Step Methodology

Step	Description
Step 1. Readability Evaluation of Original Document at Whole Document, Paragraph, and Sentence Levels	Determination of key low-literacy readability statistics at the document, paragraph, and sentence levels: Reading grade level % passive voice Number of words per sentence Number of characters per word
Step 2. Medical Terminology/Scientific Jargon of Original Document	Identification of concentration of essential medical terms and scientific jargon for defining of terms, lower literacy word or phrase substitutions, or alternate formatting (e.g. tables).
Step 3. Literacy Adaptation Process at the Sentence Level	Identification of sentences with ≥ 5th grade readability and elements of those sentences not meeting low literacy criteria.
	Modification of sentences not meeting criteria: Change passive voice to active voice sentence construction. Reduce number of words per sentence by using simple sentence construction and removing clauses and phrases expressing additional messages and making them new sentences for clarity. Substitute multisyllabic words for common, familiar words and phrases. For essential medical and technical terms, provide a brief explanation prior to the term, introducing the technical term as parenthetical to the explanation. Use alternate formatting (e.g. bullets, tables) to shorten sentences, provide visual organization and grouping of information, and increase white space for document readability.
Step 4. Readability Evaluation of Adapted Document	Determination of document, paragraph, and sentence level readability statistics, as described in Step 1 to confirm all criteria have been met.
	Confirmation of number of syllables per word at the document level using Doak et al11 method of 3 text excerpts.
Step 5. Comparison of Pre- and Post-Adaptation Readability	Direct comparison of document level readability statistics of the original and adapted documents on the low literacy readability statistics

Process of Adapting Documents for Low Literacy

Following document evaluation, low-literacy adaptation was approached strategically, targeting those sentences that were at or above a 5^th grade reading level, and the specific elements of the evaluated sentence that were not within the low-literacy targets. Specific adaptation methods included use of common, familiar words; use of resources (e.g. thesaurus) for word and phrasing alternatives; use of a simple, direct, and straightforward manner for sentence wording; expressing one main thought at a time; refraining from adding multiple phrases, clauses, or modifiers within a sentence; use of formatting changes such as bullets or tables when appropriate; avoidance of scientific jargon or medical terminology that was above the target readability; and consistency in use of the chosen words or phrases used to substitute for frequently occurring medical language. When medical terminology that exceeded the readability level was necessary, those technical, concept, category, or value judgment terms were introduced with an understandable explanation or example, prior to introduction of the scientific terminology. The scientific term, rather than the explanation, was placed in parentheses. This allows the reader to encounter the understandable explanation in the course of the text, with the scientific term treated as parenthetical to the explanation. Following low-literacy adaptation, the adapted documents underwent evaluation of readability criteria, and the readability statistics of the original and literacy-adapted documents were compared.

Study 1: Methodology Validation

The 5-step methodology was tested using a set of 18 publicly disseminated patient education documents covering a variety of diabetes and lifestyle topics, developed by a university-affiliated medical center. Three research assistants (high school, bachelor’s and master’s level education, respectively, with no prior experience in low literacy evaluation, writing, or document adaptation) were hired and trained as literacy adaptation specialists for the study. Assistants received an initial 2-hour in-person training in the 5-step methodology prior to working with the documents. They then received a 1-hour supplemental training, during which instruction and materials were provided to address specific challenges in the treatment of medical and legal terminology, and supplemental material in documents (e.g. resource lists). A combined literacy adaptation manual was developed for training new personnel, with the following content: Performance of the 5-Step Method, Additional Exclude/Include Criteria for Readability Evaluations, Treatment of Medical Terminology, Legal Content, Tip Sheet, and Demonstration of Method(s) and Practice. Each assistant received supervision to ensure quality of the adaptation and consistency of content with the original document. Each document was evaluated by two assistants for determination of inter-rater reliability.

Study 2: Methodology Replication

Study 2 was conducted using a set of 11 publicly disseminated patient education documents from the same original source. Two different literacy evaluators/adapters (bachelor’s level education, with prior editorial experience, but without formal training in low literacy evaluation and adaptation) participated in this study. They completed the comprehensive 2-hour total training, using the manual created in Study 1.

Data Analysis

Inter-rater reliability for document-level literacy statistics was assessed using Pearson product-moment correlation coefficients. Inter-rater reliability was determined for original and adapted documents. Document-level literacy statistics of the original and adapted documents were compared using paired-samples t-tests to determine statistical significance of mean improvements in literacy variables. Analyses were conducted using PASW Statistics 18, Release Version 18.0.0 (© SPSS, Inc., 2008, Chicago, IL).

RESULTS

Study 1

Following the basic training, inter-rater reliability for document-level readability evaluation ranged from 0.81 – 0.99 for the set of original documents. After adding the 1-hour supplemental training, inter-rater reliability of the literacy evaluations increased to 0.99 – 1.00 for each pair of evaluators, and for each original document. Inter-rater reliability remained at 0.99 – 1.00 for adapted documents.

Readability statistics for the documents are shown in Table 2. Mean reading grade level of the original documents was 10.4 ± 1.8, with no documents meeting low-literacy criteria investigated. Following adaptation, mean reading grade level of the documents was 3.8 ± 0.6, with each document meeting the < 5^th grade criterion. All adapted documents met targets for passive voice and number of words per sentence. Table 3 illustrates an original document excerpt and the literacy-adapted version of that excerpt. Rephrasing of text, shortening of sentences to simple sentence constructions, and word substitutions were the most commonly used text adaptations in the set of 18 documents, while the most commonly used format changes were bulleting and tables, particularly for content requiring medical terminology (i.e. medication lists and descriptors).

Table 2.

Comparison of Readability Statistics of Original and Literacy-Adapted Documents in Method Validation Study (Study 1)

Patient Education Article	Reading Grade Levela,b		Number of Words Per Sentencec		Passive Voice (%)d		Characters Per Worde
	Original	Adapted	Original	Adapted	Original	Adapted	Original	Adapted
	Microvascular Disease Due to Diabetes	15.3	4.3	24.0	11.0	15.0	4.0	5.3	4.1
Managing Type 2 Diabetes Through Diet	12.3	4.6	20.5	10.4	0	2.0	5.1	4.3
Diabetes Mellitus, Type 2	12.0	4.4	21.9	10.1	12.0	4.0	5.5	4.2
The Skinny on Visceral Fat	11.9	4.0	22.2	10.2	3.0	3.0	4.8	4.1
Diabetes and Your Lifestyle	11.7	4.0	18.6	10.6	12.0	3.0	5.0	4.1
Infections and Diabetes	11.4	4.6	17.8	10.2	37.0	0	5.1	4.3
Losing Weight When You Have Diabetes	10.6	4.0	16.1	10.5	0	0	5.1	4.2
Hot Weather Exercise	10.5	3.1	17.5	9.6	8.0	0	4.9	4.0
Pass this Test	10.0	4.1	18.1	11.7	5.0	4.0	4.7	4.1
Women and Diabetes	9.8	3.7	17.0	10.8	1.0	1.0	4.0	4.1
Get Off the Blood Glucose Roller Coaster	9.8	3.6	18.3	11.4	0	0	4.8	4.1
Mixing Alcohol With Your Diabetes	9.4	3.6	16.3	10.9	5.0	0	4.9	4.2
Exercise in Disguise	9.3	4.6	14.4	10.8	0	0	4.8	4.1
Foot Care for People with Diabetes	9.2	3.7	14.7	10.5	6.0	0	5.0	4.3
Diabetes and Your Heart	9.1	2.8	18.4	9.5	4.0	3.0	4.8	4.4
No More Carb Confusion	8.6	3.2	17.0	10.2	4.0	1.0	4.6	4.1
Diabetes and Stress	8.4	2.7	20.2	10.1	2.0	1.0	4.5	4.0
Nine Ways to Avoid Diabetes Complications	7.9	3.9	13.4	9.6	2.0	0	4.8	4.4
Mean ± SD	10.4 ± 1.8	3.8 ± 0.6	18.1 ± 2.8	10.4 ± 0.6	6.4 ± 8.9	1.4 ± 1.6	4.9 ± 0.3	4.2 ± 0.1
T-test		18.04		11.53		2.40		9.50
P-value		<0.001		<0.001		0.028		<0.001

^aThe Flesch-Kincaid Grade Level score, calculated in Microsoft Word 2003/2007 using the following formula: (0.39 × ASL) + (11.8 × ASW) − 15.59, where ASL is the average sentence length (the number of words divided by the number of sentences), and ASW is the average number of syllables per word (the number of syllables divided by the number of words).

^bLow literacy criterion is <5.0.

^cLow literacy criterion is <15 words per sentence.

^dLow literacy criterion is <5% passive voice at the document level.

^eThere is no criterion for number of characters per word. See Table 4 for comparison of this statistic with number of syllables per word.

Table 3.

Sample Original and Literacy-Adapted Document Excerpts with Low-Literacy Readability Evaluation

Original Document Excerpt	Literacy-Adapted Document Excerpt
Retinopathy	Diabetic Retinopathy
Diabetic retinopathy, the most common diabetic eye complication and the leading cause of blindness among U.S. adults, damages the retina, the light- sensitive nerve tissue at the back of the eye that transmits visual images to the brain. This damage is caused by changes in the tiny blood vessels that supply the retina.	This is the most common eye problem from diabetes. It is the leading cause of blindness among U.S. adults. It is due to damage to the retina; this is the nerve tissue at the back of the eye. This tissue is sensitive to light. It sends visual images to the brain. Tiny blood vessels send blood to this tissue. Diabetes can lead to damage to those blood vessels.
Reading grade level = 14.7	Reading grade level = 3.8
Sentences meeting <5th grade readability = 0%	Sentences meeting <5th grade readability = 100%
Passive voice sentences = 50%	Passive voice sentences = 0%
Number of words per sentence = 26.0	Number of words per sentence = 9.7
Number of characters per word = 5.0	Number of characters per word = 4.1

Syllables per word was compared with the number of characters per word readability statistic (Table 4). Both statistics improved with document literacy adaptation. The use of common, familiar words in the adaptation process resulted in a decrease in mean percentage of words with >3 syllables from 8.3% ± 4.9 in the original documents to 2.5% ± 2.2 in the adapted documents.

Table 4.

Average Characters per Word and Average Syllables per Word in Paragraph Excepts^a

Patient Education Article	Characters Per Worda		Syllables Per Wordb		Words >3 Syllables (%)c
	Original	Adapted	Original	Adapted	Original	Adapted
Microvascular Disease Due to Diabetes	5.5	4.1	1.9	1.4	9.0	0
Managing Type 2 Diabetes Through Diet	5.6	4.5	1.9	1.5	8.0	4.0
Diabetes Mellitus, Type 2	5.4	4.7	1.8	1.4	16.0	6.0
The Skinny on Visceral Fat	5.1	4.7	1.8	1.5	6.0	0
Diabetes and Your Lifestyle	5.3	4.5	1.7	1.4	10.0	6.0
Infections and Diabetes	5.2	4.7	1.8	1.4	10.0	3.0
Losing Weight When You Have Diabetes	5.2	4.5	1.8	1.5	13.0	4.0
Hot Weather Exercise	5.1	4.1	1.9	1.4	12.0	4.0
Pass this Test	4.7	3.7	1.7	1.3	5.0	0
Women and Diabetes	4.8	4.1	1.6	1.3	5.0	2.0
Get Off the Blood Glucose Roller Coaster	4.8	3.7	1.5	1.3	3.0	0
Mixing Alcohol With Your Diabetes	5.8	4.4	1.9	1.3	21.0	0
Exercise in Disguise	5.3	4.2	1.8	1.3	7.0	1.0
Foot Care for People with Diabetes	5.3	4.4	1.8	1.4	12.0	3.0
Diabetes and Your Heart	5.1	4.4	1.6	1.3	10.0	3.0
No More Carb Confusion	4.5	3.9	1.4	1.2	1.0	0
Diabetes and Stress	4.1	3.8	1.3	1.3	4.0	4.0
Nine Ways to Avoid Diabetes Complications	4.8	4.6	1.6	1.5	7.0	5.0
Mean ± SD	5.1 ± 0.4	4.3 ± 0.3	1.7 ± 0.2	1.3 ± 0.1	8.3 ± 4.9	2.5 ± 2.2
T-test		9.99		9.58		5.71
P-value		<0.001		<0.001		<0.001

^aCharacters per word was calculated automatically in Microsoft Word 2003/2007.

^bNumber of syllables per word was determined using the Doak et al.12 method of systematic extraction of three samples of text (from the beginning, middle, and end), from which mean number of syllables per word was calculated.

^cLow literacy criterion is multisyllabic words (>2 – 3 syllables) are avoided.

Study 2

Readability statistics are shown in Table 5. After 2-hour comprehensive training, inter-rater reliability for literacy evaluation of original documents was 0.99 – 1.00, and for adapted documents, 0.98 – 1.00. Mean reading grade level of the original documents was 11.0 ± 1.8., and no original documents met target low-literacy criteria. Study 2 yielded equally successful adaptations as Study 1 (p-values <0.0001).

Table 5.

Comparison of Readability Statistics of Original and Literacy-Adapted Documents in Method Replication Study (Study 2)

Patient Education Article	Reading Grade Levela		Number of Words Per Sentenceb		Passive Voice (%)c		Characters Per Word
	Original	Adapted	Original	Adapted	Original	Adapted	Original	Adapted
Macrovascular Disease Due to Diabetes	13.9	4.1	23.8	9.5	21.0	4.0	5.2	4.3
Diabetes Mellitus: Type 1	13.5	4.2	22.1	9.5	16.0	2.0	5.2	4.2
Keeping Kidneys Healthy	12.2	4.8	20.6	10.3	2.0	3.0	5.0	4.4
Heart-Smart Medications	11.4	4.9	18.9	8.4	20.0	3.0	5.1	4.6
Preserving Kidney Function When You Have Diabetes	11.3	4.7	19.2	10.1	14.0	5.0	4.9	4.4
Young at Heart	11.0	4.5	16.3	9.0	6.0	5.0	5.2	4.5
Exercising Like Your Life Depends On It	10.9	4.5	17.4	8.8	5.0	2.0	4.9	4.3
Make the DASH to Lower Your Blood Pressure	10.8	4.9	17.8	9.2	5.0	3.0	5.0	4.4
Wake-Up Call (Sleep Apnea)	9.4	4.8	15.8	9.9	5.0	1.0	4.9	4.5
Men and Diabetes	9.0	4.8	18.2	11.0	6.0	3.0	4.5	4.2
Stop Smoking!	8.0	4.3	16.7	10.4	0	0	4.6	4.3
Mean ± SD	11.0 ± 1.8	4.6 ± 0.3	18.8 ± 2.5	9.6 ± 0.8	9.1 ± 7.3	2.8 ± 0.2	5.0 ± 0.2	4.4 ± 0.1
T-test		11.17		11.64		3.18		8.66
P-value		<0.001		<0.001		0.01		<0.001

^aLow literacy criterion is <5.0.

^bLow literacy criterion is <15 words per sentence.

^cLow literacy criterion is <5% passive voice at the document level.

CONCLUSIONS

The 5-step methodology, which relies on commonly available electronic readability tools, demonstrated effectiveness as an evaluation and adaptation approach to meet the <5^th grade readability criterion for public health information. Effectiveness of the approach was demonstrated in both the method validation and replication studies. Personnel with no prior experience were able to be trained to perform the method successfully.

Multiple techniques, used in combination, were needed to meet targeted readability statistics. Mere word substitution was not effective as an overall technique for document adaptation; it was prone to inaccurate or awkward sentence constructions, and it often did not reduce sentence-level readability sufficiently. Rephrasing was necessary to preserve meaning and to introduce essential medical terms, while still reducing literacy demand. All instances of passive voice required rephrasing. Changing passive voice phrasings (e.g. Eating foods that have fewer calories is used to help with losing weight) to declarative statements (e.g. Eat foods with fewer calories to help lose weight) not only reduced reading grade level but made the messages more actionable for the reader. Further, we found that meeting the <5^th grade criterion was realistic, and reading levels as low as 2^nd and 3^rd grade were achievable without using presentation styles or language that might be perceived as more appropriate for children than adults.

Our method utilized the Flesch-Kincaid formula to assess reading grade level. It is important to note additional available readability formulas that are widely used, including the FOG Index, the Simple Measure of Gobbledygook (SMOG), and the Fry readability graph. Although these formulas are each calculated differently, they have been found to correlate highly with each other, and there has been no evidence to support increased accuracy of one over another in determining reading grade level.^17,18 All four formulas can be calculated manually; the Flesch-Kincaid formula has an accessible electronic calculation, which provides the benefit of time efficiency and practicality in routine use and with large numbers of documents. A limitation of using the Microsoft Word automated readability statistics is that the syllables per word statistic is not provided; characters per word is calculated. This study evaluated number of syllables per word using the Doak¹¹ method, and a comparison table, with number of characters per word, is provided. During the adaptation process, multisyllabic words were readily identified without a formal calculation. The process of using common, more familiar words automatically resulted in fewer syllables at the sentence and document levels of analysis.

With regard to literacy adaptation content evaluation, the articles did not undergo a process of pre-/post- evaluation for preservation of meaning as part of research. However, after completing literacy adaptation, the adapted articles were reviewed alongside the original articles, by the institutional business office responsible for materials development and copyright. The business office approved the literacy-adapted articles as consistent in content and meaning. The articles then underwent review by representatives of a national healthcare organization which, after reviewing the articles, requested use of the full sets of literacy-adapted (but not the original) articles for dissemination as routine patient education materials to their members/clients. Although data were not collected during these reviews, both professional entities determined the literacy-adapted materials to be of high quality and value, consistent with or exceeding that of their non-literacy-adapted counterparts.

Testing acceptability and effectiveness of the materials in educating the public is ultimately needed to evaluate utility of the 5-step method. The method has been used to adapt patient education materials in previously reported clinical outcome studies, with findings of high satisfaction and ease of understanding ratings by adults with literacy as low as ≤3^rd grade as well as those with average literacy,¹⁹ and by persons with mild cognitive and/or visual impairment,²⁰ another subgroup for whom clear communication is particularly necessary. Intervention studies using this literacy-adaptation method have resulted in increased patient knowledge^19,21 and improved behaviors and disease control²¹ in lower-literate samples.

There are directions deserving of further research and development. First, the 2-hour training, which was efficient and reliable, warrants testing in alternative instruction modes (e.g. web-based, electronic media such as CD-Rom). Second, in developing and testing the 5-step method, consistent “rules” for reducing literacy demand emerged for several elements of language (e.g. treatment of numbers, contractions, sentence structure for active voice) that may be amenable to software programming. Third, there is a need to evaluate the efficacy of the 5-step process in other languages. The overall framework for a systematic approach to low literacy adaptataion is transferrable, as are some elements of formatting. However, we do not expect that the specific literacy demand criteria will hold across languages and cultures. Effective low literacy text modification is based upon understanding of the language construction, parts of speech, characters utilized (length, letter pairings, etc.), and meaning/messaging, all of which determine how information is processed, stored, and learned. The neuropsychology literature reveals significant differences in the structure, processing and comprehension of Spanish language, for example, such that the English language low-literacy criteria may not be directly transferrable. Low-literacy adaptation in Spanish, and other languages, does require examination of those features of the respective language itself.

Finally, the problem of declining literacy in the U.S. over the past decade is one of great importance that likely requires both educational and political reform to reverse. The methodology described herein allows health information to be accessible to the growing proportion of the population with basic to below basic literacy; this is essential for improving patient self-management and health outcomes.^22–25 However, meeting this current demand should not obfuscate the concurrent need for actions directed toward stopping and reversing the low literacy trends in the U.S.