A health literacy analysis of online patient-directed educational materials about mycobacterium avium complex

Highlights

• •Mycobacterium avium complex (MAC) is a complex lung infection.
• •Patient education materials help patients make informed medical decisions.
• •Health literacy is a key social determinant of health outcomes.

Abstract

Introduction

Mycobacterium avium complex (MAC) is a complex lung infection requiring multi-disciplinary approach and management. Due to limited clinician-patient interactions, clinicians may refer patients to online resources to learn about the diagnosis, prognosis, and treatment of MAC. The American Medical Association (AMA) recommends educational materials be written at a sixth-grade reading level and the National Institutes of Health (NIH) recommends that patient education materials be written at an eighth-grade reading level; however, several evaluations found these materials inaccessible due to high literacy levels. To date, there has never been a health literacy assessment of MAC patient education materials. The study aims to assess the health literacy of online patient education materials about MAC.

Methods

The patient education materials were evaluated for readability, actionability, understandability and clarity. Readability was assessed through the Flesch-Kincaid Grade Level Scale (FkGL), SMOG Index, Coleman Liau Index (CLI), Gunning Fog Index (GFI), and Automated Readability Index (ARI). Actionability and understandability was evaluated using the Patient Education Materials Assessment Tool (PEMAT). The Centers for Disease Control (CDC) Clear Communication Index (CCI) was used to assess clarity.

Results

Ten patient education resources were evaluated: CDC, Cleveland Clinic, Mayo Clinic, JAMA, American Thoracic Society (ATS), National Jewish Health, UpToDate, CHEST, WebMD, and Medline. The mean readability scores were as follows: FkGL (9.76), SMOG index (9.82), CLI (13.54), GFI (11.66), ARI (9.15). Four patient education materials were written at a sixth-grade reading level and eight patient education materials were written at an eighth-grade reading level. The majority of the materials received a passing score for understandability but failed to achieve a passing score for actionability. Cleveland Clinic, JAMA, and ATS all received a passing clarity score, indicating that they are easy to read. No patient education materials were available on UpToDate.

Conclusion

Most patient education materials scored poorly for actionability and clarity while scoring highly for readability and understandability. This study should serve as a guide for clinicians interested in offering online education materials to their patients. Increasing access to readable MAC educational materials should be a priority for those working at the intersection of public health, clinical care, and communications.

1. Introduction

Nontuberculous mycobacteria (NTM) are opportunistic pathogens that are susceptible to immunocompromised patients and individuals with underlying lung diseases [1], [2]. The prevalence of NTM infections has been increasing globally, and it continues to be a problem, particularly in developing countries [3], [4]. Mycobacterium avium complex (MAC) is the most common NTM species [5]. Infections with MAC can be challenging to diagnose. The American Thoracic Society (ATS) and Infectious Diseases Society of America (IDSA) have published recommendations for MAC diagnosis and treatment, suggesting triangulation of symptoms, radiology, microbiology, and drug-susceptibility testing [6], [7]. MAC is difficult to treat because of antibiotic resistance, negative side effects of prolonged multidrug treatment, and disease recurrence [8], [9].

Both clinicians and patients have reported that the amount of time scheduled for appointments is often insufficient. Diseases such as MAC, which are complex and nuanced, can be difficult to fully explain in the span of the allocated thirty minutes. In a study conducted at New York clinics, more than 50 % of physicians reported feeling under time pressure during patient visits, which may have an impact on how much quality time doctors can devote to each patient [10]. As there is less time for dedicated clinician-patient interactions, several clinicians across the spectrum of medicine refer patients to publicly available websites as a reference for learning on their own time. The American Medical Association (AMA) recommends educational materials be written at a sixth-grade reading level and National Institutes of Health (NIH) recommends health related patient educational materials should be prepared at a eighth grade or lower reading level, however; several evaluations found these materials to be written at high reading levels [11], [12], [13]. Considerable work has been done on health literacy evaluations of several infections, such as COVID-19 [14], HIV [15], [16], and Influenza [17], [18]. Patients have become increasingly reliant on the Internet to learn more about their health conditions. However, the context of these sources is often not evaluated for health literacy which can impair patients' ability to make health-related decisions [19]. An estimated 74% of adults in the United States use the Internet, with up to 80% of them looking for health related information [20]. Through online patient education materials, the Internet significantly contributes to patients' knowledge and comprehension [21]. As evidenced by the rapidly evolving information flow that has occurred since the beginning of COVID-19 pandemic, the provision of information that is inaccessible to all reading levels can promote lack of comprehension.

Given the high likelihood of reinfection after treatment, one of the greatest needs is for patients to have a better understanding of prevention strategies and management [22]. Therefore, patients diagnosed with this long-term prognosis disease must fully understand the symptoms, treatment, and risks to care for their disease. Effective communication and patient-led conversations with providers will help patients avoid any uncertainty about their prognosis and health outcomes. The overall objective of the study is to evaluate the health literacy of online patient education materials about MAC disease. The objective will be assessed through three goals. The first goal is to assess the availability of online patient education materials about MAC on academic, nonacademic, medical education websites commonly used by patients and physicians. The second goal is to use validated health literacy tools to measure (1) Readability (2) Understandability and Accountability and (3) Clarity of publicly available MAC educational materials. The third goal is to suggest how health literacy can be implemented to improve online patient education materials.

2. Materials and Methods

2.1. Identification and selection of educational materials

Two reviewers (OA and SA) performed an independent search using the following keywords “mycobacterium avium complex patient education” following by the patient education websites for example “mycobacterium avium complex patient education WebMD.” Both reviewers visited the patient education resources to screen, evaluate, and score the materials. Materials were included in the study if they contained information geared toward patients, were pertinent to mycobacterium avium complex disease, able to be printed or read online, and were among the top five search results. Materials were excluded if they were patient forums, blogs, case studies, clinical trials, peer-reviewed journal studies, reviews of medications, and unrelated to mycobacterium avium complex. The material's scores, titles, types, and URLs were recorded in a Microsoft Excel spreadsheet (Appendix A). Research on click-through rate reports found that more than 70% of clicks on search engines like Google are typically on the first 10 results [23]. In a Forbes article about search results, it observed that 67.7% of clicks went to the first 5 results and the first page receives 92% of clicks [24]. In light of these statistics, we focused on the search results from the first page specifically the top five results from each of the patient education resources.

2.2. Health literacy tools

2.2.1. Readability

The Flesh-Kincaid GradeLevel (FKGL) formula is a commonly used and validated tool that evaluates the reading grade level of patient education materials [25], [26]. The FKGL score was determined using word length, average word syllable count per word and sentence length, and average number of words per sentence; (0.39 × number of easy words) + (11.8 × average number of words per sentence) − 15.59 [27], [28]. The FKGL score can also be determined by performing a readability analysis in Microsoft Office Word. The Simple Measure of Gobbledygook (SMOG) Index evaluates the number of polysyllabic words and total number of sentences. The SMOG formula is 1.043 (√ total number of polysyllabic words × (30 ÷ total number of sentences) + 3.1291 [28]. The Coleman Liau Index (CLI) uses the characters per word and average number words per sentence to assess readability. The CLI formula is [(0.0588 × average number of letters per 100 words) − (0.296 × average number of sentences per 100 words)] − 15.8 [28]. The Gunning Fog Index (GFI) measures the difficulty of a material by the average sentence length and number of complex words (words with more than 3 syllables). The GFI formula is 0.4 × [(average number of words per sentence ÷ average number of sentences) + 100 (number of characters ÷ average number of words per sentence)] [28]. The Automated Readability Index (ARI) provides a score through average number of characters per word and average number of words per sentence. The ARI formula is 4.71 × (number of characters ÷ average number of words per sentence) + 0.5 (average number of words per sentence ÷ average number of sentences) – 21.43 [28]. Five readability calculators; Readable, Microsoft, Readability Formula, WebFX and Online Utility, were utilized to determine the readability scores. Each of the scores correspond with a reading grade level (e.g., a score of 9 corresponds to a 9th grade reading level). The scores range from a 5th grade reading level to a 14th grade reading level.

2.2.2. Understandability and actionability

The Patient Education Materials Assessment Tool (PEMAT), specifically PEMAT for printable materials (PEMAT-P) and PEMAT for audiovisual materials (PEMAT-A/V), was used to evaluate the materials' understandability and actionability [29]. PEMAT-P was used to evaluate printable materials such as webpages, brochures, and PDFs. This tool consists of 17 items that assess understandability and 7 items that assess actionability. PEMAT-A/V was employed to evaluate audiovisual materials such as pictures, audio recordings, and videos. There are 13 items that assess understandability and 4 items that assess actionability. Patient education materials are considered understandable when they can be analyzed and explained by people from different socioeconomic backgrounds and health literacy levels. Individuals of different socioeconomic backgrounds and health literacy levels must be able to identify what actions they should take based on the information provided for patient education materials to be considered actionable [30]. The materials were scored using the PEMAT auto-scoring form with the guidance of the PEMAT manual. The PEMAT auto-scoring form assigns a score of 0 for disagree, 1 for agree, or not applicable to each item. A score of less than 70% indicates poor understandability or actionability, whereas a score of 70% or greater is perceived as understandable or actionable.

2.2.3. Clarity

The CDC Clear Communication Index (CCI) is a set of research-based guidelines for developing and evaluating public communication materials [31]. This tool measured the clarity and usefulness of the patient education materials. There are two types of CCI tools: the Full Index and Modified Index. These indexes begin with four open ended questions about the audience, health literacy skills, the communication objective and the main message [32]. They evaluate seven components of the materials: main message and call to action, language, the information design, state of science, behavioral recommendations, numbers, and risk [31]. The Full Index contains 20 research-based items that are used to score printable materials such as PDFs and webpages. The Modified Index consists of 13 research-based components and is designed to evaluate short-form media (i.e., videos and infographic). The first four introductory questions are open-ended and ungraded. The remaining items were given numerical scores of 0 for no and 1 for yes. The Index User's Guide states that separate scores are combined to provide an overall score out of 100; a score of 90 or greater is regarded as passing or easy to read. The score from this evidence-based tool can be used to improve the clarity of health-related materials.

2.3. Quality Assessment instruments

The patient education material quality was evaluated using the Journal of the American Medical Association (JAMA) benchmark criteria and modified DISCERN. The JAMA benchmark criteria, a 4-point scale, assesses the reliability and accuracy of videos and resources through criteria such as authorship, attribution, disclosure, and currency. Each criterion contributes to a comprehensive evaluation. The modified DISCERN questionnaire, featuring five yes/no questions, serves as a scoring scale to assess the quality of health-related information, providing a structured approach for evaluation.

2.4. Ethical considerations

2.4.1. IRB approval

This was an evaluation of publicly available educational information and is non-human subjects’ research.

3. Results

A total of 19 patient educational materials from 9 academic, nonacademic, and medical education websites were screened and evaluated. The number of materials available on these patient education resources ranged from one to five, with five resources offering only one publicly available patient education material. Fifteen of the patient materials were webpages, four patient materials were PDFs and three of the webpages contained videos. The following patient education resources were selected for evaluation: CDC, Cleveland Clinic, Mayo Clinic, JAMA, American Thoracic Society (ATS), National Jewish Health, UpToDate, CHEST, WebMD, and Medline. The health literacy tools score are shown in Table 1 and Fig. 1, Fig. 2, Fig. 3. The quality and realability scores are given in Table 2.

Table 1.

Average Readability Scores.

Patient Education Material	Flesch Kincaid	SMOG Index	Coleman Liau Index	Gunning Fog Index	Automated Readability Index
1	13.38	12.31	16.57	14.52	11.75
2	12.44	11.06	15.41	14.66	10.92
3	11.64	12.42	14.8	15.29	9.87
4	10.38	10.44	13.42	10.86	10.62
5	10.04	10.53	12.84	12.16	10.78
6	10.23	10.52	13.27	12.13	11.24
7	13.58	11.9	20.7	14.49	12.84
8	7.1	7.52	10.42	7.91	6.62
9	7.87	8.1	13.58	8.65	7.21
10	6.15	7.3	10.27	7.78	5.95
11	7.67	7.92	11.4	11.4	5.79
12	6.74	7.75	11.39	11.39	6.7
13	8.68	8.55	12.55	9.15	8.17
14	8.7	8.17	13.64	10.28	8.67
15	10	9.36	13.8	11.59	9.57
16	12.6	12.01	16.84	14.32	11.71
17	10.4	10.36	13.34	11.86	9.91
18	9.7	10.84	11.88	11.88	9.35
19	8.2	9.52	11.23	11.23	6.19

Fig. 1.

Mean Readability Grade Level Scores. Five formulas—the Flesch Kincaid, Coleman Liau Index, Automated Readability Index, SMOG Index, and Gunning Fog Index—were used to determine the average readability grade level. The solid line represents the NIH Recommendation for materials to be written at an 8th grade reading level. The dotted line represents the AMA recommendation for materials to be written at an 6th grade reading level. Readability The average reading grade level (mean ± standard deviations) of the five readability scales were as follows: Flesch Kincaid Grade Level 9.76 ± 2.23 [range: 6.15–13.58]; recommended score ≤ 8.0, SMOG index 9.82 ± 1.72 [7.3–12.42]; recommended score ≤ 8.0, Coleman Liau Index 13.54 ± 2.53 [range 10.27–20.7]; recommended score ≤ 8.0, Gunning Fog Index 11.66 ± 2.27 [range 7.78–15.29]; recommended score ≤ 8.0, Automated Readability Index 9.15 ± 2.21 [range 5.79–12.84]; recommended score: ≤ 8.0.

Fig. 2.

Patient Education Materials Assessment Tool (PEMAT) Scores. Understandability and Actionability scores of patient education materials. According to the PEMAT tool, a score of 70% or higher is considered to be both understandable and actionable. This is shown by the solid line. Understandability and Accountability The patient education materials used were from CDC (1–3), Medline (4–6), Mayo Clinic (7), WebMD (8–12), Cleveland Clinic (13–15), JAMA (16), ATS (17), National Jewish Health (18) and CHEST (19). A total of 5 education materials (8, 9,12,14,19) met the acceptable threshold (>70%) for both understandability and actionability. The patient education resources that met that threshold were WebMD, Cleveland Clinic and CHEST. The majority: 16 out 19 or 84% of education material met the acceptable threshold for understandability.

Fig. 3.

CDC Clear Communication Index (CCI) Scores. Clarity scores of the patient education materials. According to the CCI scale, scores above 90% is considered passing or “easy to read”. This is represented with the solid line. Clarity The Index-Full was applied to 18 materials and the Index-Mod was applied to 4 materials. The patient education materials used were from CDC (1–3), Medline (4–6), Mayo Clinic (7), WebMD (8–12), Cleveland Clinic (13–15), JAMA (16), ATS (17), National Jewish Health (18) and CHEST (19). Only 3 patient education resources specifically Index-Full material scored above 90 or easy to read. None of the Index-Mod sources were considered easy to read, the closest one was an online patient education material from Medline.

Table 2.

JAMA and Modified DISCERN Score Distribution.

Patient Education Materials	JAMA Score	Modified DISCERN Score
1	2	4
2	2	5
3	2	3
4	2	3
5	2	3
6	3	4
7	2	2
8	4	3
9	4	4
10	3	5
11	3	4
12	3	4
13	3	4
14	3	3
15	2	4
16	4	4
17	4	5
18	1	3
19	3	5
Mean ± SD	2.73 ± 0.87	3.78 ± 0.85

3.1. Readability

The American Medical Association (AMA) recommends that patient education materials be written at a sixth-grade reading level, and four materials—WebMD and CHEST—met this recommendation. 8 out of 9 or 88% of patient education resources contained materials written at an eight-grade reading level which met the National Institutes of Health (NIH) recommendation. Only one educational resources, WebMD, met both the AMA and NIH recommended reading level on the FkGL scale. The educational materials assessed varied in readability scores across different indices: FkGL ranged from 6th to 13th grade, SMOG index ranged from 7th to 12th grade, CLI ranged from 10th grade to 20th grade, GFI ranged from 7th grade to 15th grade, and ARI ranged from 5th to 12th grade.

3.2. Understandability and Accountability

A total of 5 education materials (8,9,12,14,19) met the acceptable threshold (>70%) for both understandability and actionability. The patient education resources that met that threshold were WebMD (8,9,12), Cleveland Clinic (14) and CHEST (19). The majority: 16 out of 19 or 84% of education material met the acceptable threshold for understandability.

3.3. Clarity

The Index-Full was used for eighteen materials and the Index-Mod was utilized for four materials. Only three patient education materials, specifically Index-Full sources scored 90 or above and were considered easy to read. None of the Index-Mod sources received a passing score, the closest one was an online patient education material from Medline with a score of 87.5.

3.4. Quality Assessment

The evaluation scores included a mean JAMA score of 2.73 ± 0.87 and a mean modified DISCERN score of 3.78 ± 0.85. Higher scores indicated high reliability for both JAMA and modified DISCERN assessment tools. For JAMA, 42% of the patient materials scored a two, 37% of patient materials scored a 3, and 21% of patient education materials scored a 4. For modified DISCERN, 5% of patient materials scored a 2, 32% of patient materials scored a 3, 42% of patient materials scored a 4, and 21% of patient materials scored a 5.

4. Discussion

MAC education materials scored poorly for actionability and clarity while scoring highly for readability and understandability. Many patient-facing materials were written at a high grade level. Our research identifies a major gap in the treatment of MAC that has never been explored.

Since Simonds first proposed the idea of health literacy [33], it has evolved from the simple definition of being able to carry out reading and computation-intensive tasks related to one's health to the ability to access, analyze, and comprehend basic health information that one may need to make appropriate health decisions [34], [35]. Patients are more likely to actively participate in medical decision-making if they have high health literacy [36]. Studies have found limited health literacy is associated with poor management of health conditions, higher rate of hospitalizations and low socioeconomic status [37], [38]. Paasche-Orlow and Wolf proposed a conceptual framework for health literacy based on their research on the impact of limited health literacy in health outcomes. Their research found casual pathways connecting health literacy to health outcomes through three factors: access and utilization of health care, patient-provider relationship, and self-care [39]. Understanding these causal pathways will help mitigate low health literacy and reduce health disparities. Furthermore, the National Assessment of Adult Literacy Survey found considerable disparities in health literacy across age, race/ethnicity categories and sex, with 36% of adults in the United States having basic or below-basic health literacy [40].

Based on our findings, the following strategies should be implemented in future patient education materials to ensure these materials are readable, understandable, actionable, and clear for individuals of all health literacy levels. Future patient education materials should be prepared at the AMA recommended reading level with consideration for populations with low health literacy. The authors and publishers should work collaboratively to assess the health literacy levels of the patient communities and develop materials in the appropriate grade level. Additionally, the health-related materials should consider presenting the information with visual cues or infographics. Websites containing visual cues, such as videos, had greater levels of understandability and were simpler to read than materials that contained text-only. According to Schubbe's research, the use of images in health communication has been shown to increase patient understanding in communities with low health literacy [41]. Furthermore, more than one validated health literacy tools should be employed to evaluate the health literacy of the patient education materials. By using a combination of health literacy tools, it can increase accessibility and improve understanding.

This study has limitations, such as the lack of certainty that these patient education materials were used by patients or in clinical settings. Additionally, there are disadvantages to employing online reading tools such as not all of the information is accurately translated into the scoring box. The items may receive a higher grade than they should because of misspelled or shortened terms. The online tool does not consider how the terms are used in context. Any use of visual aids, such as tables, pictures, or diagrams, might affect how the content is graded. Moreover, the website’s design may not be user-friendly, making it challenging for patients to access the resources. Another important limitation is the analyzed resources were written in English language only and did not target patients with non-English speaking backgrounds. Despite these limitations, our research reveals gaps that exist in the patient education materials about MAC and identifies opportunities to reduce these health literacy inequalities.

Health literacy is a key social determinant of health outcomes. Patient education materials increases patient comprehension and adherence to medical instructions, which can impact health outcomes. It is our responsibility as clinicians and members of patients’ care team to make information regarding a patient’s diseases accessible to readers of all health literacy levels. This study should serve as a guide for clinicians interested in offering online education materials to their patients. Increasing access to readable MAC educational materials should be a priority for those working at the intersection of public health, clinical care, and communications. To close the gap between medical information given and its implementation, we need clinicians to improve the way they communicate information to patients to include health literacy considerations.

Funding

AGW: K08HS026008-01A.

CRediT authorship contribution statement

Olabimpe Asupoto: Data curation, Formal analysis, Investigation, Visualization, Writing – original draft, Writing – review & editing. Shamsuddin Anwar: Formal analysis, Investigation, Supervision, Validation, Writing – review & editing. Alysse G. Wurcel: Formal analysis, Investigation, Supervision, Validation, Writing – original draft, Writing – review & editing.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following are the Supplementary data to this article: