Abstract
The goal of this study was to assess the readability of online resources pertaining to Alcohol Use Disorder (AUD) as perceived by patients seeking treatment. The National Institutes of Health (NIH) and American Medical Association (AMA) have recommended that medical resources should be written at a sixth-grade reading level. However, prior investigations in various medical fields have revealed that online materials often fail to adhere to these guidelines. An internet search was conducted to simulate the experience of patients seeking information on AUD treatment. The first thirty websites that did not require login credentials were examined using established readability tests. The main outcomes included: Flesch-Kincaid Reading Grade Level, Gunning Fog index, Simple Measure of Gobbledygook (SMOG) Readability Formula, and Coleman-Liau index. Thirty records were identified with the mean readability level was 12.37 (2.54). There were no significant differences in mean readability across readability indices or author type. None of the thirty records met the reading level recommendations as set by the NIH and AMA. In order to enhance accessibility and ultimately improve AUD health outcomes, it is recommended that patient-oriented resources be crafted with adherence to these guidelines. Consequently, future AUD resources ought to prioritize the enhancement of their readability.
INTRODUCTION
Some of the most common queries searched on the world wide web are related to health concerns (Powell & Clarke, 2002). Specifically, individuals with chronic diseases (e.g. diabetes) are utilizing the internet to look for novel therapeutic approaches and social support (Zrebiec & Jacobson, 2001). Health literacy refers to the extent to which individuals possess the skills necessary to access, comprehend, and utilize information in order to make informed decisions regarding their health (Administration, 2022). Extensive research has established health literacy as a crucial determinant of health outcomes, with implications for morbidity and mortality rates. Unfortunately, only about 36% of adults in the US have basic or below basic health literacy proficiencies, hindering their ability to navigate the healthcare system and promote their well-being (White et al., 2007).
Given the vast number of adult Americans who rely on the internet as a primary source of patient-related information, numerous national organizations have recognized the need to enhance the readability of online resources. Notably, both the National Institutes of Health (NIH) (Health, 2018) and the American Medical Association (AMA) (Weiss, 2007) have recommended that patient-oriented medical materials adhere to a reading level no higher than sixth grade. To this end, studies assessing readability from the perspective of patients or caregivers have already been conducted for many medical conditions (e.g. craniosynostosis) (Soliman et al., 2023) or procedures (e.g. radiology) (Bange et al., 2019).
Previous research has examined the adherence to accessibility guidelines for online patient information on substance use disorders, particularly opioid use disorder (OUD) (Dang et al., 2023). However, no studies to date have investigated the readability of online materials specifically addressing alcohol use disorder (AUD) treatments. In the United States, nearly 30 million individuals over 18 years old suffer from AUD (SAMHSA, 2021). Unfortunately, less than 20% of these patients seek treatment or support, and less than 8% have sought treatment in recent years (SAMHSA, 2021). In light of this research gap, our main aim is to assess the readability of online resources pertaining to AUD treatments from the perspective of patients.
By employing historically validated outcome measures for readability, we comprehensively evaluated the accessibility and readability of online materials addressing AUD. This investigation is essential in determining whether existing resources align with nationally recommended reading levels and can effectively facilitate the comprehension and utilization of information by individuals seeking support and guidance for AUD. Through this study, we aimed to contribute to the ongoing efforts for enhancing health literacy and promoting well-informed decision-making among patients.
METHODS
This study employed the use of a Google Boolean search from the perspective of a patient or caregiver searching for relevant health materials. We utilized the keywords “alcohol abuse” and “alcoholism” as search queries rather than “alcohol use disorder” as patients are unlikely to use updated medical terms from the Diagnostic and Statistical Manual of Mental Health (DSM)-IV to DSM-5 (Fama et al., 2023; Haass-Koffler & Kenna, 2013). The search was performed using a single internet protocol (IP) address. The search was done in both incognito and normal mode in order to assess if differences arose due to pre-existing cookies. Exclusion criteria included: (1) non-English content, (2) duplicate entries, (3) sponsored results by Google, and (4) those requiring login information. Eligible texts pertaining to AUD were collected from the first thirty websites in the search results and saved as separate text files for subsequent analysis.
As conducted in other similar research (Soliman et al., 2023), we utilized the following measures that have been historically employed to evaluate the readability of texts by: 1) the United States military technical manuals: the Flesch-Kincaid Reading Grade Level, 2) Public newspapers: Gunning Fog index, 3) Healthcare materials: Simple Measure of Gobbledygook (SMOG) Readability Formula, and 4) United States Office of Education textbooks: Coleman-Liau index. These indices factor various metrics to determine the reading level required to comprehend each entry such as word count, sentence length, character count, number of complex words, and average syllables per word. The equations for calculating each index are provided in Table 1. The resulting values indicate the level of formal education necessary to understand the text, where a score of “1, 2, 3 etc” corresponds to a grade level. The mean readability level was calculated as the average of the four separate indices
Table 1.
Readability Indices with Respective Formulae
| Index | Equation |
|---|---|
| Flesch-Kincaid Reading Grade Level | (0.39×(words/sentences)+11.8×(syllables/words)-15.59 |
| Gunning fog Index | 0.4×[(words/sentences)+100× complex Words/words)] |
| SMOG Index | 1.0430×V(30 × complex words/total sentences)+3.1291 |
| Coleman-Liau Index | 5.89×(characters/words)-0.3×(sentences/words)-15.8 |
Note: the mean readability level was calculated as the average of the four separate indices
The results were categorized by author type, including hospital systems, national health organizations, academic journals, blogs, rehabilitation/detoxification centers, and other health information sources. Statistical analysis was performed using Stata 15 (StataCorp, College Station, Texas, USA), and a One-way analysis of variance (ANOVA) was conducted to compare the readability across different author types. Results were reported as Mean (M) ± standard deviation (SD) and statistical significance was accepted for a * p-value < .05. GraphPad Prism (v.7) was used to generate figures (La Jolla, CA, USA) Finally, although most standard readability tools cannot interpret the readability enhancements provided by visuals, the presence of visual aids was documented for each recorded by percentage (%).
RESULTS
Thirty results which met inclusion criteria were collected and assessed. Of the first thirty entries, four (13.33%) were published by hospital systems, with the remaining records published as followed: three by national health organizations (10%), two by academic journals (6.66%), two by blogs (6.66%), three by rehabilitation/detoxification centers (10%) and 16 by other health information sources (53.33%).
Mean readability levels calculated as the average of the four separate indices are reported in Table 2. The level across all records was 12.37 ± 2.54, with records published by blog sites demonstrating the lowest average level (10.8 ± 3.36), and rehabilitation/detoxification centers displaying the highest average level (13.73 ± 1.66). There was a large range of reading levels across all readability tests for any source type (6.8 – 21.3).
Table 2.
Readability Levels and Article Characteristics for Overall Cohort and Respective Author Type. Reported as Mean (SD)
| Overall | Hospital System | Nat. Health Org. | Acad. Journal | Blog | Rehab/Detox center | Other | ANOVA | |
|---|---|---|---|---|---|---|---|---|
|
| ||||||||
| N | 30 | 4 | 3 | 2 | 2 | 3 | 16 | |
|
| ||||||||
| Flesch-Kincaid Reading Grade Level | 11.2 (2.3) | 10.4 (1.4) | 12.33 (4.9) | 10.55 (2.33) | 10.55 (2.33) | 12.77 (0.87) | 11.21 (1.97) | F(5)=0.796; p>.05 |
| Gunning fog | 14.12 (2.47) | 13.825 (1.66) | 15.20 (5.31) | 12.75 (2.61) | 12.25 (4.88) | 15.8 (0.98) | 14.075 (1.96) | F(5)=0.715; p>.05 |
| SMOG | 10.68 (186) | 10.28 (1.2) | 11.63 (4.0) | 10.15 (1.63) | 9.1 (3.25) | 12.07 (0.81) | 10.61 (1.52) | F(5)=0.824; p>.05 |
| Coleman Liau | 13.48 (1.66) | 13.45 (1.1) | 13.57 (2.5) | 12.15 (3.61) | 12.8 (3.1) | 14.3 (0.53) | 13.58 (1.5) | F(5)=0.437; p>.05 |
| Mean Readabilit y Level | 12.37 (2.54) | 11.98 (1.91) | 13.18 (1.56) | 11.4 (2.3) | 10.8 (3.36) | 13.73 (1.66) | 12.37 (2.27) | F(5)=1.873; p>.05 |
| Words | 1655 (1638) | 1193 (644) | 424 (312) | 1719 (1558) | 530 (120) | 1518 (610) | 2160 (2020) | F(5)=0.879; p>.05 |
| Use of Visual Aids | 11 (36.7) | 1 (25%) | 2 (66.7%) | 0 (0%) | 0 (0%) | 1 (33.3%) | 7 (43.8%) | NA |
The average readability levels by source type, relative to the recommended national guidelines, is depicted in Figure 1. Differences in mean readability level were not statistically different by source type (p = .61). The readability systems with the highest and lowest mean scores were the Coleman-Liau Index (13.48 ± 1.66) and SMOG Formula (10.68 ± 1.86), respectively. The average number of words per record was 1,655.93 ± 1,638.99 with national health organizations providing the shortest text at 424.67 ± 312.26 words per record. There were no statistical differences amongst source type for the mean number of words (p = .51). Finally, although most standard readability tools cannot interpret the readability enhancements provided by visuals, the presence of visual aids was also documented for each record. Eleven entries (36.67%) of all sources utilized visual aids.
Figure 1.
Readability levels by source type relative to the recommended national guidelines
DISCUSSION
To our knowledge, this study represents the first attempt to comprehensively assess the readability of online resources pertaining to AUD from the perspective of patients and possible caregivers. Our investigation revealed a noteworthy disparity between the reading levels of these materials and the recommended guidelines established by NIH and AMA, which both advocate for a sixth-grade reading level for medical resources.
Upon analyzing the collected data, we observed that the average reading level of the evaluated patient resources was 12.37, indicating a comprehension level expected of a recent high school graduate. Remarkably, none of the resources met the prescribed guidelines for readability. These findings highlight a concerning lack of accessibility in commonly available materials concerning AUD, rendering them challenging for the majority of patients to comprehend and leverage to inform potential treatments.
The implications of these findings are profound, considering the crucial role of health literacy in patient outcomes. Accessible and easily understandable information is essential for patients and caregivers to make informed decisions about the management and treatment of AUD. By adhering to the recommended reading level guidelines established by reputable medical organizations, resources can be optimized to empower patients with the knowledge necessary to navigate the complexities of AUD. A secondary data analysis of the National Cancer Institute’s Health Information National Trends Survey (HINTS) has also shown that, compared to men, women are more likely to search for health care resources on the web (Atkinson et al., 2009). This observation further highlights the importance of providing resources at guideline readability levels, as women often seek and receive less AUD treatment than men (McCrady et al., 2020) and are less represented in clinical trials for AUD (Haass-Koffler et al., 2020).
A potential limitation of our methodology could be the inclusion of reading materials that were not explicitly intended for patients, such as research articles. While the inclusion of non-patient-specific materials may introduce a certain degree of variability, it also reflects the reality of the information landscape encountered by patients and caregivers seeking resources online. By encompassing a broader range of materials, including those not explicitly aimed at patients, we provide a more accurate assessment of comprehension challenges faced in the real world. This approach offers valuable insights into the accessibility of online information related to AUD and underscores the urgent need for improvements in readability across various types of resources.
The present study reveals that the readability of online materials pertaining to AUD surpasses the thresholds recommended by prominent national society guidelines. In order to enhance accessibility and ultimately improve health outcomes, it is recommended that patient-oriented resources be crafted with adherence to these guidelines. Consequently, future resources addressing AUD ought to prioritize the enhancement of their readability. In line with this, it is important to notice that the National Institute on Alcohol Abuse and Alcoholism (NIAAA) at NIH has also developed a Navigator Tool to help patients to find AUD treatments.
HIGLIGHTS.
Health literacy refers to the extent to which individuals possess the skills necessary to access, comprehend, and utilize information and services to make informed decisions regarding their health
The National Institutes of Health (NIH) and American Medical Association (AMA) have recommended that medical resources should be written at a sixth-grade reading level.
None of the thirty records pertaining AUD retrieved by this study met the reading level recommendations as set by the NIH and AMA.
Future resources addressing AUD ought to prioritize the enhancement of their readability to improve AUD health outcomes.
FUNDING
The author(s) received no financial support for the research, authorship, and/or publication of this article. Dr. Haass-Koffler’ laboratory is supported by the National Institute on Alcohol Abuse and Alcoholism (NIAAA, R01 AA026589; R01 AA027760; R21 AA027614) and by the National Institute of General Medical Sciences (NIGMS), Center of Biomedical Research Excellence (COBRE, P20 GM130414).
Footnotes
DECLARATION OF CONFLICT OF INTERESTS
CLH-K received mifepristone and matching placebo for another trial and travel support to CA to present the data to the Corcept Therapeutic Conference (September 2022). The other authors declare no competing interests.
ETHICAL STATEMENT
This study did not involve patients or human subjects. Therefore, this study was deemed Institutional Review Board exempt and patient consent was not applicable.
The manuscript has not been previously published nor is under consideration for publication elsewhere. All authors have contributed substantially both to the research process, evaluation of the cited literature and have revised the manuscript with a critical eye. All authors have approved the final version of the manuscript.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
REFERENCES
- Administration HRS (2022). Health Literacy. https://www.hrsa.gov/about/organization/bureaus/ohe/health-literacy#:~:text=Personal%20health%20literacy%20is%20the,actions%20for%20themselves%20and%20others. [Google Scholar]
- Atkinson N, Saperstein S, & Pleis J (2009). Using the internet for health-related activities: findings from a national probability sample. Journal of Medical Internet Research, 11(1), e1035. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bange M, Huh E, Novin SA, Hui FK, & Yi PH (2019). Readability of patient education materials from RadiologyInfo. org: has there been progress over the past 5 years? American Journal of Roentgenology, 213(4), 875–879. [DOI] [PubMed] [Google Scholar]
- Dang N, Khunte M, Zhong A, & Chatterjee A (2023). Naloxone Online Information Exceeds the Recommended Reading Level for Patient Education Materials. J Stud Alcohol Drugs. 10.15288/jsad.22-00343 [DOI] [PubMed] [Google Scholar]
- Fama R, Sassoon SA, Fama LD, & Sullivan EV (2023). Alcohol and other substance use. Neuropsychological Interviewing of Adults, 203. [Google Scholar]
- Haass-Koffler CL, & Kenna GA (2013). Bacchus by Caravaggio as the Visual Diagnosis of Alcohol Use Disorder from the Fifth Edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5). Front Psychiatry, 4, 86. 10.3389/fpsyt.2013.00086 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haass-Koffler CL, Piacentino D, Li X, Long VM, Lee MR, Swift RM, Kenna GA, & Leggio L (2020). Differences in Sociodemographic and Alcohol-Related Clinical Characteristics Between Treatment Seekers and Nontreatment Seekers and Their Role in Predicting Outcomes in the COMBINE Study for Alcohol Use Disorder. Alcohol Clin Exp Res, 44(10), 2097–2108. 10.1111/acer.14428 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Health N. I. o. (2018). Clear communication: clear & simple. Maryland: US Department of Health & Human Services. [Google Scholar]
- McCrady BS, Epstein EE, & Fokas KF (2020). Treatment Interventions for Women With Alcohol Use Disorder. Alcohol Res, 40(2), 08. 10.35946/arcr.v40.2.08 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Powell J, & Clarke A (2002). The WWW of the World Wide Web: who, what, and why? Journal of Medical Internet Research, 4(1), e4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- SAMHSA C. f. B. H. S. a. Q. (2021). National Survey on Drug Use and Health. https://www.samhsa.gov/data/sites/default/files/reports/rpt39441/NSDUHDetailedTabs2021/NSDUHDetailedTabs2021/NSDUHDetTabsSect5pe2021.htm#tab5.6a [Google Scholar]
- Soliman L, Soliman P, Gallo Marin B, Sobti N, & Woo AS (2023). Craniosynostosis: Are Online Resources Readable? Cleft Palate Craniofac J, 10556656231154843. 10.1177/10556656231154843 [DOI] [PubMed] [Google Scholar]
- Weiss BD (2007). Health literacy and patient safety: Help patients understand. Manual for clinicians. American Medical Association Foundation. [Google Scholar]
- White S, Bennett I, Cordell T, & Baxter SL (2007). Health Literacy of America’s Adults: Results from the 2003 National Assessment of Adult Literacy. [Google Scholar]
- Zrebiec J, & Jacobson A (2001). What attracts patients with diabetes to an internet support group? A 21-month longitudinal website study. Diabetic medicine, 18(2), 154–158. [DOI] [PubMed] [Google Scholar]