Re: ‘Readability of online patient education material for the novel coronavirus disease (COVID-19): a cross-sectional health literacy study’

The COVID-19 virus pandemic is currently one of the worst global health emergencies to date.¹ We wish to congratulate Szmuda et al.² on the publication of their cross-sectional health literacy study addressing COVID-19 online information in your journal. The authors investigated the readability of 61 Websites using the search terms ‘Coronavirus,’ ‘COVID-2019,’ ‘SARS CoV-2,’ ‘2019-nCoV,’ and ‘What is the coronavirus’, concluding that online information around COVID-19 was too difficult to read and understand for the average individual. All articles were set at or above a readability level of a high school sophomore grade student (15-16 years old) compared with the 5th to 6th-grade or below reading level (11- to 12-year olds) recommended by the American Medical Association and United States Department of Health and Human Services for patient information.³

It was to our interest if any other work had been published to date evaluating online information for COVID-19 due to the high level of worldwide interest and concern. On reviewing the published literature on PubMed using the search terms ‘readability’, ‘reading’, ‘quality’, ‘online’, ‘coronavirus’ and ‘COVID-19’, we discovered four other articles with similar methodologies, reviewing either the quality or readability of COVID-19 information online.4, 5, 6, 7 The compiled results of all the studies are represented in Table 1 . The findings of these other studies reflect the conclusions of Szmuda et al., with the average United States reading level of Websites across readability scores ranging from 8th to 11th grade. The range of search terms varied across studies but had the term ‘Coronavirus’ shared throughout. Cuan-Baltazar et al.⁷ and Jayasinghe et al.⁵ both assessed the quality of Websites using the DISCERN instrument (max = 80).⁸ The scores were 28.91 and 49.5, respectively, equating to ‘Poor’ and ‘Fair’ in terms of information quality.⁹ Cuan-Baltazar et al.⁷ also assessed the Journal of American Medical Association benchmark¹⁰ scoring (max = 4) displaying poor adherence with the mean being 1.28 ± 1.34.

Table 1.

Summary of the quality and readability results.

Articles	Number of websites	Search date	Search terms	JAMA	DISCERN	FRES	FKG	GFI	SMOG	CLI
T. Szmuda et al., 20202	150	13th March, 2020	‘Coronavirus’, ‘COVID-2019,’ ‘SARS CoV-2,’ ‘2019-nCoV,’ ‘What is the coronavirus’	NA	NA	47.82 ± 12.76	11.51 ± 3.06	13.57 ± 3.10	10.17 ± 2.16	12.65 ± 1.82
L. Treanor et al., 20204	57	24th March, 2020	‘Coronavirus information’	NA	NA	51.5 ± 9.7	10.6 ± 2.2	12.6 ± 2.3	9.6 ± 1.6	10.8 ± 1.6
C. H. Basch et al., 20206	100	NA	‘Coronavirus’	NA	NA	46.4 ± 11.1	10.0 ± 1.9	10.5 ± 2.8	11.9 ± 1.8	11.9 ± 1.8
J. Y. Cuan-Baltazar et al., 20207	110	6th February, 2020	‘Coronavirus’, ‘Wuhan’	1.28 ± 1.34	28.91 ± 10.34	NA	NA	NA	NA	NA
R. Jayasinghe et al., 20205	84	1St week of May, 2020	‘Novel coronavirus’, ‘SARS CoV-2’ ‘severe acute respiratory syndrome coronavirus-2’, ‘COVID-19’, ‘Coronavirus’	NA	49.5	54.1	NA	NA	NA	NA

Overall results for each scoring system, presented as mean ± standard variation if it was published. JAMA, The Journal of the American Medical Association benchmark criteria; DISCERN, DISCERN instrument; FRE, Flesch Reading Ease; FKG, Flesch-Kincaid grade level; GFI, Gunning Fog Index; SMOG, Simple Measure of Gobbledygook; CLI, Coleman Liau Index.

While these online information quality and readability studies can give valuable insight into the resources patients are using to research disease conditions, there are a number of limitations. Although the current most popular search engines (Google, Bing, and Yahoo) were used to evaluate Websites amongst the articles, the returned pages may vary depending on factors including geographical location and the popularity of the websites at a given point in time. As the Internet is constantly being updated with new information, especially in an evolving global pandemic, the search would likely yield different results at a later point in time. The readability tools used are capable of text analysis only and do not provide any assessment of graphics that may be housed on the pages and videos.

Overall, from reviewing the current studies pertaining to COVID-19, the information is set at a readability level far exceeding that which is recommended for patient information. On limited quality testing, the online information for COVID is at best ‘Fair’ on DISCERN instrument scoring. Over the past five months, five articles have been published investigating the quality and/or readability of online COVID-19 information, showing the heightened interest in the topic. We recommend fellow medical professionals to continue research into public information and awareness about COVID-19 online and from other sources. Patient information should be set an appropriate reading level that is of a high-quality standard, listing authors, date of update, sponsorship, and references.