Readability assessment of patient-facing online educational content for pyoderma gangrenosum

To the Editor:

Pyoderma gangrenosum (PG) is a rare skin condition characterized by a progressive and painful ulceration that is associated with underlying inflammatory or neoplastic disorders.¹ Due to the broad accessibility of the internet, patients often look online for information about their health.² Here, we used an unbiased approach to quantitatively assess the readability of online articles pertaining to the education and management of PG.

To test out hypothesis, the query “pyoderma gangrenosum patient information” was inputted into the Google Search engine, which yielded 512,000 results. Two independent reviewers screened the top 50 results for relevancy to patient education. After discarding 2 websites for not meeting the inclusion criteria, the pertinent text was inputted into the readabilityformulas.com calculator (https://readabilityformulas.com/free-readability-formula-tests.php). As a control, this approach was applied for the analysis of Sweet syndrome, which is another inflammatory skin disease. All the sources and raw data are available on request.

The readability of the 48 PG articles was assessed using 7 different tools; 6 of which measured the reading grade level (Flesch-Kincaid, Coleman-Liau, Linsear Write Formula, Automated Readability Index, Gunning Fog Index, and Simple Measure of Gobbledygook Index). The readability was also determined using the Flesch Reading Ease score, which assigned a numerical value (0–100) based on the complexity. A higher Flesch Reading Ease score indicated an easier level of readability for an article.

Each grade level analytic tools demonstrated an average grade level greater than 12, indicating inter-rater reliability among the formulas (Fig 1, A). We then compared the readability of articles written by Doctor of Medicine (MD) authors affiliated with major dermatology academic centers with those written by non-MD authors using the Flesch Reading Ease Score (Fig 1, B). Our findings demonstrated that compared with the online PG articles written by non-MD authors (n = 12, mean = 41.38) those written by MD authors (n = 36, mean = 25.49) required the audience to have a higher level of education (P = .003, 2-tailed t test). Interestingly, a similar result was observed for Sweet syndrome (MD, n = 6, mean = 20.81; non-MD, n = 14, mean = 43.62), validating our findings and indicating that this may be a widespread occurrence (Supplemental Fig 1 available via Mendeley at https://doi.org/10.17632/7y9rzrvj6b). There was no significant difference between providers in dermatology (n = 25) versus those in other fields for PG (n = 23) (P = .31, 2-tailed student t test).

Fig 1.

Readability in online articles regarding pyoderma gangrenosum. A, Readability of 48 online sources was assessed using the Flesch-Kincaid (mean = 13.5), Coleman-Liau (mean = 13.67), Linsear Write Formula (mean = 12.56), Automated Readability Index (mean = 12.25), Gunning Fog (mean = 16.38), and Simple Measure of Gobbledygook Index (mean = 12.02). Two outliers were identified and removed per robust regression and outlier removal criteria. Whiskers demonstrate maximum and minimum values for each category. B, MD authors have a lower Flesch Reading Ease score (n = 36, mean = 25.49) compared with non-MD authors (n = 12, mean = 41.38) (P = .003, 2-tailed t test). No statistical significance was observed between providers in dermatology (n = 25, mean = 27.15) versus non-dermatology providers (n = 23, mean = 31.98) (P = .31, 2-tailed t test). Error bars represent the standard error mean.

These results suggest that regardless of the scale used to calculate the readability index, online content focused on PG required readers to have a minimum of a high school graduate level of education (Fig 1, A). This is 6 grade levels above the target of sixth grade as recommended by the American Medical Association.³ Moreover, the MD authors produced articles with a higher difficulty of reading compared with non-MD authors (Fig 1, B). While these findings are robust, some of the limitations include small sample size and articles written solely for English speaking audiences.

It is well-documented that the patients with a higher health literacy have better health outcomes.⁴ Roughly 10% of the adults in the United States have less than a high school education and therefore would have difficulties understanding the online PG resources.⁵ Our study underscores the continued challenges associated with scientific writing for a lay audience and highlights the need for greater collaborations between physicians and online content providers to improve PG patient education.