Readability of British and American medical prose at the start of the 21st century

Articles published in the BMJ and JAMA are available on the internet, albeit for a fee in the case of JAMA. We wanted to determine whether the materials published by these two pre-eminent journals, while physically accessible to a broad population, are likely to be comprehensible to them.

Methods and results

We obtained electronic versions of articles from the BMJ and JAMA published in the first six months of 2001. We limited our analysis to articles that were published as “Papers” in the BMJ or “Original Papers” in JAMA, had structured abstracts, and had first authors with either British or US institutional affiliations. The BMJ published 42 such articles and JAMA 68.

For each article, we noted the national affiliation of the first listed author. We used Readability Calculations software from Micro Power and Light (Dallas, TX) to calculate two validated readability scores—the Flesch ease of readability index¹ and the FOG index.² Flesch scores of <30 and FOG scores of >16 indicate extremely difficult reading, comparable to perusal of a legal contract.³

We performed independent t test analysis of these dependent variables, using both the journal (BMJ or JAMA) and the national affiliation of the first author (UK or US) as grouping variables. We performed a stepwise regression analysis to determine the independent contribution of journal, national affiliation of the first author, and the number of tables, figures, and references to the variation in ease of readability scores.

Articles published in the BMJ were easier to read than those published in JAMA, as indicated by higher mean Flesch scores (31.5 (SD 8.1) v 27.8 (6.4), P=0.009) and lower FOG scores (16.9 (1.6) v 17.8 (1.3), P=0.001). Similarly, articles written by British affiliates were easier to read than those written by US affiliates, as indicated by higher Flesch scores (31.9 (8.0) v 27.7 (6.5), P=0.003) and lower FOG scores (16.7 (1.5) v 17.9 (1.4), P<0.001) (figure).

In stepwise multivariate regression analyses, only first author's nationality significantly contributed to the model, accounting for 7% of the variance in the model predicting Flesch scores (F=9.2, P=0.003) and 13% of the variance in the model predicting FOG scores (F=16.7, P<0.001).

Comment

Medical articles published by two major international journals are extremely difficult to read, according to two readability formulas that have been validated in many settings. Articles in the BMJ were easier to read than those in JAMA, and articles written by British authors were easier to read than those written by US authors. These differences persisted after correction for potential confounders.

The study has several limitations. Firstly, our outcome measures have been used in, but not validated for, the analysis of medical journal content.⁴,⁵ However, the similarity of the scores we obtained to scores reported in other manuscripts suggests that readability scores are replicable for medical journals and have remained stable over time. Secondly, we examined only two journals for a brief period. To generalise findings will require broader studies. Finally, other factors are likely to contribute to variation in readability scores.

Despite these limitations, our findings have two implications. Firstly, researchers who use ease of readability measures in the analysis of medical articles should consider separating articles by journal and author's nationality. More importantly, virtually all of the medical manuscripts we evaluated were extremely difficult to read. Improving the readability of medical manuscripts may enhance their consumption—both by clinicians and the general public.

Figure.

Mean ease of readability scores by journal and first author nationality. (For Flesch scores, higher values indicate easier readability; for FOG scores, lower values indicate easier readability)