Readability of Online Patient Educational Materials for Coronary Artery Calcium Scans and Implications for Health Disparities

Abstract

Background

Coronary artery calcium (CAC) scans can help reclassify risk and guide patient‐clinician shared treatment decisions for cardiovascular disease prevention. Patients increasingly access online patient educational materials (OPEMs) to guide medical decision‐making. The American Medical Association (AMA) recommends that OPEMs should be written below a 6th‐grade reading level. This study estimated the readability of commonly accessed OPEMs on CAC scans.

Methods and Results

The terms “coronary artery calcium scan,” “heart scan,” and “CAC score” were queried using an online search engine to identify the top 50 commonly accessed websites based on order of search results on December 17, 2019. Grade‐level readability was calculated using generalized estimating equations, with observations nested within readability metrics from each website. Results were compared with AMA‐recommended readability parameters. Overall grade‐level readability among all search terms was 10.9 (95% CI, 9.3–12.5). Average grade‐level readability of OPEMs for the search terms “coronary artery calcium scan,” “heart scan,” and “CAC score,” was 10.7 (95% CI, 9.0–12.5), 10.5 (95% CI, 8.9–12.1), and 11.9 (95% CI, 10.3–13.5), respectively. Professional society and news/media/blog websites had the highest average reading grade level of 12.6, while health system websites had the lowest average reading grade level of 10.0. Less than half of the unique websites (45.3%) included explanatory images or videos.

Conclusions

Current OPEMs on CAC scans are written at a higher reading level than recommended for the general public. This may lead to patient misunderstanding, which could exacerbate disparities in cardiovascular health among groups with lower health literacy.

Nonstandard Abbreviations and Acronyms

AMA

American Medical Association

ASCVD

atherosclerotic cardiovascular disease

CAC

coronary artery calcium

OPEM

online patient educational material

Clinical Perspective

What Is New?

• The readability of online patient educational materials about coronary artery calcium scans was above the 10th‐grade level, far exceeding the 6th‐grade reading level recommended by the American Medical Association.

What Are the Clinical Implications?

• Online patient educational materials are influential to patient decision‐making but may cause confusion or misunderstanding if written at a reading level that is too complex.

• Clinicians, professional societies, and other contributors to online health information should be mindful of the readability of their materials to reduce disparities in care caused by low health literacy, which disproportionately affects underserved groups.

Coronary artery calcium (CAC) is strongly correlated with atherosclerotic cardiovascular disease (ASCVD) risk and has been shown to significantly enhance risk prediction beyond traditional risk factors. ¹ The 2018 American College of Cardiology/American Heart Association cholesterol treatment guidelines recommend considering CAC scans when patients are at borderline to intermediate ASCVD risk if there is still uncertainty about shared treatment decisions. ² Although CAC scanning has been shown to be a useful noninvasive tool to reclassify ASCVD risk, CAC scans are rarely covered by medical insurance, requiring patients to pay out of pocket. ³ To guide shared decision‐making about CAC scans, patients are likely to access online patient educational materials (OPEMs) for more information on this topic. ⁴ , ⁵

The utilization of evidence‐based tools in medicine is heavily influenced by patients' knowledge and understanding of health information, also known as health literacy. ⁶ The American Medical Association (AMA) recommends that health information be written below a 6th‐grade level to ensure comprehension by broad audiences. ⁷ OPEMs have been shown to greatly influence patient decision‐making, yet several studies have found that readability of OPEMs for a wide variety of medical conditions may far exceed that which the general public may understand. ⁴ , ⁸ , ⁹

Patients with limited health literacy have poorer overall health status and higher mortality and use fewer preventive services. ¹⁰ Confusion or misunderstanding of CAC‐related OPEMs that are written above an appropriate reading level may prompt patients to forego CAC scans altogether. Thus, we sought to quantify the readability of frequently accessed OPEMs about CAC scans.

METHODS

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Data Acquisition and Refinement

We used the Google search engine to query the first 50 results for each of the following synonymous search terms: “coronary artery calcium scan,” “heart scan,” and “CAC scan.” Location, cookies, and user account information were disabled to avoid search bias. After pooling the websites from the above 3 terms, a total of 150 websites were accessed and downloaded as PDFs and URLs recorded on December 17, 2019. No human patients were recruited for the study. All of the data were collected from websites that are publicly accessible, and thus do not require institutional review board review. The target audience for each source was determined by referring to the website's informational page or mission statement by 2 independent reviewers. Patient‐directed sources were defined as materials intended for patients and the general public. All research journal articles, podcasts, promotional, advertised, or nonpatient‐directed sources such as websites only intended for health professionals or researchers were excluded. We grouped OPEMs into 5 categories: health systems, news/media/blog, governmental, professional society, or unspecified. Each OPEM was then reviewed for advertisements, appointment scheduling tools, images/videos, cited references, and links to additional resources or information.

Assessment of Readability

PDFs of patient‐directed information from each website were formatted into plain text in a separate Microsoft Word document. Similar to the design of prior readability studies, advertisements, images, figures, captions, videos, citations, hyperlinks, disclaimers, and copyright notices were removed. ⁸ , ¹¹ Periods were used to denote the end of all sentences; all other punctuation was removed. Symbols and numerals were converted to text to limit erroneous increases in average reading grade level. Readability was assessed using Readable.com, as done in prior literature. ¹² , ¹³

Statistical Analysis

All analyses were conducted using SAS software 9.4 (SAS Institute Inc.). Grade‐level readability estimates were calculated for each search term using generalized estimating equations with sandwich estimation where the 5 readability scores were nested within each OPEM using the GLIMMIX procedure. All interval estimates were calculated for 95% confidence. Because there is no single best readability metric that has been established, grade‐level readability was calculated using 5 standard readability metrics (Automated Readability Index, SMOG Index, Coleman‐Liau Index, Gunning Fog Index, Flesch‐Kincaid score) in order to produce robust point (mean) and interval (CI) estimates of readability. ⁸ , ¹³ The interval estimates reflect the variability of readability for the 5 readability metrics for each OPEM.

RESULTS

Figure 1 illustrates how CAC OPEMs were selected for analysis. After excluding websites that did not meet our inclusion criteria, our final analysis included 41 websites under “coronary artery calcium scan,” 50 websites under “heart scan,” and 34 websites under “CAC score.” Of these 125 websites, 95 were unique without overlapping results among the 3 search terms. In the excluded websites, 13 were peer‐reviewed journal articles, 2 were podcasts, and 10 were not considered patient‐directed materials.

Figure 1. Data refinement.

An initial query of the first 50 results for each of the 3 search terms “coronary artery calcium scan,” “heart scan,” and “CAC score” provided a total of 150 results. We excluded 25 nonpatient‐directed materials, yielding 125 coronary artery calcium (CAC) online patient educational materials (OPEMs). Of these 125 OPEMs, 95 were unique results with no overlap among search terms.

Website Categories and Characteristics

Table 1 summarizes the website characteristics and categories for each of the 3 search terms. Of the 95 unique results, 58 websites belonged to the health systems category, accounting for 61.1% of the unique results. News/media/blogs was the second largest category with 21 websites, comprising 22.1% of unique results. There were 8 professional society websites and 4 governmental websites. The remaining 4 websites did not fall into any of the above categories and were marked as unspecified.

Table 1.

Website Characteristics and Categories Compared Among Search Terms

	CAC Scan (n=41)	Heart Scan (n=50)	CAC Score (n=34)
Website characteristic, N (%)
Advertisement	0 (0.0)	16 (32.0)	5 (14.7)
Appointment	18 (43.9)	28 (56.0)	10 (29.4)
Image/video	23 (56.1)	21 (42.0)	17 (50.0)
Cited references	13 (31.7)	8 (16.0)	14 (41.2)
Resources/links	26 (63.4)	25 (50.0)	26 (76.5)
Website category, N (%)
Health system	23 (56.1)	34 (68.0)	14 (41.2)
Governmental	2 (4.9)	2 (4.0)	1 (2.9)
Professional society	6 (14.6)	2 (4.0)	6 (17.7)
News/media/Blog	7 (17.1)	10 (20.0)	10 (29.4)
Unspecified	3 (7.3)	2 (4.0)	3 (8.8)

There were 39 unique websites with >1 website characteristic. Appointment scheduling tools were the most common website characteristic observed among all search terms. Scheduling tools appeared more frequently with the search term “heart scan” (n=28, 56.0%) compared with “CAC scan” (n=18, 43.9%) and “CAC score” (n=10, 29.4%). Nearly one third of all health system websites had CAC‐specific advertising (n=19, 32.8%). Less than half of the unique websites (n=43, 45.3%) included an image or video about CAC.

Readability

The average grade‐level readability among all 3 search terms was 10.9 (95% CI, 9.3–12.5). Of the 95 unique websites, Health Imaging had the highest grade‐level readability (18.0; 95% CI, 16.1–19.8), while WebMD had the lowest (6.2; 95% CI, 4.7–7.7). The average grade‐level readability for all OPEMs in the first 50 search results for “coronary artery calcium scan,” “heart scan,” and “CAC score” was 10.7 (95% CI, 9.0–12.5), 10.5 (95% CI, 8.9–12.1), and 11.9 (95% CI, 10.3–13.5), respectively. Figure 2 illustrates the grade‐level readability for each website under the search term “coronary artery calcium scan” (Table S1). Figures S1 and S2 show the grade‐level readability for websites under search terms “heart scan” and “CAC score.”

Figure 2. Average grade‐level readability of the top 50 online patient educational materials (OPEMs) for the “coronary artery calcium scan” search term.

Point (blue) and interval (black) estimates display average grade‐level readability with 95% CIs of the 5 readability metrics for each unique OPEM. Full website titles are listed in Table S1. OPEMs are organized by increasing grade level. All OPEMs surpassed the American Medical Association (AMA)–recommended readability parameters for online health information (6th‐grade reading level depicted by a vertical red line) except for WebMD, which had the lowest average grade‐level readability of 6.2 (CI, 4.7–7.7).

The grade‐level readability for each website category is illustrated in Figure 3. Average grade‐level readability ranged from 10.0 for health systems (95% CI, 9.5–10.4) to 12.6 for professional societies (95% CI, 11.0–14.2) and news/media/blogs (95% CI, 11.6–13.6). Table 2 summarizes website characteristics and readability of the 5 website categories among the 95 unique results. CAC‐specific advertisements and appointment scheduling tools were only present in the health system category. Websites with appointment scheduling tools had an average grade‐level readability of 9.9, while websites with supplemental images/videos had an average grade‐level readability of 10.8.

Figure 3. Average grade‐level readability with 95% CIs for each website category.

Each circle represents a readability score for 1 online patient educational material (OPEM); there are 5 mean readability scores for each unique OPEM. Governmental and health system websites had the lowest average reading grade levels of 10.2 and 10.0, respectively. News/media/blog and professional society websites both had an average reading grade level of 12.6. Websites in the unspecified category had the highest average reading grade level of 12.1. The lower bounds of the 95% CIs for all categories exceed the American Medical Association–recommended 6th‐grade level.

Table 2.

Average Grade‐Level Readability With 95% CIs and Website Characteristics Among Categories for All Unique Websites (n=95)

Category	Average Grade Level [CI]	Advertisement, N (%)	Appointment, N (%)	Image/Video, N (%)	Cited References, N (%)	Resources/Links, N (%)
Health system, n=58	10.0 [9.5–10.4]	19 (32.8)	46 (79.3)	23 (39.7)	7 (12.1)	25 (43.1)
Governmental, n=4	10.2 [8.0–12.4]	0 (0.0)	0 (0.0)	1 (25.0)	2 (50.0)	4 (100.0)
Professional Society, n=8	12.6 [11.0–14.2]	0 (0.0)	0 (0.0)	4 (50.0)	4 (50.0)	6 (75.0)
News/media/blog, n=21	12.6 [11.6–13.6]	0 (0.0)	0 (0.0)	10 (47.6)	8 (38.1)	16 (76.2)
Unspecified, n=4	12.1 [7.8–16.4]	0 (0.0)	0 (0.0)	2 (50.0)	2 (50.0)	2 (50.0)
Total, n=95	10.9 [9.3–12.5]	19 (20.0)	46 (48.4)	40 (42.1)	23 (24.2)	53 (55.8)

DISCUSSION

We found that the average reading grade level of OPEMs pertaining to CAC scans far exceeded AMA readability recommendations that OPEMs be written at or below a 6th‐grade reading level to meet the needs of the general public. Average grade‐level readability of OPEMs on CAC scans was 10.9 and ranged from 9th to 12th grade, exceeding the average reading level of US adults (8th grade). ⁷ Of 95 unique websites reviewed, only 1 website, WebMD, had a reading grade level (6.2) that came close to the maximum AMA‐recommended reading level. Our results suggest that CAC OPEMs are written at a reading level that is too high for the general public. These findings have several important implications for how patients may make decisions about an important diagnostic test commonly recommended in ASCVD prevention treatment decisions.

It is noteworthy that OPEMs in the professional society and news/media/blog categories had the highest average reading levels of 12.6. Professional societies are regarded as reliable sources given that many organizations seek to advance public health and knowledge, but patients may have difficulty in comprehending health information from these websites. ¹⁴ Despite the importance of these OPEMs, professional societies demonstrate the largest gaps between patient reading skills and OPEM reading level on CAC scans.

These findings align with results from prior studies among a broad range of health conditions that show OPEMs commonly exceed the recommended readability level. Ayyaswami et al ¹¹ recently demonstrated that 99.5% of OPEMs relating to cardiovascular disease were written above the 6th‐grade level recommended by the AMA. OPEMs written at a reading level too difficult for the general public to comprehend are frequent, and high readability levels have been documented among disciplines including common topics related to surgery, oncology, and radiology. ¹³ , ¹⁵

It is recommended that the accessibility of OPEMs be improved by simplifying complex words and sentences, and incorporating images, videos, and diagrams. ¹³ , ¹⁶ , ¹⁷ In our study, we found that fewer than half of all websites used images or videos related to CAC. Interestingly, health system OPEMs, which had the lowest average readability level (10.0) of all website categories, also had the greatest number of websites with images or videos (25, 43.10%). Government websites had similar average readability levels (10.2), but only 1 of 4 included an image.

The percentage of adults with below basic health literacy is considerably higher for populations who identify as Black (24%), Hispanic (41%), American Indian/Alaska Native (25%), and Asian/Pacific Islander (13%) compared with non‐Hispanic White (9%). ¹⁸ Similarly, over half of adults aged >65 years report below a basic health literacy level. ¹⁸ Notably, these populations are known to face a disproportionate burden of cardiovascular risk, which may be incompletely captured with current risk prediction models such as the Pooled Cohort Equations. ² , ¹⁹ , ²⁰ , ²¹ CAC scanning is not only highly predictive of ASCVD risk regardless of age, sex, or ethnicity, but may motivate lifestyle changes that improve cardiovascular risk factors and outcomes. ²² , ²³ , ²⁴

Adhering to the recommended 6th‐grade reading level for OPEMs is a necessary first step to ensure that all patients, including those with limited health literacy, can comprehend online health information. Many clinical preventive and screening services are underutilized by racial/ethnic minorities and older adults because of inadequate healthcare access and low health literacy, but providing more readable OPEMs may help bridge this gap in care. ²⁵ , ²⁶ , ²⁷ , ²⁸ Despite the consensus that OPEMs are frequently written above the general public's reading ability, regulations needed to address this can be challenging to enforce. In the interim, our study findings remind clinicians to consider the readability of OPEMs and patient literacy when identifying information on CAC scans. Ensuring shared decision‐making requires a shared understanding of the risks and benefits of any diagnostic testing procedure.

A unique strength of our study is that we incorporated readability results from 5 different standard readability metrics. Although there was some variation among the different scales, the majority of reading grade‐level estimates supported our hypothesis that CAC OPEMs are written above the recommended 6th‐grade reading level. We also included a thorough review of possible patient queries by analyzing the first 50 results for 3 synonymous search terms.

Our study should be interpreted in the context of some limitations. We did not account for other search engines besides Google. However, 83% of participants in a Pew Research Center survey report using the Google search engine most frequently. ²⁹ Common to other OPEM readability studies, the readability metrics used do not consider the inherent complexity of some medical terms. ⁸ , ⁹ The assumption that the number of syllables is correlated to readability may also generate artificial increases in reading grade‐level assessments for multisyllabic words, regardless of how well defined or easily understood it is by the general public.

Conclusions

We found that the grade‐level readability of OPEMs relating to CAC greatly exceeded the 6th‐grade reading level recommended by the AMA. This gap in readability may disproportionately affect patients with low health literacy. With 70% of people making medical decisions based on health information they find online, OPEMs should be mindful of the readability of their content. ⁶ Ensuring that online content is targeted to broad audiences is essential as patients continue to rely on web‐based searches to guide clinical decision‐making about important diagnostic cardiovascular tests.

Sources of Funding

Rodriguez was funded by a career development award from the National Heart, Lung, and Blood Institute (K01 HL 144607) and the American Heart Association/Robert Wood Johnson Harold Amos Medical Faculty Development Program. Publication made possible in part by support from the UCSF OPen Access Publishing Fund.

Disclosures

None.

Supporting information

Table S1

Figures S1–S2

(J Am Heart Assoc. 2020;9:e017372 DOI: 10.1161/JAHA.120.017372.)