Parental Actionability of Educational Materials Regarding Laryngotracheal Reconstruction

Abstract

This study assesses the understandability and actionability of parental educational materials for laryngotracheal reconstruction in children.

Laryngotracheal reconstruction (LTR) can be a life-saving procedure for children with subglottic stenosis. Nevertheless, as many as 20% of patients experience some complications, such as scarring, infection, restenosis, or fistulization. Rarer complications, such as anastomotic separation and laryngeal edema, can be deadly. In such circumstances, when the medical decisions that parents make for their children have important consequences, parents must understand the information that they read to make the most informed decisions. In 2014, the Agency for Healthcare Research and Quality developed a validated instrument called the Patient Education Materials Assessment Tool (PEMAT) that determines the understandability and actionability of written health information. We evaluated LTR-related patient educational materials by using the PEMAT.

Methods

This study was deemed to be exempt from approval by the institutional review board of Boston University School of Medicine. A Google search was performed using the search term laryngotracheal reconstruction. The first 100 results were screened for LTR patient educational materials. Exclusion criteria consisted of articles unrelated to LTR, articles not written for patients, and non-English articles. Three of us (K.R.K., A.G., and C.J.C.-S.L.) scored each article using the PEMAT. The PEMAT is divided into the domains of understandability and actionability. Both understandability and actionability are scored from 0 to 100, with higher scores denoting more understandable or actionable materials. Reading difficulty was calculated using the Flesch-Kincaid Grade Level readability test, Gunning-Fog Index, Coleman-Liau Index, Automated Readability Index, and Simple Measure of Gobbledygook. Readability formulas are graded on a numerical scale, with scores reflecting the equivalent academic grade needed to understand a document (eg, 5 indicates a fifth grade reading level). Interrater reliability was assessed using the Fleiss κ statistic. We used the Spearman correlation (r value) to determine the association among readability, understandability, and actionability.

Results

Thirty-five unique articles met inclusion criteria. Interrater agreement was substantial for understandability (κ = 0.72; 95% CI, 0.68-0.76) and actionability (κ = 0.76; 95% CI, 0.72-0.78). Median understandability score was 72.2 (range, 25.0-93.8) (Table). Most articles provided information in a logical sequence, used everyday language, and organized material into sections. Only a minority of articles provided visual cues, defined medical terms, or provided titles and/or captions. The median actionability score was 50.0 (range, 20.0-100.0). No articles provided instructions for calculations or explained how to use diagrams. We found a positive correlation between understandability and actionability (r = 0.58).

Table. PEMAT Scores for 35 Patient Education Material Articles.

Domain	Scorea
	Reviewer 1	Reviewer 2	Reviewer 3
Understandability
Median (IQR)	76.9 (74.6-82.3)	80.0 (75.0-83.3)	58.3 (50.0-75.0)
Range	50.0-88.2	50.0-94.1	25.0-93.8
Actionability
Median (IQR)	80.0 (80.0-90.0)	40.0 (20.0-60.0)	60.0 (60.0-80.0)
Range	80.0-100.0	20.0-60.0	40.0-100.0

Abbreviations: IQR, interquartile range; PEMAT, Patient Education Materials Assessment Tool.

^aScores range from 0 to 100, with higher scores denoting more understandable or actionable materials.

The median Flesch-Kincaid Grade Level score was 13.2 (range, 7.3-18.3); Gunning-Fog Index, 15.4 (range, 8.8-20.7); Coleman-Liau Index, 13.3 (range, 6.8-20.0); Automated Readability Index, 12.6 (range, 4.7-18.7); and Simple Measure of Gobbledygook, 14.5 (range, 9.4-18.4). We found a negative correlation between readability and understandability (r = −0.34) and between readability and actionability (r = −0.43).

Discussion

Although the internet has become a popular source of health information, the quality of online patient education materials is often questionable. Previous studies evaluating written health information are limited by the validity of methods used. To overcome previous limitations, we used the validated PEMAT instrument.

Results suggest that revisions may be warranted for current LTR information to benefit a larger readership. We found that understandability and actionability often did not meet minimum recommended levels (a score of 70) necessary for patients to understand or act on the information they receive. Most LTR-related information was also too difficult for the average American adult to read. Articles with more difficult reading levels were harder to understand, which is consistent with results by Balakrishnan et al, who also found a negative correlation between reading difficulty and understandability. Correlation studies also demonstrated that articles with more difficult reading levels were less actionable and that patients are more likely to act on information they can understand. To our knowledge, this study is the first to assess these last 2 associations and further underscores the importance of understandable health information. An important limitation is that the PEMAT cannot assess the quality of information provided. For example, text can be understandable and actionable but still contain information that is biased or inaccurate.

We propose several ways to improve current LTR-related information. Providing visual cues, limiting medical jargon, and including summaries are important first steps to improve understandability. Writers should also address the reader directly, break down actions into individual steps, and provide tangible tools to help readers take action, all of which are important components of actionable health information.