Does Animation Improve Comprehension of Risk Information in Patients with Low Health Literacy? A Randomized Trial

Abstract

Introduction:

Enhanced visual effects, like animation, have the potential to improve comprehension of probabilistic risk information, particularly for those with lower health literacy. We tested the effect of presentation format on comprehension of colorectal cancer (CRC) screening probabilities to identify optimal risk communication strategies.

Methods:

Participants from a community foodbank and a cancer prevention center were randomized to 1 of 3 CRC screening risk presentations. The presentations used identical content, but varied in format: 1) video with animated pictographs, 2) video with static pictographs, and 3) audiobooklet with static pictographs. Participants completed pre- and post- presentation surveys. The primary outcome was knowledge of probability/risk information regarding CRC screening, calculated as total, verbatim, and gist scores.

Results:

187 participants completed the study and were included in this analysis. Median age was 58 years (Interquartile Range [IQR]: 14 years), most participants were women (63%), and almost half had a high school education or less (46%). Approximately one quarter had inadequate health literacy (S-TOFHLA marginal/inadequate: 28%; Brief Health Literacy Screener low: 18%) and about one half had low numeracy (Subjective Numeracy Scale low: 54%; Graphical Literacy Measure low: 50%). We found no significant differences in total, verbatim, or gist knowledge across presentation formats (all p > 0.05).

Discussion:

Use of an animated pictograph to communicate risk does not appear to augment or impede knowledge of risk information. Regardless of health literacy level, difficulty understanding pictographs presenting numerical information persists. There may be a benefit to teaching or priming individuals on how to interpret numerical information presented in pictographs before communicating risk using visual methods.

Trial Registry:

NCT02151032

INTRODUCTION

Understanding and applying probabilities and risk information is an essential part of making informed healthcare decisions.¹ An informed decision occurs when the patient has an adequate understanding of the health condition, treatment options, and benefits and harms, and the decision is consistent with the patients’ personal values.^1–4 Knowledge, values, attitudes and behavior are integral features of an informed decision; yet, knowledge is one of the most important indicators of informed decision making.^4,5 To enhance understanding, when communicating probabilities to patients, evidence supports using natural frequencies and illustrations (e.g., cartoons, bar charts, pictographs).^6,7 Moreover, structured, tailored or interactive communication tools can increase patient knowledge.^6,7 These strategies are particularly important for those with low health literacy.¹

Health literacy is “the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions.”⁸ It is estimated that approximately half (50%) of adults in the United States (US) demonstrate health literacy related challenges and over one-third (36%) demonstrate skills that are consistent with basic or below basic health literacy.⁹ Skills like retaining and applying specific numerical information, presenting information visually (e.g., graphs), and using technology to access information all contribute to one’s health literacy.¹⁰ Numeracy, defined as how comfortable one is with numeric information, is a critical component of health literacy in risk information processing.¹¹ Thus, individuals with low health literacy may have a diminished capacity to understand risk information. In addition, these individuals are often less aware of preventive services, like cancer screening. Combined, these may inhibit their ability to make informed cancer screening decisions.¹ Optimal strategies to communicate risk information in the context of cancer screening are needed.

In a review of computer-based visual presentations of risk information by Stellamanns and colleagues,¹² static graphical depictions were found to improve comprehension, yet the results were inconsistent across studies. In the case of dynamic depictions, the authors found that additional evaluation is needed among those with limited numeracy and graphical literacy to make a determination about theimpact on risk communication.¹² Recommendations exist for presenting risk information to those with low numeracy (e.g., plain language, absolute risk, incremental risk, pictographs, presentation of only critical information, and time interval for risk occurrence).¹³ However, strategies for optimal risk communication are also lacking. Moreover, Hawley et al.¹⁴ found that viewing pictographs resulted in adequate gist and verbatim risk knowledge in those with low numeracy, and the information was considered more trustworthy by those with high or low numeracy. In addition, Tait et al.¹⁵ found that regardless of numeracy levels, pictographs significantly increased both gist and verbatim knowledge and were better accepted by patients when compared to text and tables. Therefore, using dynamic depictions like animation in combination with pictographs may be an effective risk communication strategy for those with low health literacy.

Communication tools designed for various health conditions have used pictographs in multiple ways; for example, computer-generated graphics accompanied by text, voice narration, or both;^16–18 animated icon arrays;^19,20 interactive games;²¹ risk calculators;²² and the use of personalized, animated avatars.²³ While a number of these interventions were successful in reducing decisional conflict,¹⁶ increasing risk perception and intentions,²³ and eliciting emotional responses to risk messages,²¹ few studies have measured patient recall, and their results are mixed.^18,19,24,25 Additionally, few animated interventions have been developed specifically for low health literacy populations.^26–28

With the goal of improving risk communication among low health literacy populations, we developed a decision aid for colorectal cancer (CRC) screening suitable for those with low health literacy in 3 formats: an audiobooklet, a video with static images and a video with animated images. The objectives of this investigation were to: 1) adapt an existing decision aid for 3 strategies for communicating probabilities, 2) test their effectiveness in increasing patient understanding of risk information, and 3) examine the intervention effects by health literacy levels.

METHODS

Study Participants

Participants for this randomized controlled trial were recruited from 2 sites in Houston, Texas: 1) The Cancer Prevention Center at The University of Texas MD Anderson Cancer Center and 2) Bethel’s Heavenly Hands. These settings were selected to recruit those with a broad range of health literacy levels. The Cancer Prevention Center serves approximately 23,000 patients per year and provides cancer screening, cancer risk assessments, and other cancer prevention programs. Patients can make their own appointments or can be referred by physicians. Bethel’s Heavenly Hands is a non-profit organization assisting individuals with food, clothing, shelter, and medical care. Bethel’s Heavenly Hands serves approximately 5,500 people monthly through their food pantry service.

Men and women aged 45–75 years who were English speakers were eligible to participate. Individuals that reported a previous diagnosis of colorectal cancer were excluded. At the Cancer Prevention Center, participants were recruited in November and December, 2012. Trained research assistants approached men and women in the center’s waiting room and invited them to participate in the study. Participants completed the study either immediately or after their appointments at the center. At Bethel’s Heavenly Hands, individuals were invited to participate during weekly food assistance service between December 2012 and January 2013. Participants completed the study immediately or were invited to return the following week if they did not have time to complete it the same day.

The Institutional Review Board at The University of Texas MD Anderson Cancer Center approved the study protocol and consent process.

Randomization and Study Procedures

After potential participants were screened for eligibility and consented, research assistants administered a baseline survey. All participants then viewed a 3:51 minute introductory video that introduced colorectal cancer screening and 3 screening tests: fecal occult blood test, flexible sigmoidoscopy, and colonoscopy. Next, participants were randomized using a double-blind procedure to 1 of 3 intervention arms, as indicated in a sealed randomization assignment envelope: 1) audiobooklet with static pictographs; 2) video with static pictographs; or 3) video with animated pictographs (Table 1). We used computer generated, permuted-block randomization per study site with a block size of 3 and an allocation ratio of 1:1. After randomization, participants viewed the corresponding intervention. The audiobooklet ran for 3:54 (7:45 total minutes), the static video ran for 3:20 (7:11 total minutes), and the animated video ran for 3:22 (7:13 total minutes). Research assistants then administered post-surveys. All participants received a $30 gift card for participating. Participants at the Cancer Prevention Center were also compensated for their parking expenses.

Table 1.

Description and comparison of colorectal cancer screening decision aid intervention arms

Feature	Audiobooklet	Static Video	Animated Video
Description	Booklet with accompanying narration	FIMDM patient decision aid video	FIMDM patient decision aid with animation features
Introduction/Overview with Risk Communication	Addresses 3 questions: 1) How many people get colorectal cancer in their lifetime? 2) How many will die from colorectal cancer? 3) How many will be saved by screening?	Same	Same
Voice-over Narration	Yes	Yes	Yes
Risk Presentation	Pictographs	Pictographs	Pictographs
Animation	None	Minimal (overlay reinforcing information)	Enhanced (animation highlights the randomness of affected individuals)
Time (minutes)
Introduction Time	3:51	3:51	3:51
Intervention Time	3:54	3:20	3:22
Total Time	7:45	7:11	7:13

^*Note: Abbreviation: FIMDIM=Foundation for Informed Medical Decision Making.

Intervention Development

Our initial aim was to develop a decision aid by identifying the essential tasks patients perform in order to understand probabilistic health information and use the information to make informed, quality decisions. To this end, our research team reviewed relevant empirical literature and identified essential probabilistic information a patient needs to understand to interpret the magnitude of risks and benefits. (Appendix Table 1). This probabilistic information included: 1) the relevant group, 2) the denominator, 3) the numerator, 4) the time frame, 5) the incremental benefit, 6) rare events, and 7) uncertainty with risk estimates. Our strategies for intervention design were based on these tasks. To further support our development approach, we referred to existing literature to understand what attributes of screening tests patients consider important while making a decision.^15,29–31 Based on these reviews, we identified animated visual cues (e.g., highlighting, flashing, movement) as a strategy to enhance attention to these essential tasks.

To incorporate visual cues into our intervention development, we reviewed an existing CRC screening decision aid (booklet and video) developed by the Foundation for Informed Medical Decision Making and distributed by Health Dialog.³² We studied the probabilities communicated by the aid including incidence and mortality information about colorectal cancer, screening for CRC, and different tests and their advantages and disadvantages. Based on this research, we developed risk communication modules to communicate risk estimates about colorectal cancer and CRC screening. A risk communication expert panel, made up of clinicians and researchers, reviewed these modules and provided feedback based on their clinical perspective and expertise.

We edited the original 30-minute Health Dialog video down to 3:51 minutes to reduce patient burden and adapted it for a low health literacy audience; changing language to 8^th grade reading level from 9^th-11^th grade level by bulleting the text, adding new pictographs, simplifying by removing information on testing options not usually available to low-literacy populations in Houston, TX (e.g., imaging tests), and eliminating patient and physician vignettes. This version of the video was used to enhance understanding of CRC screening introductory concepts among all participants.

Next, we created a prototype of an animated video to communicate the elements of our risk communication modules (i.e. incidence, incremental benefit, and efficacy of detecting cancer per screening test). The video presented information via bulleted text, graph and static images (i.e., pictographs), with voice-over narration. Finally, we developed 2 other versions of this video for comparison: a version with static images, and a print booklet with an audio narration track.

Cognitive Testing of the Animated Prototype

We conducted 3 rounds of in-person cognitive interviews with eligible individuals at increased risk for low numeracy and low health literacy (n=20). We included English-speaking men and women aged 45 to 75 years with no history of colorectal cancer. All participants first watched an introductory video on CRC screening. They then watched and evaluated the animated video prototype in terms of appearance, layout, and color. Their understandability of the graphics, animation, and narration describing probabilistic information was assessed by interviews, which lasted 45–60 minutes. We changed and improved the video iteratively after each iteration. The updated animated prototype was sent to the expert panel for a final review.

Final Products

One of our final products consisted of a short introductory video (3:51 minutes): an on-screen narrator who introduced screening, CRC, and recommendations for regular screening. The narrator briefly described colon anatomy and polyps, with accompanying graphics. Next, the video introduced 3 CRC screening tests: fecal occult blood test (FOBT), sigmoidoscopy, and colonoscopy. Using a combination of narrative, text, and minimal graphics, the video gave a brief description of each test procedure, its advantages, an explanation of test results, and recommended testing frequency. It also informed the patient about what to expect before, during, and after each test, including sedation: colon preparation, potential discomfort or unpleasantness, and time commitments. All participants viewed the introductory video and CRC risk information.

Participants were then randomized into viewing 1 of our 3 other final products with the same basic content, i.e., 1 of our 3 planned interventions: the audiobooklet, the static video, or the animated video (Table 1).

The first intervention, the audiobooklet, consists of ten pages, plus front and back covers, and has exactly the same content, text and graphics as the videos. The audio-track is also the same narrative, with a prompt at the end of each page asking the patient to turn to the next page.

The second intervention, the static video, presents CRC risk information. After recapitulating the introductory video, it puts forth and then addresses 3 questions sequentially: 1) how many people get colorectal cancer in their lifetime? (incidence); 2) how many will die from colorectal cancer? (mortality); and 3) how many will be saved by screening? (mortality benefit). Static images begin with an icon array of 100 images depicting “smiley” faces (Figure 2), which represents a random sample of 100 persons in the US. Differing from the audiobooklet, as the narration advances in the static video, the icon colors change to depict the percentage of people who will get CRC, die from CRC, or be saved via early detection by screening. Images are presented without animation. The icons representing “affected” individuals are presented in a non-random manner (i.e., all in a row). A graph explains that the risk of colorectal cancer increases with age, and the benefit of CRC screening decreases after age 75 years. The video also contains text, which gives information about the recommended age for CRC screening, states that it can detect cancer early and advises the patient to talk to their doctor about CRC screening.

Figure 2. Images of Risk Concepts, Task Descriptions and Multimedia Strategies for Displaying Risk Estimates.

Images of the Risk Concepts, Task Descriptions and Multimedia Strategies for Displaying Risk Estimatesdescribed in Appendix 1.

*Note: These numbers refer to the tasks described in Appendix Table 1.

The third intervention, the animated video, has the same basic content and narration as the static video, with the addition of enhanced computer-generated animation: highlighting, flashing, and movement for emphasis. For example, the animation highlights the randomness of who is selected from the population and who is affected by the disease: the 6 yellow icons out of 100 representing “affected” individuals appear randomly among the other green icons and keep changing to different icons until the graphic finally settles on “highlighting” the top left 6 icons as “affected,” similar to the static video. For all presented questions, the particular group of icons discussed to answer those questions will flash and sometimes move forward for emphasis when the narrator talks about them. The video contains the same text as the static video, plus an animated version of the graph.

Study Measures

Baseline Measures

Participants completed a 3-item CRC screening indicator questionnaire and a 7-item CRC baseline knowledge questionnaire (Table 2).

Table 2.

Sociodemographic and baseline characteristics of the participants.

	Total	Audiobooklet	Static Video	Animated Video
Sociodemographic Characteristics (N=187)	n (%)	n (%)	n (%)	n (%)
Age
45–59 years	110 (58.8)	34 (54.8)	36 (58.1)	40 (63.5)
60–75 years	77 (41.2)	28 (45.2)	26 (41.9)	23 (36.5)
Gender (female)	118 (63.1)	40 (64.5)	35 (56.5)	43 (68.3)
Race/Ethnicity
African American/Black	131 (70.1)	39 (62.9)	45 (72.6)	47 (74.6)
White	38 (20.3)	17 (27.4)	11 (17.7)	10 (15.9)
Hispanic/Latino	7 (3.7)	1 (1.6)	4 (6.5)	2 (3.2)
Other/Mixed	11 (5.9)	5 (8.1)	2 (3.2)	4 (6.2)
Education (n=185)
Less than High School	36 (19.5)	11 (18.3)	15 (24.2)	10 (15.9)
High School Graduate	49 (26.5)	11 (18.3)	16 (25.8)	22 (34.9)
Some College	68 (36.8)	24 (40.0)	23 (37.1)	21 (33.3)
College Degree or More	32 (17.3)	14 (23.3)	8 (12.9)	10 (15.9)
Study site
BHH	122 (65.2)	40 (64.5)	41 (66.1)	41 (65.1)
CPC	65 (34.8)	22 (35.5)	21 (33.9)	22 (34.9)
Health Literacy Measures*
S-TOFHLA (marginal/inadequate)	53 (28.3)	11 (17.7)	20 (32.3)	22 (34.9)
BHLS: Brief Health Literacy Screener (low)	34 (18.4)	9 (14.8)	17 (27.4)	8 (12.9)
SNS: Subjective Numeracy Scale (low)	100 (53.8)	36 (58.1)	34 (54.8)	30 (48.4)
Graphical Literacy Measure (low)	94 (50.3)	33 (53.2)	34 (54.8)	27 (42.9)
CRC Screening Indicator Questions
Heard of colon cancer screening?† (yes)	161 (86.1)	52 (83.9)	54 (87.1)	55 (87.3)
Screened for colon cancer?‡ (yes)	93 (49.7)	32 (51.6)	29 (46.8)	32 (50.8)
Plan to screen for colon cancer?§ (yes)	166 (88.8)	58 (93.6)	51 (82.3)	57 (90.5)
Baseline Knowledge	Mean (SD) n=186	Mean (SD) n=61	Mean (SD) n=62	Mean (SD) n=63
7-item Knowledge of CRC Questionnaire‖	65.1 (22.0)	69.1 (19.4)	63.8 (21.8)	62.4 (24.4)

Note. Abbreviations: SD (standard deviation); BHH (Bethel’s Heavenly Hands); CPC (Cancer Prevention Center); CRC (colorectal cancer); S-TOFHLA (Short Test of Functional Health Literacy in Adults)

^*Cut-points derived from Housten et al., 2018³⁸

^†Before today, had you ever heard of colon cancer screening?

^‡Have you ever been screened for colon cancer?

^§In the future, do you plan to get screened for colon cancer?

^‖7-item CRC Knowledge Questionnaire: 1) Does your risk of colon cancer increase with age?; 2) Does smoking affect your chance of getting colon cancer?; 3) Does eating foods high in fiber reduce the risk of colon cancer?; 4) If one of your parents gets colon cancer, does it increase your chance of getting colon cancer?; 5) Are white people more likely to get colon cancer than Blacks of African-Americans?; 6) Can a person have colon cancer without having pain or other symptoms?; 7) Is there a test that can find colon cancer early?

Primary Outcomes: Total, Verbatim, and Gist CRC Screening Knowledge

Participants’ knowledge of colorectal cancer risks and benefits were measured on the post-intervention survey. Verbatim knowledge, or the ability to correctly identify specific probabilities, was evaluated by 5 questions pertaining to numbers and risk estimates referred to within the interventions.³³ Gist knowledge was measured by 6 questions eliciting responses based on the participants’ overall understanding of the key messages in the interventions.³³ Total knowledge was measured by combining the verbatim and gist knowledge questions, and adding 3 additional questions that were quantitative in nature, but without specific numeric information (14 total questions). Scores were assigned for each question; for analyses, the percentage of questions answered correctly was calculated to be used as the main outcome.

Health Literacy Measures

We used objective and subjective measures of health literacy to capture the multiple facets that make-up health literacy. We measured: health literacy, numeracy, and graphical literacy since they were most relevant to the developed interventions.

The Short Test of Functional Health Literacy in Adults (S-TOFHLA) is a health literacy measure made up of 36 items to be completed in 7 minutes. Scores range on a continuous scale from 0–36, based on the scoring system developed by Baker et al.³⁴ Participants were classified into 3 levels based on their scores: “inadequate” (0–16), “marginal” (17–22) and “adequate” (23–36) health literacy. The “inadequate” and “marginal” categories were combined into 1 category, and were compared to “adequate” health literacy category in this analysis (inadequate/marginal vs. adequate).

The Brief Health Literacy Screener (BHLS) assess participants’ health literacy using 3 items with Likert-type response options ranging 1–5.^35,36 Higher scores indicate higher health literacy. Scores on the 3 questions were summed to give an overall health literacy score ranging from 3–15. Participants with scores ≤9 were classified as “low” scores and >9 were “high” scores.³⁷

Numeracy was measured using the 8-item Subjective Numeracy Scale (SNS) that asked participants to indicate their level of comfort with performing certain simple mathematical operations.¹¹ Participants could choose a number ranging from 1–6 to indicate their comfort with numbers. The average of their responses to all 8 items was calculated to give their total subjective numeracy score (range 1–6). Participants with scores ≤4 were classified as “low” scores and >4 were “high” scores.³⁸

The Graphical Literacy Measure (GLM)³⁹ was used to assess participants’ understanding of visual depictions of information. The GLM asked participants to interpret graphical images using a 13-item questionnaire. Each correct answer received a score of 1, and each incorrect answer, a score of 0. The summed score for total graphical literacy was a continuous variable with possible scores ranging from 0–13. Participants with scores ≤6 were classified as “low” scores and >6 were “high” scores.³⁸

Program Evaluation

To ensure the interventions materials were perceived as credible by the participants, we measured the perceived credibility using the following 5 questions adapted from Han et al.³⁰: 1) How accurate does the information seem to you?; 2) How much do you trust the information presented in this program?; 3) How believable is the information presented in this program?; 4) How trustworthy is the information that was presented in this program; and 5) How reliable is the information that was presented in this program? Each question used a response format ranging from “Not at all” (0) to “Completely” (10). The credibility scale was summed to yield a range from 0 (lowest credibility) to 50 (highest credibility).

Hypothesis

We hypothesized that participants who viewed the video with animated pictographs, compared to participants who viewed the video with static pictographs or audiobooklet, would have higher total, verbatim, and gist knowledge scores. Secondarily, we hypothesized that the differences in knowledge scores between the 3 intervention arms would be greater for patients with lower health literacy than for patients with higher health literacy.

Statistical Analysis

Participants in the different intervention arms were compared for demographic differences at baseline. We evaluated the primary hypothesis using a regression model with an indicator of static versus animated group (the audiobook group was excluded from this analysis). Additional analyses with all three randomized groups were conducted (animated versus audiobook; static versus audiobook). Covariates were included in the model in a subsequent analysis. We used robust standard errors to protect against residual heterogeneity. For our secondary outcome, we modeled the primary outcome using indicator variable of treatments (animated versus static), the health literacy score, and their interaction.

RESULTS

All models met the assumptions for linear regression. Residual plots and histograms showed no patterns indicating lack of normality, linearity or homoscedasticity.

Demographic Characteristics

One hundred and eighty-seven participants completed the post-intervention survey and were included in these analyses. Participant characteristics are described in Table 2. A majority of the participants were female (63.1%), African-American/Black (70.1%) and between ages 45–59 years (58.8%). Approximately half (46.0%) of participants had a high school education or less. Few participants (13.9%) had never heard of CRC screening; however, half (49.7%) of participants had previously completed CRC screening and almost 90% said that they had plans to be screened in the future. Participants had moderate knowledge of CRC screening (mean score: 65.1% correct answers on baseline knowledge questionnaire).

Effect of Intervention Format on CRC Screening Knowledge

The intervention format (i.e., audiobooklet, static video, animated video) did not have a significant effect on the total, verbatim, or gist knowledge scores (Table 3). Participants in all 3 intervention arms had, on average, high gist knowledge (>70% correct answers), and moderately high total and verbatim knowledge (>50%). Analysis with all 3 randomized arms did not show any significant result in the comparison between audiobooklet vs. static video (mean [standard deviation]; audiobooklet: 66.8 [16.4] vs. static video: 62.0 [20.4]; p=0.15) or audiobooklet vs. animated video (audiobooklet: 66.8 [16.4] vs. animated video 65.0 [18.6]; p=0.56).

Table 3.

Knowledge scores between the 3 intervention groups (unadjusted).

	Intervention Group				P-values
	Total (n=187)	Audiobooklet (n=62)	Static Video (n=62)	Animated Video (n=63)	Audiobooklet vs. Static Video	Audiobooklet vs. Animated Video
Knowledge Measures	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)
Total Score*	64.6 (18.5)	66.8 (16.4)	62.0 (20.4)	65.0 (18.6)	0.15	0.56
Verbatim Score†	56.6 (22.7)	58.3 (21.4)	55.0 (23.7)	56.6 (23.2)	0.43	0.67
Gist Score‡	75.2 (20.6)	78.2 (16.7)	71.2 (23.4)	76.2 (20.7)	0.06	0.55

Note. Abbreviations: SD (standard deviation).

^*Total Score: percent correct out of the total 14-items measuring knowledge.

^†Verbatim Score: percent correct out of 8-items pertaining to specific numbers and risk estimates referred to within the interventions.

^‡Gist Score: percent correct out of 6-items pertaining to overall understanding of the key messages within the interventions.

Health Literacy Scores

A quarter of participants had inadequate health literacy (S-TOFHLA marginal/inadequate: 28.3%; BHLS low: 18.4%) and over half had limited numeracy (SNS low: 53.0%; GLM low: 50.3%). Mean scores for various health literacy and numeracy measures did not significantly differ by intervention arm when examining total and verbatim knowledge scores (Table 4). However, mean gist scores were significantly different for those who had adequate health literacy based on S-TOFHLA scores (audiobooklet mean: 81.4; static video mean: 74.2; animated video mean: 85.8; p=0.01). Additionally, there was no statistically significant interaction between health literacy and the 3 randomized study arms.

Table 4.

Comparison between health literacy levels and intervention group on knowledge scores.

	Audiobooklet		Static Video		Animated Video		P-values
	n	Mean (SD)	n	Mean (SD)	n	Mean (SD)
Total Score
S-TOFHLA adequate	51	69.8 (14.5)	42	65.7 (19.1)	41	71.6 (15.4)	.24
S-TOFHLA inadequate/marginal	11	53.3 (18.2)	20	54.3 (21.2)	22	52.6 (18.0)	.96
BHLS high	52	65.9 (16.4)	45	64.0 (19.5)	54	65.2 (18.7)	.87
BHLS low/inadequate	9	69.8 (16.7)	17	56.7 (22.2)	8	62.5 (19.8)	.31
SNS high	26	71.4 (17.7)	28	64.8 (23.8)	32	72.5 (15.0)	.25
SNS low/inadequate	36	63.5 (14.7)	34	59.7 (17.0)	30	56.0 (18.0)	.19
Graph high	29	77.1 (11.8)	28	74.2 (13.5)	36	72.8 (15.3)	.46
Graph low/inadequate	33	57.8 (14.4)	34	51.9 (19.6)	27	54.5 (17.5)	.38
Verbatim Score
S-TOFHLA adequate	51	61.0 (19.8)	42	59.2 (21.8)	41	61.0 (21.7)	.90
S-TOFHLA inadequate/marginal	11	45.5 (24.5)	20	46.3 (25.7)	22	48.3 (24.2)	.94
BHLS high	52	56.5 (20.9)	45	56.1 (24.4)	54	56.7 (24.1)	.99
BHLS low	9	65.3 (22.3)	17	52.2 (22.2)	8	54.7 (18.8)	.34
SNS high	26	62.5 (22.6)	28	58.5 (25.0)	32	65.6 (19.1)	.47
SNS low	36	55.2 (20.1)	34	52.2 (22.5)	30	45.8 (22.8)	.22
Graph high	29	69.4 (19.1)	28	68.3 (18.5)	36	63.9 (20.0)	.47
Graph low	33	48.5 (18.4)	34	44.1 (22.0)	27	46.8 (23.9)	.70
Gist Score
S-TOFHLA adequate	51	81.4 (14.8)	42	74.2 (22.8)	41	85.8 (13.7)	.01
S-TOFHLA inadequate/marginal	11	63.6 (18.0)	20	65.0 (24.1)	22	58.3 (19.8)	.57
BHLS high	52	78.5 (17.3)	45	74.4 (22.4)	54	76.5 (20.6)	.61
BHLS low	9	75.9 (14.7)	17	62.8 (24.7)	8	72.9 (23.5)	.31
SNS high	26	83.3 (18.3)	28	73.2 (27.7)	32	81.8 (16.6)	.17
SNS low	36	74.5 (14.6)	34	69.6 (19.5)	30	69.4 (22.8)	.45
Graph high	29	87.4 (13.1)	28	82.1 (17.5)	36	84.7 (16.6)	.47
Graph low	33	70.2 (15.5)	34	62.3 (24.0)	27	64.8 (20.3)	.27

Note. Abbreviations: SD (standard deviation). Means are percent correct on the total, verbatim, and gist knowledge questions.

Program Evaluation

The intervention materials were rated as highly credible study the participants, with mean score above 47 out of a possible 50 maximum score. No credibility mean score differences between intervention arms were observed.

DISCUSSION

Few investigations have compared enhanced visual effects for communication of risk information using a cancer screening patient decision aid among those with limited health literacy and numeracy. Total, verbatim, and gist scores were not significantly different across intervention arms (audiobooklet, static, and animated). Moreover, knowledge scores did not differ significantly across intervention arms when stratified by health literacy and numeracy levels. Of minor note, we observed that participants with adequate health literacy according to S-TOFHLA scores had significantly different gist knowledge scores when stratified by intervention arm; however, this was not observed when examining health literacy using BHLS or numeracy using SNS and GLM. Overall, gist knowledge scores were high for all intervention arms. This suggests that our presentation strategies were successful in communicating key risk information to all participants. Thus, these findings suggest that the use of animation in patient decision aids had similar effects on cancer screening risk comprehension as a well-designed static decision aid.

We prioritized recruiting those from minority and potentially vulnerable populations, namely racial/ethnic minorities and those with low health literacy. Among study participants, more than 70% reported their race/ethnicity as Black/African American and most participants were women. In addition, over 50% had low numeracy and about 20% had low health literacy. Enrolling participants from potentially vulnerable populations provides unique insight into the effects of risk communication strategies among a population often motivated to engage in shared decision making yet may have limited true opportunities to actually engage in the decision-making process.⁴⁰ The decision aids used in this study were developed using strategies for a low health literacy audience, including: using plain language; conveying risk information using absolute risk and incremental risk; providing only relevant information about options; and, using simple pictographs both with and without enhanced animation features.^13,41 Our results suggest that use of a well-designed static decision aid may be not more beneficial when compared to a more resource intensive animated video. These findings can help inform risk communication strategies to be used to support decision making among potentially vulnerable populations and in low resource settings.

Our findings regarding the effects of animation in risk communication have been supported by other studies. In a review by Stellamanns and colleagues, static visual aids were found to be effective in increasing understanding; however, they cite a need for further research to evaluate the effectiveness of dynamic formats in risk communication.¹² Our investigation builds upon this work as it is one of the first to examine the use of animation when compared to static depictions of risk in a cancer screening patient decision aid. While use of animation may enhance message relevance, use of well-designed static materials may be as beneficial and may also enhance decision aid feasibility in various contexts.¹⁸ Enhanced animation techniques may be distracting to viewers; thus, using animation to enhance risk communication may obscure the overall risk messaging.^42,43 In contrast, our investigation found that enhanced animation did not augment or impede total, verbatim, or gist knowledge scores. Hence, this unique contribution highlights that animated decision aids may not be differentially impactful in communicating risk information when compared to a well-designed static decision aid. Therefore, researchers may consider their purpose and priorities to decide if the resources associated with developing, maintaining, and implementing decision aid interventions with animation is warranted.

Variations in the presentation of risk information can influence understanding, perceived risks and benefits, and the trustworthiness of the information being conveyed.^{13–15,18,44} Likewise, the context of the medical decision to be made can also influence how risk is perceived and applied.¹³ For example, in the context of a preventive screening decision, like CRC screening, healthy individuals are learning about the frequencies of the benefits and harms associated with screening. In contrast, when a diagnosed patient is making a treatment decision for an aggressive disease, the emotional and probabilistic tasks required of the decision are much higher and may necessitate multiple risk communication strategies to support the decision making process.¹³ For those with limited health literacy, employing pictographs to convey risk accompanied by a plain language description that is easy to process may support the multiple risk communication strategies a complex, high stakes decision demands and may be as beneficial as enhanced animation.^13,18 In addition, teaching or priming those with limited health literacy on how to interpret numerical information presented in pictographs may further provide decision support when making complex decisions.⁴⁰

Limitations

This study recruited participants from a cancer prevention center and community services center located in a large US metropolitan city. The racial/ethnic make-up of our participants did not represent all populations and the ability to communicate in English was an inclusion criterion. Participants recruited from different settings may have had different exposure to health information. Thus, future research that includes participants from various racial/ethnic groups as well as those who prefer to communicate in languages other than English and recruited from various settings may enrich and build upon these findings. Our study was cross-sectional and therefore the long-term follow-up for knowledge retention was lacking. Health literacy and numeracy measures provided valuable insight into patient abilities; however, they are limited in how they measure all factors associated with health literacy and numeracy.³⁸ Additionally, the animation strategies used in the decision aids did not represent all possible uses of animation, e.g., the depiction of time; therefore, future studies investigating the different types and uses of animation may yield insight into the impact of various features on total, verbatim, and gist knowledge scores

CONCLUSION

This study suggests that animation offers no advantage over well-designed static materials in improving knowledge and recall of numerical risk among those with low health literacy. Future work should pursue longitudinal follow-up to assess if these knowledge gains are sustained over time. Use of static visual aids to communicate risk may be as effective as animation and more feasible to implement in a variety of settings. Available resources, such as access to technology and funding, may guide developers’ choice for the use of animation in decision aids.

Figure 1.

Flowchart of study events (CONSORT diagram)

APPENDICES

Appendix Table 1.

Risk Concepts, Task Descriptions and Multimedia Strategies for Displaying Risk Estimates.

Risk Concept	Task Description	Multimedia Strategies
1. Relevant Group	Relevant group: Understand the nature and size of relevant population Group identification: Understand that pictograph “represents individuals like you” Personal risk identification: User should be able to identify that he/she is in the at-risk group	Relevant group: Icons are highlighted/colored/halo’d when narrator mentions the relevant group. Group identification: Uses U.S. map to indicate the entire population Is the relevant group Group identification/Personal risk: Graph shows slope indicating colorectal cancer risk increases sharply after age 50, stops increasing after age 75
2. Denominator	Denominator is a sample representing total population.	We use bulleted text to show both the numerator and denominator We consistently use 100 as the denominator for incidence, mortality, and incremental benefit We present the sample denominator from the population: Sample population is shown from U.S. map and then immediately the 10×10 icon array Narration explicitly states taking a random sample from the U.S.
3. Numerator	Affected and unaffected group: User should understand what the numerator represents (how many affected and how many unaffected) in relationship to the denominator Identify how the numerator is drawn from the denominator Uncertainty around will be affected and who will not be affected	Affected and unaffected group: Both positive and negative framing is used throughout (i.e., narration and bulleted text) Affected and unaffected group: Icons representing the numerator are highlighted and haloed while the unaffected group remains static – no highlights Numerator drawn from the denominator: Narration says, “Of the 100 people” Uncertainty: We use narration such as: “we do not know today who will get colon cancer” 3 of 6 people with colon cancer will die from it. We do not know today who these 3 will be” Uncertainty: Affected group Icons are chosen at random to indicate the affected to indicate the randomness of who will get colorectal cancer or who will die from colorectal cancer. After they are chosen/highlighted, the affected group is lined up in the first row for easier counting. When the narration states “We do not know today who will get colon cancer” the affected icons are chosen again at random
4. Time Frame	Time period for event to occur: User should understand time frame for risk estimate as well as the change per time frame (the change in rate of change) Personalize risk over time: Interpret the given time from in regards to the user (i.e. their age) Change in the rate of change: Understand the concept of a change in the rate of change (e.g., as you age, the risk goes up even faster)	Time period: Narration uses “over their lifetime” to indicate the time period for the event to occur Change in the rate of change: Narration: “People are more likely to get colon cancer as they get older” Graph: Slope indicating colorectal cancer risk increases sharply after age 50, stops increasing after age 75
5. Increment-al Benefit	Incremental benefits/risk: Understand the benefit/risk associated with 2 or more strategies (e.g., no screening graphic and screening graphic, depicting mortality benefit).	Changes in the icon array for benefit of screening (with screening, without, deaths with and without) 1.5 of the 3 icons change from red to white to indicate that they are saved from dying by being screened Text on the bottom of the screen: 1–2 out of 100 people will die from colon cancer 1–2 out of 100 people will be saved from dying from colon cancer Narration emphasizes how many people are saved from colon cancer by screening
6. Rare Events	Magnitude of rare events: Understand how common a rare event is within the content of the at-risk population Fractions of a person: How to handle an event that occurs as a fraction of a person, such as 1.5 people will…..?	Fractions: Narration uses “number of deaths due to colon cancer will be cut in half” 1.5 of the 3 icons change from red to white to indicate that they are saved from dying by being screened
7. Uncertainty with Risk Estimate*	Uncertainty around point estimates: Understand that probability is an estimate involving a degree of imprecision Issues for research: Understand what the uncertainty means in terms of preferences and decisions.

^*Note: Understand uncertainty with risk estimate (7) was not included.