Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Jul 1.
Published in final edited form as: J Health Commun. 2010 Jul;15(5):487–501. doi: 10.1080/10810730.2010.492560

The Effect of Format on Parents’ Understanding of the Risks and Benefits of Clinical Research: A Comparison between Text, Tables, and Graphics

ALAN R TAIT 1, TERRI VOEPEL-LEWIS 2, BRIAN J ZIKMUND-FISHER 3, ANGELA FAGERLIN 4
PMCID: PMC2915576  NIHMSID: NIHMS122453  PMID: 20677054

Abstract

There is a paucity of information regarding the optimal method of presenting risk/benefit information to parents of pediatric research subjects. This study, therefore, was designed to examine the effect of different message formats on parents’ understanding of research risks and benefits. 4,685 parents completed an Internet-administered survey in which they were randomized to receive risk/benefit information about a study of pediatric postoperative pain control presented in different message formats (text, tables, and pictographs). Survey questions assessed participants’ gist and verbatim understanding of the information and their perceptions of the risks and benefits. Pictographs were associated with significantly (P<0.05) greater likelihood of adequate gist and verbatim understanding compared with text and tables regardless of the participants’ numeracy. Parents who received the information in pictograph format perceived the risks to be lower and the benefits to be higher compared to the other formats (P<0.001). Furthermore, compared with text and tables, pictographs were perceived as more “effective,” “helpful,” and “trustworthy” in presenting risk/benefit information. These results underscore the difficulties associated with presenting risk/benefit information for clinical research but suggest a simple method for enhancing parents’ informed understanding of the relevant statistics.

Introduction

The disclosure of the risks and benefits of a study is perhaps the most important determinant of a subject’s consent to participate in a research study (Susman, Dorn, & Fletcher, 1992; Tait, Voepel-Lewis, Robinson, & Malviya, 2002). Yet, despite this, there is a paucity of data that address the presentation of these elements to research participants, particularly parents and children. Although a few studies have examined different strategies for the presentation of risks and benefits for treatment decisions or diagnoses (Paling, 2003; Schapira, Nattinger, & McHorney, 2001; Slaytor & Ward, 1998; Woloshin & Schwartz, 1999), these have often focused on the presentation of single risk rather than comparative risk statistics (Edwards, Elwyn, & Mulley, 2002; Lipkus & Hollands, 1999). Although single risk/benefit assessment is important, decisions to participate in a research study typically require an understanding of the comparative differences in risks and benefits between two or more experimental groups. However, despite this, there appears to be no consensus regarding the optimal method of framing comparative risk/benefit information for research (Forrow, Taylor, & Arnold, 1992; Malenka, Baron, Johansen, Wahrenberger, & Ross, 1993; O’Connor, 1989). Relative risk is sometimes used as a means to provide information on the magnitude of a change in risk, however, it does not provide a reference by which to place that change in context. Recently, Zikmund-Fisher et al. showed that women who viewed an incremental risk presentation regarding the use of tamoxifen for breast cancer reported significantly less worry and greater understanding about the risk compared to women who were given absolute risk information (Zikmund-Fisher, Fagerlin, Roberts, Derry, & Ubel, 2008). These authors suggest that this may occur because incremental risk helps focus attention on the actual change in risk by providing information regarding both the baseline risk and the change.

Traditionally, risks and benefits are presented either verbally e.g., the risk of headache is “low”, or numerically e.g., the risk of headache is 1% or 1 in 100. These methods, however, have several drawbacks (Burkell, 2004). Interpretation of verbal descriptors of risk/benefit, for example, is often confused by difficulties in the translation of terms such as “low,” “moderate,” or “high” into numerical values (Burkell, 2004; Theil, 2002). In addition, low levels of numeracy (quantitative literacy) which are common among lay individuals may serve to moderate the communication and understanding of basic medical data (Kirsch, Jungelblut, Jenkins, & Kalstad, 1993; Lipkus, Samsa, & Rimer, 2001; Schwartz, Woloshin, Black, & Welch, 1997; Zikmund-Fisher, Smith, Ubel, & Fagerlin, 2007).

There is a long history of research in psychology and education that suggests that presenting information in multiple formats increases understanding. For example, dual coding theory argues that people process visual and verbal information differently and create separate representations for information presented in each way (Paivio, 1986). Obviously, for this to occur, the information presented verbally and visually must be similar and not in conflict with one another. When verbal and visual information is congruent, the parallel learning processes reinforce each other, leading to better memory of the information as well as greater integration with existing knowledge. Consistent with this work, several studies have suggested that graphical presentation (e.g., histograms, pictographs, etc.) of the risks and benefit of treatment may be helpful in promoting understanding of these important elements (Brundage et al., 2005; Burkell, 2004; Hawley et al., 2008; Mazur & Merz, 1993; Price, Cameron, & Butow, 2007; Schapira, Nattinger, & McAuliffe, 2006; Timmermans, Molewijk, Stiggelbout, & Kievit, 2004). Indeed, compared to standard text, visual presentations of risks and benefits have been shown to reveal patterns that otherwise may go undetected (Lipkus & Hollands, 1999), better attract and hold an individual’s attention, and improve the processing of mathematical operations (Hollands & Spence, 1998; Simkin & Hastie, 1987). In a recent study, Hawley et al. surveyed subjects to assess their understanding and perceptions of six different graphical formats (i.e., bar graph, table, pictograph, sparkplug, pie chart, and clock graph) for the presentation of information related to the treatment of cardiac disease. Results showed that pictographs resulted in more accurate gist (overall impression) and verbatim (actual numerical) knowledge compared to other graphical formats, however there were no comparisons of these graphical formats with standard numerical text (Hawley et al., 2008). The goal of the current study was to test whether presenting information both verbally and visually improves processing, and thereby knowledge.

Increasing the visual salience of statistical information, however, could also have a detrimental effect. Specifically, it could cause individuals to ignore important differences in the severity of the side-effects. For instance, a drug can cause a relatively large increase in minor side-effects (e.g., itching), while causing a smaller increase in a more serious side-effect (e.g., slowed breathing). If graphical or tabular presentations focus the individuals’ attention on the statistical information so much that they ignore severity differences, they may make decisions that are inconsistent with their true preference. A possible, but untested way to compensate for such effects would be to provide a second type of visual display to augment the visual salience of the severity of the risks and benefits of treatment. However, little, if any, research has tested methods for communicating the severity of the risks and benefits of treatment or research.

To our knowledge there are no data regarding the effect of graphical presentations on parents’ understanding of the risks and benefits of their children’s participation in research. This study, therefore, was designed to test the hypothesis that graphical presentation of research risks and benefits results in greater parental gist and verbatim understanding compared to standard textual presentation. Because information presented as pictographs is represented both visually and verbally, we hypothesized that parents who receive pictographs will demonstrate increased ability to understand and use numerical the information. Furthermore, we are also testing one possible visual approach for clarifying the differences in the severity of the relevant health conditions. If our hypothesis that pictographs increase the visual salience of probabilistic information is correct, then inclusion of a visual severity scale may prove to be beneficial by balancing the salience of both probability and severity information.

Methods

Participants

This study was deemed exempt by the University of Michigan’s Institutional Review Board. The study sample consisted of parents drawn from a large panel of individuals administered by Survey Sampling International (SSI, Fairfield CT) who had previously agreed to participate in Internet surveys. SSI was responsible for contacting eligible subjects via e-mail to preserve participant anonymity. SSI also performed a stratified random sampling process to offset differential response rates among various demographic subgroups, thereby providing a demographically diverse sample pool. Eligible subjects included parents or legal guardians aged 25–55 years who had at least one child under the age of 18 years. Subjects who were not parents were screened out before beginning the survey. Subjects were provided with a $3 incentive for completion of the survey and were subsequently entered into an SSI-run lottery with cash prizes.

Survey

The Internet survey was designed to determine the effect of different message formats on parents’ gist and verbatim understanding of the risks and benefits of a hypothetical research study comparing two drugs for postoperative pain in children. Participants read a scenario wherein they were asked to imagine themselves in a hospital waiting room where their child was about to undergo a minor but painful surgical procedure. While waiting for their child’s operation, they are told that a researcher approaches them and tells them about a randomized study to compare two drugs that can be used to control their child’s pain after surgery. Drug A was described as the standard drug for postoperative pain control in children, while Drug B was described as a new drug which had proven effective for pain relief in adults but had not yet been tested in children. Both drugs were noted to have the same side-effects; one of which occurred quite frequently but was considered mild (itching) and the other which was much less common but potentially serious (slowed breathing). We should note that although this represented a hypothetical study, the scenario is consistent with many anesthesia-related studies wherein consent for research is typically obtained on the day of surgery.

We used a fully randomized between-subjects factorial design to test the effects of 3 different risk/benefit message formats (text, tables, or pictographs) and the presence or absence of a risk severity graphic. Programmed randomization of subjects into groups occurred automatically on entry to the survey.

a) Risk/Benefit Communication Formats

Risks (itching and slowed breathing) and benefit (pain relief) were presented using text, tables, or pictographs. In the text format, the risks and benefit were described as the number of children out of 100 (%) experiencing the outcome e.g., “60 out of 100 (60%) children who take Drug A will experience good pain relief after surgery.” In the table format the same information was presented in a table (table 1). The pictograph format included a matrix of 100 rectangular blocks representing the reference population of 100 children. The blocks were colored to represent the presence (blue) or absence (grey) of the risk or benefit (figure 1). For those randomized to receive the pictographs, an explanation of pictographs, as well as examples of the pictograph, were provided in the survey’s introduction to familiarize the subjects with the concepts.

Table 1.

Example of the Table format showing absolute values

Outcome Children Who Take Drug A Children Who Take Drug B
Will experience good pain relief after surgery 60 out of 100 (60%) 55 out of 100 (55%)
Will experience itching 25 out of 100 (25%) 20 out of 100 (25%)
Will experience slowed breathing 7 out of 100 (7%) 5 out of 100 (5%)
Open in a new tab
Figure 1.

Figure 1

Open in a new tab

Example of the Pictograph format for depicting benefit (pain relief)

b) Comparing Risks and Benefit between Drugs A and B

For all of the risks and benefits associated with Drug A, our text, table, or pictograph showed the absolute risk of occurrence, i.e., the total number of children out of 100 who would experience each outcome. However, to clarify the increase or decrease in risk or benefit associated with choosing Drug B instead of Drug A, we presented the risk/benefit information for Drug B as the incremental risk increase or decrease, rather than the total risk of occurrence. This novel approach is consistent with previous work (Zikmund-Fisher et al., 2008) that tested incremental risk formats for communicating medication side effect risks. For example, if the risk was lower for Drug B, the text, table, or pictograph noted how many fewer children would experience the outcome (e.g., “compared to children who take Drug A, 5 fewer children out of 100 (5%) who take Drug B will experience itching.” An example of the table and pictograph formats for presenting incremental risk and benefit changes is shown in table 2 and figure 2, respectively. In figure 2, the reference outcome for Drug A (60% pain relief) is shown simultaneously with the decrease in the frequency of pain relief (5%) when using Drug B. This incremental change in risk is highlighted by the 5 grey blocks with blue borders.

Table 2.

Example of the Table format showing incremental change

Outcome Compared to Children Who Take Drug A, Children Who Take Drug B
Will experience good pain relief after surgery 5 fewer children out of 100 (5%)
Will experience itching 5 fewer children out of 100 (5%)
Will experience slowed breathing 2 fewer children out of 100 (2%)
Open in a new tab
Figure 2.

Figure 2

Open in a new tab

Example of the Pictograph format for depicting incremental change in pain relief between drugs A and B

c) Risk Severity Graphic

In addition to information on the frequency of a particular risk or side-effect, approximately half of all participants were randomized to receive a graphic depicting the severity of that risk. Increasing risk was represented by blocks of different sizes and colors (yellow through red). A block with a grey outline and arrow depicted where the risk or side-effect fell on the severity scale (figure 3). This was accompanied by a description of whether the risk was considered “mild” (itching), “moderate” (pain), or “severe” (slowed breathing). For those randomized to receive the risk severity graphics, an explanation of the graphic, as well as examples of the graphic, were provided in the survey’s introduction to familiarize the subjects with the concepts.

Figure 3.

Figure 3

Open in a new tab

Example of the risk severity graphic for pain

Outcome measures

a) Verbatim Understanding

Verbatim understanding or knowledge was defined as the ability to correctly report the actual risk and benefit frequencies of drugs A and B. Seven questions related to verbatim understanding were included i.e., “If 100 children were randomized to take drug B, approximately how many would experience 1) pain; 2) itching, or; 3) slowed breathing after surgery”? Additionally, subjects were asked “Compared to children who take Drug A, approximately how many fewer or more children would experience 4) good pain relief; 5) pain; 6) itching, or; 7) slowed breathing if they took Drug B”? The order of some of these items was changed to avoid an ordering bias. Each response was coded as 0 (incorrect) or 1 (correct) and a total verbatim score (0–7) recorded. Adequate verbatim knowledge was defined as ≥ 5 correct answers out of 7. This definition of “adequate” was based on a previous study by Hawley et al. using similar methodology that examined gist and verbatim understanding of information related to cardiac disease (Hawley et al., 2008). The rationale was that since 100% understanding is neither likely nor absolutely necessary for informed decision-making, a specified proportion of correct responses may be a more practical measure of understanding. Furthermore, a sensitivity analysis (not reported here) found qualitatively similar results using either a continuous measure of knowledge or a 6 out of 7 threshold. We only report results using the ≥ 5 correct-answer threshold for brevity.

b) Gist Understanding

Gist understanding was defined as the ability to identify the essential meaning about the observed differences between the risks and benefits of Drugs A and B. Four questions related to gist knowledge were included: 1) “Who is less likely to experience pain after surgery: A child who was randomized to receive Drug A or a child who was randomized to receive Drug B”?; 2) “Who is more likely to experience itching, or; 3) slowed breathing: A child who was randomized to receive Drug A or a child who was randomized to receive Drug B”? Finally, subjects were asked: “If a child was randomized to Drug B, which of the following is most likely: a) experiencing pain after surgery, b) experiencing itching, or c) experiencing slowed breathing”? A total gist score was determined based on the number of “correct” (i.e, understood the intended meaning) responses (0–4). Adequate gist knowledge was again based on the work of Hawley et al. and was defined as ≥ 3 correct answers out of a possible 4 (Hawley et al., 2008).

Consistent with normal practice wherein subjects are able to browse the information in a consent document, subjects were able to review the risk/benefit information as they responded to the gist and verbatim questions. A summary screen of all the information provided in previous screens was also provided.

c) Perceptions of the Risks and Benefits of Drugs A and B

Parents were given information about the risks and benefit of each drug followed by several questions to elicit their perceptions of the risks and benefits. For example parents were asked “how likely would your child be to experience pain after surgery?” “how worried would you be about your child experiencing pain after surgery?” and “how effective would Drug A (B) be at reducing your child’s chance of experiencing pain after surgery?”

Similar questions were asked regarding the participants’ perceptions of the likely frequency and severity of the side-effects. Parents scored their responses using 1–11 interval scales where 1 equaled the minimum response (e.g., “not at all likely”) and 11 equaled the maximum response (e.g., “extremely likely,” “extremely worried” etc.).

d) Perceptions of the Risk/Benefit Communication Format

In addition to an assessment of understanding, subjects were asked questions related to their perceptions of the method of message delivery to which they were randomized. For example, subjects were asked their opinion regarding how “effective,” “helpful,” “scientific,” “trustworthy,” and “easy,” they thought their specific method of message delivery (i.e., numbers and percentages, tables, or pictographs) was in making sure that they correctly understood the information about the risks and benefit. Responses were scored using 1–7 interval scales where 1 equaled the minimum response (e.g., “extremely difficult”) and 7 equaled the maximum response (e.g., “extremely easy,” “extremely helpful” etc.). Furthermore, in an attempt to provide some sense of the effect of format on decision-making, parents were asked how likely they would have been to enroll their child in the study had it been real. Responses were scored on a 1–11 interval scale where 1 = “not at all likely” and 11 = “extremely likely.”

f) Statistical Analysis

Statistical analyses were performed using SPSS® statistical software (v 16.0, SPSS Inc., Chicago, IL). Sample size determination was based on a previous study which showed that approximately 64% of parents had good understanding of the risks and benefits of studies presented using traditional verbal and numerical formats (Tait, Voepel-Lewis, & Malviya, 2003a). Based on this information and data from another study which used similar methodology (Hawley et al., 2008), we believed that the ability to detect a 20% increase in understanding when using graphical formats would be important to detect. Based on this we required a minimum sample size of approximately 4,800 (α = 0.05, β = 0.20, two-sided).

Comparisons of parametric data between groups were analyzed using analysis of variance with adequate gist or verbatim understanding as the dependent variables. Post-hoc analysis was performed using either the Tukey or Dunnett’s C tests depending on the equality of variances. Nonparametric data were analyzed using Mann Whitney-U, chi-square, and Fisher’s Exact test, as appropriate.

Results

a) Study Population

A total of 6,686 subjects accessed the Internet site. Of these, 295 were excluded because they did not meet the parent eligibility criterion. Of the remaining 6,381 subjects, 1,696 were excluded because they either did not complete the survey (n = 1,424) or completed it too quickly (n = 272). The rationale for excluding those who appeared to “skim” through the survey was that they were unlikely to be giving the information sufficient thought and consideration. Based on previous experience, we chose exclusion cut-off times of < 90 secs for completion of 6 of the core pages or < 300 secs for completion of the entire survey. Data are therefore presented for a total of 4,685 subjects. There was a trend towards higher non-completion rates among the less educated, the elderly, and female participants, however, these trends did not achieve statistical significance. Similarly, there were no differences in non-completion by race or ethnicity.

The demographics of the population are described in table 3. There were no differences in demographics between the three risk/benefit communication formats. However, subjects in the pictograph group had significantly (P = 0.01) greater subjective numeracy scores (Fagerlin et al., 2007; Zikmund-Fisher et al., 2007) compared with those in the text or table groups.

Table 3.

Demographics by Communication Message Format

Text (n = 1548) Tables (n = 1591) Pictographs (n = 1546) All (n = 4685)
Age (yrs, mean ± SD) 39.1 ± 7.9 39.3 ± 7.8 39.0 ± 39.2 ± 7.9
Gender (F/M)% 58.0/42.0 57.7/42.3 56.7/43.3 57.5/42.5
Race/ethnicity (%):
 Caucasian 68.4 67.6 68.0 68.0
 African American 12.8 11.4 11.8 12.0
 Hispanic 11.8 13.2 13.1 12.7
 Asian 4.0 3.7 4.6 4.1
 Other 3.1 4.1 2.5 3.2
Level of Education (%):
 ≤ High school graduate 18.6 19.3 19.1 18.9
 Some college/trade school 33.9 32.0 31.6 32.5
 Associate/Bachelor’s Degree 36.9 37.7 38.6 37.8
 Graduate Degree 10.5 10.9 10.7 10.7
Income Level (%):
 < $10,000 2.2 2.4 1.5 2.1
 $10,000–49,999 36.0 35.6 34.1 35.2
 $50,000–89,999 40.1 38.3 44.9 40.1
 ≥ $90,000 21.7 23.8 22.5 22.7
Numeracy: Median/range 36/40* 36/39* 36/35 36/40
 High/Low 50.6/49.4* 50.7/49.3* 55.5/44.5 52.3/47.7
Open in a new tab
*

P = 0.01 vs Pictograph

Low numeracy = 0–35, High numeracy = 36–48 on the Subjective Numeracy Scale (Fagerlin et al., 2007; Zikmund-Fisher et al., 2007)

b) Effect of Format on Parents’ Gist and Verbatim Understanding

Table 4 compares the effect of format on parents’ understanding of the risks and benefits of the study. Results show that pictographs were significantly better than tables and text in providing both adequate gist and verbatim understanding. However, given that higher subjective numeracy scores were observed in subjects randomized to the pictograph group, we also compared numeracy stratum-specific gist and verbatim understanding between groups as a means to control for the potential confounding effect of numeracy. Results show that individuals with high subjective numeracy had significantly greater adequate gist and verbatim understanding than those with low numeracy. Furthermore, independent of the level of numeracy, pictographs resulted in significantly greater gist and verbatim knowledge compared to text and tables. As an aside, we should note that changing the definition of “adequate” e.g., increasing the required number of correct responses by 1 or 2 had no effect on the patterns of responses. There was a low to moderate correlation between gist and verbatim understanding (r = 0.4) which suggests that these questions were assessing related, yet distinct, components of parental knowledge.

Table 4.

Adequate Gist and Verbatim Understanding by Communication Message Format and Numeracy (%)

Text Tables Pictographs
‡ Adequate Gist Understanding 61.3* 62.9* 66.4
 By Numeracy Level:
  Low 52.4* 57.5 60.7
  High 69.8† 68.3† 71.0†
§ Adequate Verbatim Knowledge 49.1* 44.6* 66.5
 By Numeracy Level:
  Low 35.1* 31.0* 57.6
  High 62.7*† 57.6*† 74.1†
Open in a new tab
*

P< 0.05 vs Pictographs

†

P< 0.01 vs Low numeracy,

Low numeracy = 0–35, High numeracy = 36–48 on the Subjective Numeracy Scale

‡

Adequate gist knowledge = ≥3 correct answers out of 4

§

Adequate verbatim knowledge = ≥5 correct answers out of 7

The interactions between format and other variables such as race/ethnicity (non-Caucasian vs Caucasian) and education (< Bachelor’s degree vs ≥ Bachelor’s degree) were also examined. Pictographs had no effect on gist understanding by race/ethnicity but resulted in significantly greater verbatim understanding among both Caucasians and non-Caucasians compared to text and tables (P < 0.001). Similarly, pictographs were no better than text and tables at providing adequate gist understanding among individuals with different education levels but were significantly (P < 0.001) superior to text and tables in providing adequate verbatim knowledge among these groups.

c) Effect of Format on Parents’ Perceptions of the Risks and Benefit of Drugs A and B

Subjects who received the pictographs were less likely to believe that their child would experience pain or other side effects and therefore were less worried compared to parents who received information using text or tables (Table 5). In addition, subjects who were randomized to pictographs perceived the magnitude of the risks to be lower (5.5 vs 6.0 and 6.1, P< 0.001 on a 1–11 scale, where 11 = “extreme risk”) and the benefits to be higher (6.5 vs 6.1 and 6.1, P < 0.001, 1–11 scale) than those randomized to the text or table groups, respectively.

Table 5.

Perceptions of the Risks and Benefits of Drug B by Communication Message Format

“If your child were randomized to receive Drug B … Text Tables Pictographs
…how likely would your child be to experience pain after surgery?” 6.3 ± 2.6 6.4 ± 2.6* 6.0 ± 2.4
…how likely would your child be to experience at least one side effect?” 5.2 ± 2.7* 5.2 ± 2.7* 4.6 ± 2.5
…how worried would you be about your child experiencing pain?” 7.4 ± 2.8* 7.3 ± 2.8* 7.0 ± 2.7
…how worried would you be about your child experiencing at least one side- effect?” 6.1 ± 2.9* 6.1 ± 3.0* 5.5 ± 3.0
…how serious do you think it would be if your child experienced slowed breathing?” 7.9 ± 3.0* 7.7 ± 3.1* 7.2 ± 3.3
…how effective would drug B at reducing your child’s chance of experiencing pain?” 6.5 ± 2.4* 6.3 ± 2.5* 6.7 ± 2.1
Open in a new tab

Data are presented a mean ± SD based on a scale of 1–11 (where 11 = maximum response e.g., extremely worried, likely, etc.)

*

P< 0.025 vs Pictographs

Subjects randomized to the pictograph format also reported that they would be more likely to consent to their child’s enrollment in the study (had it been real) than those receiving information via text or tables (6.6 vs 6.0 and 6.0, respectively, P< 0.001, 1–11 scale where 11 = “extremely likely”).

d) Parents’ Perceptions of the Risk/Benefit Communication Format

The perceptions of the different message formats are shown in table 6. Results show that pictographs were deemed to be significantly more “effective,” “helpful,” “trustworthy” and, “scientific” than text or tables in describing the risks and benefits of the two drugs. In addition, pictographs were shown to more clearly represent changes in the risk of pain than the other formats respectively (5.6 vs 5.0 and 4.9, P< 0.001, 1–7 scale where 7 = “extremely clear”).

Table 6.

Perceptions of the Communication Message Format

Text Tables Pictographs
Effective 5.2 ± 1.5* 5.2 ± 1.5* 5.8 ± 1.4
Helpful 5.3 ± 1.5* 5.3 ± 1.5* 5.8 ± 1.4
Scientific 5.0 ± 1.5*† 4.7 ± 1.5* 5.1 ± 1.5
Trustworthy 4.7 ± 1.5* 4.8 ± 1.4* 5.2 ± 1.3
Open in a new tab

Data are presented a mean ± SD based on a scale of 1–7 (where 7 = “extremely”)

*

P< 0.001 vs Pictographs,

†

P< 0.001 vs Tables

The use of risk severity graphics was also examined. Results showed that while the presence of severity graphics had no effect on verbatim understanding, adequate gist understanding was less among subjects who received information using the risk severity graphics compared to those who did not (60.9% vs 65.3%, respectively, P = 0.002). In addition, subjects who received severity graphics perceived the risks to be higher (6.0 vs 5.7, P< 0.01 on a 1–11 scale, where 11 = “extreme risk”) and the benefits lower (6.1 vs 6.3, P = 0.015 on a 1–11 scale, where 11 = “extreme benefit”) than those who did not receive them. Furthermore, subjects who received risk severity graphics were less likely to report that they would consent to their child’s enrollment in the study (had it been real) than those who did not receive them (6.0 vs 6.4, P< 0.001, 1–11 scale where 11 = “extremely likely”).

Discussion

The primary goal of this study was to compare the use of standard numerical text with graphical formats for communicating incremental differences in the research risks and benefits between two drugs. Results showed that pictographs were significantly superior to text and tables in providing adequate parental gist and verbatim understanding of research information. These findings are similar to those of Hawley et al. who found that pictographs promoted better gist and verbatim understanding compared to other graphical message formats, particularly among innumerate individuals (Hawley et al., 2008). However, unlike our study, these authors found that tables resulted in better verbatim understanding compared to pictographs whereas we found that text and tables resulted in similar levels of gist and verbatim understanding. The observed equivalence of text and tables was somewhat surprising given the apparent visual salience of tables. However, while this may indeed be a true finding, it may also simply be a function of the design of our tables and/or the inability of the subjects to sufficiently discriminate between the information presented in our text and table formats.

Of interest, was that parents who received information using pictographs perceived the risks to be lower and the benefits to be higher when compared to those receiving table or text information. Similar findings were reported by Schapira et al. who showed that numerically identical breast cancer risks were perceived differently depending on the graphical format (Schapira et al., 2006). In a study by Timmermans et al. surgical risk information presented as numbers and pictographs (icons) was perceived to be less complex and less threatening compared to information presented as bar graphs (Timmermans et al., 2004). Stone et al. suggests that this may be because the visual salience of risks is higher with certain message formats leading to differences in perceived threat (Stone, Yates, & Parker, 1997). In concert with these observations, parents in our study who received risk/benefit information using pictographs also stated that they would be more likely to have allowed their child to participate in the study had it been real. Although we did not directly ask the reasons for the subjects’ (hypothetical) decisions, this may have been due to the fact that the pictograph group had more positive perceptions of the risks and benefit and greater overall understanding of the information. Indeed, in one study, Waters et al. showed that tradeoff decisions for treatment were better when probabilities were presented graphically compared to text (Waters, Weinstein, Colditz, & Emmons, 2006). Furthermore, we recently showed that parents who had greater understanding of the elements of a study were significantly more likely to consent to their child’s participation compared to parents with lesser understanding (Tait, Voepel-Lewis, & Malviya, 2003b). We should emphasize however, that in developing strategies to optimize the consent process, the primary objective is to improve understanding of the information not simply to increase recruitment. Thus, it likely follows that if graphical formats can improve understanding they may also increase subject enrollment.

The effect of the severity graphics on the perception of risk and gist understanding is interesting and we speculate that they, in part, may serve to raise the emotional salience of the health conditions involved. In doing so, it would remind people of the potential for harm (by making the nature of harm more salient), thus increasing risk perceptions and reducing interest in participation. The reduction in perceived benefits is likely a contrast effect: by making pain only a moderate severity issue it now looks less important vs. the (newly salient) risk of breathing problems. Clearly, more research is required to examine this observation in more detail.

Results showed that pictographs were superior in providing adequate gist and verbatim understanding for both numerate and innumerate individuals and suggest that this format may be better for presenting comparative research risk/benefit information and in aiding decision-making for individuals with differing levels of numeracy. Other studies support these findings suggesting that pictographs are easier to understand, require less cognitive effort, and are thus more effective in presenting quantitative risk information (Burkell, 2004; Feldman-Stewart, Brundage, & Zotov, 2007; Hawley et al., 2008; Peters, Dieckmann, Dixon, Hibbard, & Mertz, 2007; Price et al., 2007; Waters et al., 2006). Slovic et al. suggest that this may occur because individuals have different affective responses related to the identification and interpretation of a numerator (Slovic, Monahan, & MacGregor, 2000). It may be therefore, that pictographs are better able than text and tables to communicate the relationships conveyed by numerical data.

A few points regarding the limitations of this study deserve mention. Our decision to use an Internet survey was based on the fact that this approach has a number of advantages over paper surveys including the ability to study large diverse populations quickly, versatility in design and interactivity, dynamic error checking, and the ability to rapidly download the responses directly to a database (Rostom, O’Connor, Tugwell, & Wells, 2002). In addition, given the randomized design, a paper survey of this size would be prohibitively expensive and time consuming. However, we also recognize that this approach limits our sample to individuals with Internet access and to those who may be more motivated by virtue of their agreement to participate in health-related surveys. Thus, while this sample may not be completely representative of a research population, we should point out that since the purpose of this study was to compare the impact of message formats across demographically similar experimental groups, the potential for response bias was likely low. We also recognize that in practice, a healthcare professional would be available to help with any questions related to the study and thus the interpretation of each of the formats in our Internet survey may be slightly different from that experienced in “real life.” Further work is therefore warranted.

Although the disclosure of risks and benefits are two of the most important elements required by parents in order to make informed decisions about their child’s participation in a research study, many have difficulty interpreting the relevant numerical data. This study shows that parents may often misinterpret the magnitude of the risks and benefits and these misunderstandings may therefore, negatively influence their decision-making. However, our study also suggests that the understanding and interpretation of research risks and benefits may be enhanced by graphical formatting and, in particular, by the use of pictographs. The results of this study may, therefore, be important in highlighting the difficulties in presenting numerical risk/benefit research information and in suggesting simple alternative methods for enhancing understanding of these critical statistics.

Acknowledgments

Supported in part by a grant to Dr. Tait from The National Institutes of Health, NICHD, (R01 HD053594). Dr. Zikmund-Fisher is supported by a career development award from the American Cancer Society (MRSG-06-130-01-CPPB), and Dr. Fagerlin was supported by an MREP early career award from the U. S. Department of Veteran’s Affairs. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs.

The authors are indebted to Bob Burbach and Aaron Pearlman for their help with the design and development of the Internet survey and the specific graphs and tables tested herein. We also thank Julie Parow, Rosemarie Pitsch, and Nicole Exe for their assistance in testing the survey instrument. Finally, we thank Dr. Peter Ubel for his helpful comments during the design of this study.

Contributor Information

ALAN R. TAIT, Department of Anesthesiology, University of Michigan Health System, Ann Arbor, Michigan, U.S.A

TERRI VOEPEL-LEWIS, Department of Anesthesiology, University of Michigan Health System, Ann Arbor, Michigan, U.S.A.

BRIAN J. ZIKMUND-FISHER, Ann Arbor VA HSR&D Center for Practice Management and Outcomes Research, Ann Arbor, Michigan, U.S.A., Division of General Internal Medicine, University of Michigan, Ann Arbor, Michigan, U.S.A., Center for Behavioral and Decision Sciences in Medicine, University of Michigan, Ann Arbor, Michigan, U.S.A

ANGELA FAGERLIN, Ann Arbor VA HSR&D Center for Practice Management and Outcomes Research, Ann Arbor, Michigan, U.S.A., Division of General Internal Medicine, University of Michigan, Ann Arbor, Michigan, U.S.A., Center for Behavioral and Decision Sciences in Medicine, University of Michigan, Ann Arbor, Michigan, U.S.A.

References

  1. Brundage M, Feldman-Stewart D, Leis A, Bezjak A, Degner L, Velji K, et al. Communicating quality of life information to cancer patients: A study of six presentation formats. J Clin Oncol. 2005;23:6949–6956. doi: 10.1200/JCO.2005.12.514. [DOI] [PubMed] [Google Scholar]
  2. Burkell J. What are the chances? Evaluating risk and benefit information in consumer health materials. J Med Library Assoc. 2004;92:200–208. [PMC free article] [PubMed] [Google Scholar]
  3. Edwards A, Elwyn G, Mulley A. Explaining risks: turning numerical data into meaningful pictures. BMJ. 2002;324:827–830. doi: 10.1136/bmj.324.7341.827. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Fagerlin A, Zikmund-Fisher B, Ubel P, Jankovic A, Derry H, Smith D. Measuring numeracy without a math test: Development of the subjective numeracy scale (SNS) Med Dec Making. 2007;27:672–680. doi: 10.1177/0272989X07304449. [DOI] [PubMed] [Google Scholar]
  5. Feldman-Stewart D, Brundage M, Zotov V. Further insight into the perception of quantitative information: Judgments of gist in treatment decisions. Med Dec Making. 2007;27:34–43. doi: 10.1177/0272989X06297101. [DOI] [PubMed] [Google Scholar]
  6. Forrow L, Taylor W, Arnold R. Absolutely relative: how research results are summarized can affect treatment decisions. Am J Med. 1992;92:121–124. doi: 10.1016/0002-9343(92)90100-p. [DOI] [PubMed] [Google Scholar]
  7. Hawley S, Zikmund-Fisher B, Ubel P, Jancovic M, Lucas T, Fagerlin A. The impact of the format of graphical presentation on health-related knowledge and treatment choices. Patient Educ Counsel. 2008;73:448–455. doi: 10.1016/j.pec.2008.07.023. [DOI] [PubMed] [Google Scholar]
  8. Hollands J, Spence I. Judging proportion with graphs: the summation model. Appl Cogn Psychol. 1998;12:173–190. [Google Scholar]
  9. Kirsch I, Jungelblut A, Jenkins L, Kalstad A. Adult literacy in America: a first look at the results of the adult national adult literacy survey. Washington, DC: National Center for Education Statistics; 1993. [Google Scholar]
  10. Lipkus I, Hollands J. The visual communication of risk. J Nat Cancer Inst Monographs. 1999;25:149–163. doi: 10.1093/oxfordjournals.jncimonographs.a024191. [DOI] [PubMed] [Google Scholar]
  11. Lipkus I, Samsa G, Rimer B. General performance on a numeracy scale among highly educated samples. Med Dec Making. 2001;21:37–44. doi: 10.1177/0272989X0102100105. [DOI] [PubMed] [Google Scholar]
  12. Malenka D, Baron J, Johansen S, Wahrenberger J, Ross J. The framing effect of relative and absolute risk. J Gen Intern Med. 1993;8:543–548. doi: 10.1007/BF02599636. [DOI] [PubMed] [Google Scholar]
  13. Mazur D, Merz J. How the manner of presentation of data influences older patients in determining their treatment preferences. J Am Geriatr Soc. 1993;41:223–228. doi: 10.1111/j.1532-5415.1993.tb06696.x. [DOI] [PubMed] [Google Scholar]
  14. O’Connor A. Effects of framing and level of probability on patients’ preferences for cancer chemotherapy. J Clin Epidemiol. 1989;42:119–126. doi: 10.1016/0895-4356(89)90085-1. [DOI] [PubMed] [Google Scholar]
  15. Paivio A. Mental representations: a dual coding approach. Oxford, England: Oxford University Press; 1986. [Google Scholar]
  16. Paling J. Strategies to help patients understand risks. BMJ. 2003;327:745–748. doi: 10.1136/bmj.327.7417.745. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Peters E, Dieckmann N, Dixon A, Hibbard J, Mertz C. Less is more in presenting quality information to consumers. Med Care Res Rev. 2007;64:169–190. doi: 10.1177/10775587070640020301. [DOI] [PubMed] [Google Scholar]
  18. Price M, Cameron R, Butow P. Communicating risk information: The influence of graphical display format on quantitative information perception-accuracy, comprehension and preferences. Patient Educ and Counsel. 2007;69:121–128. doi: 10.1016/j.pec.2007.08.006. [DOI] [PubMed] [Google Scholar]
  19. Rostom A, O’Connor A, Tugwell P, Wells G. A randomized trial of a computerized versus and audio-booklet decision aid for women considering post-menopausal hormone replacement therapy. Patient Educ and Counsel. 2002;46:67–74. doi: 10.1016/s0738-3991(01)00167-7. [DOI] [PubMed] [Google Scholar]
  20. Schapira M, Nattinger A, McAuliffe T. The influence of graphic format on breast cancer risk communication. J Hlth Commun. 2006;11:569–582. doi: 10.1080/10810730600829916. [DOI] [PubMed] [Google Scholar]
  21. Schapira M, Nattinger A, McHorney C. Frequency or probability? A qualitative study of risk communication formats used in health care. Med Decis Making. 2001;21:459–467. doi: 10.1177/0272989X0102100604. [DOI] [PubMed] [Google Scholar]
  22. Schwartz L, Woloshin S, Black W, Welch H. The role of numeracy in understanding the benefit of screening mammography. Ann Intern Med. 1997;127:966–972. doi: 10.7326/0003-4819-127-11-199712010-00003. [DOI] [PubMed] [Google Scholar]
  23. Simkin D, Hastie R. An information processing analysis of graph perception. J Am Stat Assoc. 1987;82:454–465. [Google Scholar]
  24. Slaytor E, Ward J. How risks of breast cancer and benefits of screening are communicated to women: analysis of 58 pamphlets. BMJ. 1998;317:263–265. doi: 10.1136/bmj.317.7153.263. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Slovic P, Monahan J, MacGregor D. Violence risk assessment and risk communication: The effects of using actual cases, providing instruction, and employing probability versus frequency benefits. Law Hum Behav. 2000;24:271–296. doi: 10.1023/a:1005595519944. [DOI] [PubMed] [Google Scholar]
  26. Stone E, Yates J, Parker A. Effects of numerical and graphical displays on professional risk-taking behavior. J Exp Psych: Applied. 1997;3:243–256. [Google Scholar]
  27. Susman E, Dorn L, Fletcher J. Participation in biomedical research: The consent process as viewed by children, adolescents, young adults, and physicians. J Pediatr. 1992;121:547–552. doi: 10.1016/s0022-3476(05)81142-4. [DOI] [PubMed] [Google Scholar]
  28. Tait AR, Voepel-Lewis T, Malviya S. Do they understand? (Part I): Parental consent for children participating in clinical anesthesia and surgery research. Anesthesiology. 2003a;98:603–608. doi: 10.1097/00000542-200303000-00005. [DOI] [PubMed] [Google Scholar]
  29. Tait AR, Voepel-Lewis T, Malviya S. Participation of children in clinical research: Factors that influence a parent’s decision to consent. Anesthesiology. 2003b;99:819–825. doi: 10.1097/00000542-200310000-00012. [DOI] [PubMed] [Google Scholar]
  30. Tait AR, Voepel-Lewis T, Robinson A, Malviya S. Priorities for disclosure of the elements of informed consent for research: a comparison between parents and investigators. Paediatr Anaesth. 2002;12:332–336. doi: 10.1046/j.1460-9592.2002.00851.x. [DOI] [PubMed] [Google Scholar]
  31. Theil M. The role of translations of verbal into numerical probability expressions in risk management meta-analysis. J Risk Res. 2002;5:177–186. [Google Scholar]
  32. Timmermans D, Molewijk B, Stiggelbout A, Kievit J. Different formats for communicating surgical risks to patients and the effect on choice of treatment. Patient Educ & Counsel. 2004;54:255–263. doi: 10.1016/S0738-3991(03)00238-6. [DOI] [PubMed] [Google Scholar]
  33. Waters E, Weinstein N, Colditz G, Emmons K. Formats for improving risk communication in medical tradeoff decision. J Hlth Comm. 2006;11:167–182. doi: 10.1080/10810730500526695. [DOI] [PubMed] [Google Scholar]
  34. Woloshin S, Schwartz L. How can we help people make sense of medical data. Effective Clin Pract. 1999;2:176–183. [PubMed] [Google Scholar]
  35. Zikmund-Fisher B, Fagerlin A, Roberts T, Derry H, Ubel P. Alternate methods of framing information about medication side effects: Incremental risk versus total risk difference. J Hlth Comm. 2008;13:107–124. doi: 10.1080/10810730701854011. [DOI] [PubMed] [Google Scholar]
  36. Zikmund-Fisher B, Smith D, Ubel P, Fagerlin A. A validation of the subjective numeracy scale: effects of low numeracy on comprehension of risk communication and utility elicitations. Med Dec Making. 2007;27:663–671. doi: 10.1177/0272989X07303824. [DOI] [PubMed] [Google Scholar]

RESOURCES