Presenting Research Risks and Benefits to Parents: Does Format Matter?

Abstract

Background/Aim

Several studies suggest that many parents and research participants have poor understanding of the elements of consent, particularly the risks and benefits. However, some data suggest that the format and framing of research risks and benefits may be an important determinant of subject understanding. We examined the effect of tabular and graphical presentation of risks and benefits on parents’ understanding of a research study.

Methods/Materials

Parents of children scheduled to undergo an elective surgical procedure (N=408) were randomized to receive information about the risks and benefits of a sham study of postoperative pain control using text, tables, or pictographs and then completed a questionnaire to examine their gist (essential) and verbatim (actual) understanding of the information. Parent demographics were recorded and their literacy and numeracy skills measured.

Results

Parents randomized to receive information using tables or pictographs had significantly (P<0.025) greater gist and verbatim understanding compared with parents who received the information using standard text. Tables and pictographs were also superior to text in promoting understanding among parents with low numeracy and literacy skills.

Conclusions

Many parents and patients have difficulty in assimilating and interpreting risk/benefit information for both research and treatment. This is due, in part, to the manner in which risks and benefits are communicated and to the literacy and numeracy abilities of the individual. The results of this study suggest a simple and practical method for enhancing understanding of risk/benefit statistics for parents with varying numeracy and literacy skills.

Introduction

The section of the Federal Regulations for the Protection of Human Subjects related to informed consent (21 CFR 50.25) stipulates that investigators provide descriptions of “any reasonable foreseeable risks or discomforts” and “any benefits to the subjects or to others which may be reasonably expected from the research” (1). Despite this directive, studies suggest that many adult subjects and parents of child subjects have poor understanding of the information provided (2–4).

There may be several reasons for poor understanding including information written at higher than the recommended 8^th grade reading level, use of unfamiliar medical terms, and incomplete or unclear information (4–6). Another potential impediment is that consent for anesthesia research is typically presented on the day of surgery with only a narrow window of opportunity for disclosure. Tait et al. identified several factors that influence parents’ assessments of the risks and benefits of a study including their perceptions of the clarity and amount of information provided (7). There is also evidence to suggest that the format and framing of the message used to present risks and benefits may affect subject understanding (8, 9). For example, descriptive presentation of risk (e.g., the risk of nausea is “low”) is often used but may be difficult to translate into a meaningful numerical value. Similarly, using numerical values alone (e.g., the risk of nausea is 10% or 10 of 100) may pose difficulties for individuals with low numeracy skills (8, 9).

Recently, there have been some data to suggest that graphical presentation of risks and benefits may improve understanding (10, 11); however, there is no consensus regarding the best method of presenting these statistics in the clinical and research setting, particularly to individuals with poor numeracy and literacy skills. This study, therefore, was designed to test the hypothesis that graphical presentation of research risks and benefits to parents of children scheduled for elective surgery would result in greater understanding compared with standard text presentations.

Methods

Participants

This study was approved by the University of Michigan’s IRB with a waiver of documentation of informed consent. The study sample included parents (> 18yr) of children scheduled for an elective surgical procedure. Parents were approached in the preoperative screening area or preoperative waiting area and, after a standardized verbal disclosure and consent to participate, were randomized to receive consent information for a sham pediatric study in which risk/benefit information was presented in one of three formats, i.e., text, tables, or pictographs. The choice of pictographs as the primary graphical format was based on previous studies which showed that pictographs were more consistently associated with adequate understanding than other formats such as pie charts, bar graphs, etc. (11, 12). The sham study was based on a real protocol conducted previously in our department comparing a standard drug (Drug A) with an new drug (Drug B) for postoperative pain management in children. Each consent document followed our IRB’s standard template for risk/benefit presentation and contained the same basic information but presented in different formats. Parents were informed that we were evaluating the process of giving study information and the quality of our consent documents. Parents were also told that their child would not be participating in an actual study but that they should consider the hypothetical information as if it were real. Although the study represented a sham protocol, it was nevertheless presented to simulate our standard practice for consenting research participants. In this respect, the timing of when the information was given (i.e., on the day of surgery), the time allotted for parents to read the consent information, and the environment in which consent was sought was similar to our usual practice.

After an explanation of the study and consent to participate, parents were given time to read the consent form as per our routine practice. Parents were asked if they had sufficient time to read the document and, if under real circumstances, whether they would have allowed their child to participate. They were then given a questionnaire to determine their gist (essential) and verbatim (actual) understanding of the risks and benefits of the “study.” The gist and verbatim items in the questionnaire were based on a survey previously conducted in our department (12). Additionally, parents were asked their perceptions of the effectiveness of the presentation and their preferences for the way in which risks and benefits were presented.

Describing and comparing the risks and benefits

Risks (itching, nausea and vomiting, and slowed breathing) and the benefits (pain relief) of the two drugs were presented using text, tables or pictographs. The numbers used to present risks and benefits were based loosely on actual drug data and, were derived to emphasize differences between the “study drugs” for the purposes of measuring understanding. In the text version, risks and benefits were described in absolute terms, i.e., the number of children out of 100 (%) experiencing the outcome. In the table format, the same information was presented as shown in Table 1. The pictograph format included a matrix of 100 figures representing the reference population. The figures were displayed using different colors (blue and grey) to represent the number of individuals with or without the risk or benefit (Figure 1).

Table 1.

Drug A	Drug B
Benefits: 60% (60 out of 100) children will have good pain relief	Benefits: 75% (75 out of 100) children will have good pain relief
Most common side effects: Nausea or vomiting (25% or 25 out of 100) Itching (25% or 25 out of 100)	Most common side effects: Nausea or vomiting (20% or 20 out of 100) Itching (15% or 15 out of 100)
Rare but more serious side effect: Slowed breathing (7% or 7 out of 100)	Rare but more serious side effect: Slowed breathing (5% or 5 out of 100)

Figure 1.

Example of the Pictograph format for depicting benefit (pain relief).

Outcome measures

a) Gist Understanding

Gist refers to the ability of the subject to understand the essential meaning about the differences between the risks and benefits of the two drugs. Five multiple-choice questions were designed to measure gist understanding. For example, parents were asked “Who is more likely to experience nausea/vomiting (itching, slowed breathing, pain relief): 1) a child who received Drug A, 2) a child who received drug B, or 3) they are equally likely.” Gist understanding was measured as the total number of correct responses (0–5).

b) Verbatim Understanding

Verbatim understanding refers to the subjects’ ability to correctly report the absolute risk and benefit frequencies of the two drugs. Seven items in the questionnaire measured verbatim understanding. Examples include: “If 100 children took Drug B, approximately how many would experience pain after surgery?” and “Compared to children who took Drug A, how many fewer children would experience nausea/vomiting if they took Drug B?” Each item required a fill-in-the-blank response and only correct answers were considered. A total verbatim score was based on the number of correct answers (0–7).

c) Individual Characteristics

Socio-demographic data were collected including age (parent and child), gender, racial/ethnic background (self-identified), education, surgical procedure, and prior research participation. In addition, parental numeracy and general literacy were measured using 8 items from the Subjective Numeracy Scale (13, 14) and the shortened version of the Rapid Estimate of Adult Learning in Medicine reading test, respectively (15). Need for cognition which measures the tendency of an individual to engage in and enjoy effortful thinking was measured using the shortened version of the Need for Cognition form developed by Cacciopo et al. (16, 17).

Statistical analysis

Statistical analyses were performed using SPSS^® statistical software (v 16.0, SPSS Inc., Chicago, IL). Sample size determination was based on preliminary data describing parents’ understanding of risks presented in a standard text format. Accepting a 20% difference in understanding between formats as the smallest clinically important difference, we required a minimal sample size of 134/group (α = 0.05, β = 0.10, 2-tailed, N = 402).

Nonparametric data were analyzed using Kruskal-Wallis, Mann Whitney-U, chi-square, and Fisher’s Exact test, as appropriate. Comparisons of parametric data between groups (e.g., age) were analyzed using analysis of variance (ANOVA). Factors determined to be significant by univariate analysis were entered into a multiple regression model to identify predictors of understanding. Data are expressed as percentages, median and interquartile range, and mean ± SD. Significance was accepted as P< 0.05 for single comparisons and P< 0.025 for multiple inter-format group comparisons (Bonferroni corrected).

Results

Four hundred forty-four parents of children scheduled for elective surgery were approached to participate, 36 of whom declined. Data are thus presented for 408 subjects. The demographics of the nonconsenters were similar to those who participated, although a larger percentage of nonconsenters were African American, as compared to those who participated (25% vs 13.4%, P = 0.059). There were no differences in demographics between each of the format groups (Table 2).

Table 2.

Demographics by Message Format

	Text (n = 136)	Tables (n = 136)	Pictographs (n = 136)	All (n = 408)
Age (yrs, mean ± SD)	36.2 ± 9.1	35.3 ± 8.9	36.3 ± 9.1	35.9 ± 8.9
Gender (F/M)%	78.9/21.1	74.6/25.4	69.9/30.1	74.4/25.6
Race/ethnicity (%):
Caucasian	77.4	77.8	77.0	77.4
African American	14.3	13.3	12.6	13.4
Hispanic	4.5	3.0	4.4	4.0
Other	3.8	5.9	5.9	5.2
Level of Education (%):
≤High school graduate	12.8	19.5	19.3	17.2
Some college/trade school	36.8	30.8	29.6	32.4
Bachelor’s Degree	30.1	29.3	33.3	30.9
Graduate Degree	20.3	20.3	17.8	19.5
Income Level (%):
< $10,000	9.2	6.1	10.8	8.7
$10,000–49,999	38.2	44.7	31.5	38.2
$50,000–89,999	31.3	24.2	27.7	27.7
≥$90,000	21.4	25.0	30.0	25.4
Numeracy: Median	38	38	40	39
High/Low	51/49	51/49	46/54	49/51

Data are expressed as % and mean ± SD

Low numeracy = 0–35, High numeracy = 36–48 on the Subjective Numeracy Scale

Tables 3 and 4 compare the effect of format on parents’ gist and verbatim understanding of the risks and benefits posed by the study. As shown, both gist and verbatim understanding were significantly greater among parents who received information using either tables or pictographs, as compared to the standard text presentation. Subgroup analysis also showed that both pictographs and, to a greater extent, tables were superior to text in improving gist and verbatim understanding among those with poor literacy and numeracy skills. Indeed, when presented as text, none of the parents with low literacy could answer all the gist questions correctly and only one could correctly answer all the verbatim questions.

Table 3.

Effect of Message Format on Gist Understanding by Numeracy and Literacy

	Text	Tables	Pictographs
Number of correct gist questions	2.85 ± 1.7 3 (1, 4)	3.62 ± 1.4 4 (3, 5)*	3.70 ± 1.28 4 (3, 5)*
By Numeracy Level:
Low	2.51 ± 1.8 2 (1, 4)	3.54 ± 1.4 4 (3, 4)*	3.28 ± 1.5 4 (3, 4)*
High	3.16 ± 1.7 3.5 (2, 4.25)†	3.77 ± 1.5 4 (4, 5)*	4.02 ± 1.0 4 (4, 5)* †
By Literacy Level:
Low	1.77 ± 1.4 1.5 (0.25, 3)	4.13 ± 0.9 4 (4, 5)*	3.00 ± 1.5 3 (2, 4)
High	2.92 ± 1.7 3 (1, 4)†	3.66 ± 1.4 4 (3, 5)*	3.86 ± 1.2 4 (3, 5)* †

Data expressed as mean ± SD, median (interquartile range)

^*P< 0.025 vs Text

^†P< 0.05 vs Low numeracy or Low literacy

Low numeracy = 0–35, High numeracy = 36–48 on the Subjective Numeracy Scale

Low Literacy = 0 – 60 (3^rd – 8^th grade equivalence), High Literacy = > 61 (9^th grade equivalence or higher)

Table 4.

Effect of Message Format on Verbatim Understanding by Numeracy and Literacy

	Text	Tables	Pictographs
Number of correct verbatim questions (range 0–7)	5.15 ± 2.1 6 (4, 7)	5.87 ± 1.7 7 (5, 7)*	5.50 ± 1.9 6 (5, 7)
By Numeracy Level:
Low	4.69 ± 2.3 6 (3, 7)	5.58 ± 1.80 6 (5, 7)*	4.85 ± 2.2 6 (3, 7)
High	5.84 ± 1.5 6 (5, 7)†	6.29 ± 1.38 7 (6, 7)†	6.07 ± 1.5 7 (6, 7)†
By Literacy Level:
Low	3.22 ± 1.7 3 (2, 4)	6.00 ± 0.9 7 (5.25, 7)*	4.87 ± 2.1 5 (3, 5)
High	5.52 ± 1.8 6 (4.5, 7)†	6.05 ± 1.4 7 (6, 7)*	5.59 ± 1.9 6 (5, 7)

Data expressed as mean ± SD, median (interquartile range)

^*P< 0.025 vs Text

^†P< 0.05 vs Low numeracy or Low literacy

Low numeracy = 0–35, High numeracy = 36–48 on the Subjective Numeracy Scale

Low Literacy = 0 – 60 (3^rd – 8^th grade equivalence), High Literacy = > 61 (9^th grade equivalence or higher)

Table 5 describes the effect of parental demographics and characteristics on gist and verbatim understanding. Results showed that parents with a college degree had better verbatim understanding of the information and those with greater literacy and numeracy skills had significantly better gist and verbatim understanding, as compared to those with lower abilities. Multiple regression models identified several factors predictive of gist and verbatim understanding. Predictors of gist understanding included message format (tables or pictographs vs text, P< 0.001) and high numeracy (P = 0.001). Independent predictors of verbatim understanding included message format (tables vs text, P< 0.001), high numeracy (P< 0.001), and high literacy (P = 0.01).

Table 5.

Effect of Parent Characteristics on Gist and Verbatim Understanding

	Gist (range 0–5)		Verbatim (range 0–7)
Gender:
Female (R)	3.29 ± 1.6	4 (3, 4)	5.49 ± 1.9	6 (5, 7)
Male	3.67 ± 1.5	4 (3,5)*	5.68 ± 1.9	6 (5, 7)
Race/ethnicity (%):
Caucasian (R)	3.41 ± 1.5	4 (3, 5)	5.61 ± 1.8	6 (5, 7)
African American	3.20 ± 1.5	4 (2, 4)	5.20 ± 2.0	6 (4, 7)
Hispanic	3.18 ± 1.7	3.5 (2.5, 4)	5.19 ± 2.1	6 (4.75, 7)
Level of Education (%):
≤High school graduate (R)	3.24 ± 1.5	4 (2.5, 5)	4.79 ± 2.2	5.5 (3.5, 6.5)
Some college/trade school	3.21 ± 1.6	4 (2, 5)	5.32 ± 2.1	6 (5, 7)*
≥Bachelor’s Degree	3.55 ± 1.5	4 (3, 5)	5.98 ± 1.4	6.5 (6, 7)*
Numeracy:
Low (R)	3.11 ± 1.6	4 (2, 4)	5.05 ± 2.1	6 (4, 7)
High	3.67 ± 1.4	4 (3, 5)*	6.07 ± 1.4	7 (6, 7)*
Literacy:
Low (R)	2.93 ± 1.6	3.5 (1.75, 4)	4.69 ± 2.1	5 (3, 7)
High	3.47 ± 1.5	4 (3,5)	5.72 ± 1.7	6 (5, 7)*
NFC:
Low (R)	3.38 ± 1.5	4 (3, 4)	5.69 ± 1.8	6 (5, 7)
High	3.44 ± 1.6	4 (3, 5)	5.49 ± 1.9	6 (5, 7)

Data expressed as mean ± SD and median (interquartile range)

(R) = Reference group,

^*P<0.05 vs Reference group

Low numeracy = 0–35, High numeracy = 36–48 on the Subjective Numeracy Scale

Low Literacy = 0 – 60 (3^rd – 8^th grade equivalence), High Literacy = >61 (9^th grade equivalence or higher)

Low NFC (Need for Cognition) = 0–21, High NFC ≥ 22, cutoff based on median split

Overall, there were no differences between groups with respect to the parents’ perceptions of the information presentation (Table 6), although parents who received tables did find them easier in determining the risks and benefits. There were, however, no differences between the text, table and pictograph groups with respect to how likely parents would have been to allow their child to participate in the study had it been real (5.6 ± 3.2 vs 5.4 ± 3.2 vs 5.6 ± 3.1, respectively, 0–10 scale where 10 = extremely likely).

Table 6.

Parents’ perceptions of the message format

	Text	Tables	Pictographs
Perceived risk of the study	4.18 ± 2.3	4.12 ± 2.2	3.93 ± 2.3
Perceived benefit of the study	6.29 ± 2.4	6.50 ± 2.2	6.41 ± 2.0
Effectiveness in presenting risks and benefits	7.49 ± 2.2	7.90 ± 1.9	7.61 ± 2.0
How easy to determine risks and benefits	7.70 ± 2.2	8.30 ± 1.9*	8.13 ± 1.7
Clarity of the information	8.15 ± 1.93	8.39 ± 1.9	8.42 ± 1.6

Data expressed as mean ± SD, 0–10 scale where 10 = maximum response

^*P< 0.05 vs Text

Discussion

Historically, text has been the standard method for presenting risks and benefits for treatment and research, yet several studies suggest that many parents, patients, and research subjects have poor understanding of these statistics using this format (2–4, 7). Recently, there have been some data to suggest that graphical presentation of risks and benefits are better understood and may enhance decision-making, particularly among individuals with low numeracy (8, 11, 18). Indeed, in comparison with standard text, visual depictions of risks and benefits may reveal patterns that otherwise go unnoticed (19), improve the processing of mathematical computation (20, 21), and better attract and retain an individual’s attention. Other studies also show that graphical formats require less cognitive effort and/or provide different affective responses to numerical information (22). This may explain why parents with lower numeracy and literacy skills in our study were significantly less able to interpret the risks and benefits when presented as text. Indeed, Peters et al. showed that, whereas numerate individuals are more likely to pay attention to numbers in making health care decisions, innumerate individuals rely less on numbers and more on emotion, mood, and trust or distrust of the physician or medical system (23). The improved visual salience of tables and pictographs may thus be helpful in promoting understanding among those with poor language and numerical skills. This is particularly important given that more than 90 million individuals are either illiterate or semi-illiterate in the United States (24) and a large portion of the population experience difficulties with everyday mathematical skills such as calculating a tip or discounting a store item (25, 26). Development of strategies that will enhance understanding among these vulnerable groups is therefore critical.

Results of this study showed that both pictographs and tables were superior to text in promoting gist and verbatim understanding of risk/benefit information. These results confirm the findings and establish the practical relevance of two recent internet surveys comparing different graphical formats for the presentation of risk/benefit statistics (11, 12). In one of these, a survey of parents demonstrated that pictographs were superior to both tables and text in presenting understandable risk/benefit information to both numerate and innumerate individuals (12). However, in the present study, tables were shown to be equally effective as pictographs. The reasons for this are unclear but may simply reflect differences in the populations surveyed (i.e., internet vs. hospital), differences in the design of the tables and pictographs used in the two studies, or differences in statistical power. In another survey, Hawley et al. (11) compared six different graphical formats and showed that pictographs were consistently associated with adequate gist and verbatim understanding among individuals with different numeracy levels but that tables were superior to pictographs in conveying verbatim understanding.

Although these previous studies provided evidence that use of pictograph and/or tabular presentations can improve understanding of risks and benefits, they were limited by their reliance on an internet sample reflecting circumstances far removed from that experienced by real subjects in the clinical setting. Specifically, parents in the internet survey were not exposed to the stresses of having their child undergo surgery and were not under the same time constraints typical of the preoperative setting. The present study was designed, therefore, to overcome some of these limitations by presenting information to a group of parents whose children were scheduled to undergo an elective surgical procedure. That our results were similar demonstrates that presenting risks and benefits in tabular or graphical formats can enhance parents’ understanding even at times when they are anxious and stressed.

The observed equivalence in the parents’ perceptions of the effectiveness and quality of the presented material is interesting given the differences in their understanding using the different formats. In a previous study, we showed that pictographs were deemed more effective, helpful, scientific, and trustworthy in presenting risk/benefit information compared with text (12). Another study, however, comparing an interactive computer-based consent program with standard written consent found no differences in patients’ preferences for the mode of message delivery (27). These observed differences between studies may reflect differences in statistical power or may simply reflect the observation that most individuals, unless offered an alternative, appear to be satisfied with the information they receive even though they may not fully understand it.

An important finding of this study was that the manner in which the information was presented did not appear to change parents’ risk perceptions or willingness to participate in the proposed research. Thus, while presenting information in tables or pictographs may increase knowledge, it does not necessarily affect risk perception decision making.

The results of this study beg the question of whether consent on the day of surgery is appropriate. Other institutions prohibit this practice as potentially coercive; however, the IRB at the University of Michigan allows for day-of-surgery consents. Although several studies confirm that many parents (and patients) have poor understanding of consent information (2–4), we previously showed that the timing of when consent was given (e.g., day of surgery vs. several days prior) did not appear to correlate with the level of understanding (4). Furthermore, one study suggested that providing risk information about anesthesia on the day of surgery does not increase patient anxiety (28). This suggests that a lack of understanding may have more to do with the manner in which we communicate the information rather than when it is given.

The results of this study must be interpreted in the context of some limitations. First, we recognize that consent is a process that requires both a written (for most studies) and a verbal component. For the purposes of this study, we concentrated specifically on the written component as a means to optimize the manner in which risks and benefits are presented in the consent document. This study, therefore, is limited to one component of the consent process recognizing that the addition of a verbal component (albeit quite variable) would likely further enhance parents’ understanding. Second, the consent information provided to parents in this study was presented in a sham format. While this decision could affect the ability to generalize beyond an experimental setting, we note that the study information was presented under the same circumstances (environment and time) as standard consent practice. There is also considerable precedence to the use of sham methodologies and strong evidence to show that behaviors based on real and sham or hypothetical situations are highly correlated (29, 30). Third, the results of this study are based on knowledge of one pediatric anesthesia study from a single institution and thus may not be generalizable to all situations or populations. Furthermore, although tables and pictographs appear to improve understanding of well-defined risks and benefits as described in this study, we recognize that they may be less effective if accurate incidence data are lacking. Finally, we recognize that, since the power analysis was based on expected differences in understanding between parents randomized to the different format groups, there may have been insufficient power for all the subgroup analyses.

Informed consent for anesthesiology research (and procedures) is unique in that it is typically provided on the day of surgery with only a narrow window of opportunity for investigators and physicians to impart the information necessary for a parent or patient to make an informed decision. Given these time-constraints and the potential anxiety of the subject, parent, or patient, it is perhaps even more imperative that anesthesia research and clinical personnel optimize the manner in which consent information is communicated. The results of this study suggest that disclosure of two of the most important elements of consent, the risks and benefits, can be enhanced by presenting them in simple tabular or graphical formats. Although these results were based on parents’ understanding of an anesthesia study, it is likely that these findings will also be relevant to a variety of other clinical and research settings. We therefore advocate use of these techniques as a simple, practical, and inexpensive approach to improving parents’ understanding of these critical elements of consent.