Informing the Uninformed: Optimizing the Consent Message Using a Fractional Factorial Design

Abstract

Objective

Research information should be presented in a manner that promotes understanding. However, many parents and research subjects have difficulty understanding and making informed decisions. This study was designed to examine the effect of different communication strategies on parental understanding of research information.

Participants

640 parents of children scheduled for elective surgery

Design

Observational study using a fractional factorial design

Setting

Large tertiary care children's hospital

Interventions

Parents were randomized to receive information about a hypothetical pain trial presented in one of 16 consent documents containing different combinations of 5 selected communication strategies (i.e., length, readability, processability [formatting], graphical display, and supplemental verbal disclosure).

Main outcome measures

Parents were interviewed to determine their understanding of the study elements (e.g., protocol, alternatives etc.) and their gist (main point) and verbatim (actual) understanding of the risks and benefits.

Results

Main effects for understanding were found for processability, readability, message length, use of graphics, and verbal discussion. Consent documents with high processability, 8th grade reading level, and graphics resulted in significantly greater gist and verbatim understanding compared with forms without these attributes (mean difference, 95% CI = 0.57, 0.26–0.88, correct responses out of 7 and 0.54, 0.20–0.88 correct responses out of 4 for gist and verbatim, respectively).

Conclusions

Results identified several communication strategy combinations that improved parents' understanding of research information. Adoption of these active strategies by investigators, clinicians, IRBs, and study sponsors represents a simple, practical, and inexpensive means to optimize the consent message and enhance parental, participant, and patient understanding.

Introduction

As proxy decision-makers for their children, adequate parental informed consent is of paramount importance in the research and healthcare settings.¹ For most studies, the standard vehicle for the disclosure of research information is the consent form, although the process of consent also requires a meaningful dialogue between the investigator and parent or participant.

As part of the consent process, Federal regulations mandate that research participants receive information regarding several core elements of consent (e.g., risks, protocol, etc.) in a manner that promotes understanding.² Despite this, studies show that consent documents are often difficult for the average individual to understand and that verbal disclosure is quite variable.^3–7 This is important since a lack of understanding may result in the unknowing acceptance of potential risk, non- participation, or may jeopardize the safety of participants who are unable to follow a research protocol.

The observations that understanding of consent information is often inadequate suggest that the current approaches to presenting study information are not always optimal. Indeed, although a variety of interventions to improve participant understanding have been tested, not all have been uniformly effective.⁸ The reasons for this are multifactorial but the fact that research and medical information are often presented in a “one size fits all” format ignores important differences in the way that individuals assimilate and understand complex information. Furthermore, many studies ^9–12 describe just a single intervention (e.g., shorter forms) such that the potential benefits of multiple interventions on understanding have not been fully explored. Finally, many of these studies have focused solely on the consent document rather than the process as a whole.⁸ The hypothesis to be tested, therefore, was that different combinations of active communication strategies (written and verbal) would improve parents' understanding of consent information and thus, optimize the informed consent process.

Methods

Participants

The study was approved by the University of Michigan's Institutional Review Board (IRB) with a waiver of written informed consent. Parents (>18 yr) of children who were scheduled for elective surgery were included.

Consent form development

A series of 16 consent forms were developed that included different combinations of 5 selected communication strategies. Each form was based on the University of Michigan's IRB consent template and described a hypothetical clinical trial examining the efficacy and safety of transitioning children with moderate to severe pain from a short acting to a “new” long acting pain medication (“Painaway”). Prior to distribution, each consent form was reviewed by multiple experts in informed consent and lay individuals for consistency of content across documents.

Message selection

Based on the extant literature and our previous work, we selected 5 communication strategies that have been shown individually to improve subjects' understanding of consent information. The 5 selected strategies (each with two levels) included:

• a)Readability (8^th grade versus 12^th grade reading level)^{13, 14}
• b)Processability/Formatting (low versus high)^7,12 Techniques for improving processability included larger font (size 14), more white space, and highlighting using bulleting, bolding, and underlining. Processability was measured using the techniques developed by Irwin and Davis and adapted by Philipson et al. for use with consent forms.^{15, 16}
• c)Graphic display of risks and benefits (Graphs versus text). Pictographs were chosen to display risks and benefits based on previous research showing their effectiveness.^17–20
• d)Verbal disclosure (yes versus no).^{7, 21–23} Subjects received either a supplemental verbal description of the study using a standard script or no verbal description.
• e)Message length (short versus long).^{7, 10, 24} Simplified versus more detail. On average, the longer forms were 3–4 pages longer than the shorter forms.

Regardless of which strategy combinations were used, each form contained all the elements of consent such that content between documents was not compromised.

Fractional factorial design (FFD)

Because a full factorial design for 5 factors would require an unwieldy 2⁵ = 32 experimental groups, we employed instead a 16-group fractional factorial design in which parents were randomized to receive information about the pain study in one of 16 different consent forms each of which varied in terms of the message level combinations (see table 1). In order to determine the operational effect of each of the 5 strategies, half of the parents were randomized to one level of the factor and half to the other level and the outcomes (i.e., understanding) compared. FFDs have been used effectively in engineering applications,^25–27 however, their usefulness in behavioral studies has only recently been recognized.²⁸ The advantage of FFDs is that they allow for the study of a large number of factors using fewer cells than required by a full factorial design (i.e. 16 versus 32). Thus, we chose to employ a Resolution V FFD with 16-groups that would allow us to estimate all main effects and all pre-specified 2-factor interactions among the 5 communication strategies.

Table 1.

Sixteen-cell Resolution V Fractional Factorial Design with 5 two-level covariates

Group	Readability	Processability	Graphics	Verbal	Length
1	8	Low	Text	No	Long
2	8	Low	Text	Yes	Short
3	8	Low	Graph	No	Short
4	8	Low	Graph	Yes	Long
5	8	High	Text	No	Short
6	8	High	Text	Yes	Long
7	8	High	Graph	No	Long
8	8	High	Graph	Yes	Short
9	12	Low	Text	No	Short
10	12	Low	Text	Yes	Long
11	12	Low	Graph	No	Long
12	12	Low	Graph	Yes	Short
13	12	High	Text	No	Long
14	12	High	Text	Yes	Short
15	12	High	Graph	No	Short
16	12	High	Graph	Yes	Long

N = 40/group

Procedures

Parents were approached in the preoperative waiting area by trained research assistants and told that we were evaluating the quality of our consent process. Parents were also told that their child would NOT be participating in an actual study but that they should consider the information as if real. Although the study represented a hypothetical study it was nevertheless presented to simulate our standard practice for consenting parents (i.e., place, time to read the consent, and environment). Parents received a $5 gift-card for their participation.

Primary outcome measures

Understanding of the consent elements

Parents were interviewed to measure their understanding of 11 required elements of consent (45.CFR.46).² The semi-structured interview was based on the Deaconess Informed Consent Comprehension Test (DICCT)²⁹ and followed the format described previously.^{3, 7, 30} Outcome measures included understanding of the study purpose, procedures, alternatives, direct and indirect benefits, risks, voluntariness, freedom to withdraw, study duration, contact information, and confidentiality. The open-ended responses were written down verbatim by trained research assistants. The interviewer was allowed to clarify individual questions and prompt parents for additional information but could not offer specific details. The transcribed responses were scored independently by two assessors who knew the study but were blinded as to which consent form was administered. The scoring system utilized a 0–2 scale based on the parent having no (0), partial (1), or complete (2) understanding.⁷

Gist and Verbatim understanding of risks and benefits

Gist understanding refers to the ability to understand the essential meaning of the risks and benefits.^{31, 32} Seven gist questions were included, one of which was: “If a child received Painaway which of the following is most likely: 1) good pain relief, 2) nausea and vomiting, 3) constipation, 4) slowed breathing. Gist understanding was defined as the ability to respond correctly to >5 questions out of 7. This threshold definition was based on previous studies,^17,19 however, as validation, a sensitivity analysis was conducted and found qualitatively similar results using either a continuous measure of understanding or different thresholds e.g., 7 out of 7 correct answers.

Verbatim understanding refers to the ability to correctly report the actual risk/benefit statistics.^{31, 32} Four verbatim questions were included and required parents to respond to the question, “If 100 children took Painaway, how many would experience: 1) good pain relief, 2) nausea and vomiting, 3) constipation, and 4) excessive sleepiness?” Per previous work, verbatim understanding was defined as the ability to correctly respond to ≥ 3 out 4 verbatim questions.^{17, 19}

Secondary outcomes and parent characteristics

Parents completed a questionnaire and two validated instruments regarding factors that influenced their understanding of the consent document.

Secondary outcome measures

• 1)Perceptions of the risks, benefits, and study importance (0–10 numbers scale, where 10 = high).
• 2)Satisfaction with the consent document format (5-point Likert scale of “extremely dissatisfied” to “extremely satisfied”).
• 3)
  Consent form factors:

  • a)
    degree to which parents listened (if applicable) to the verbal disclosure and read the consent document (i.e., completely, partially, or not at all).
  • b)
    clarity of the information (i.e., not at all clear, fairly clear, or very clear),
  • c)
    amount of information (i.e., too little, just right, or too much)
  • d)
    readability (i.e., very difficult to read, about right, or very easy to read).

Parent characteristics

• 1)Socio-demographics: Age, gender, race/ethnicity, education, parent/child prior research participation.
• 2)Medical literacy: Measured using the validated Rapid Estimate of Adult Literacy in Medicine (REALM) instrument.³³
• 3)Numeracy: Measured using the validated 8-item subjective numeracy scale (SNS).^{34, 35}
• 4)Parental anxiety: Measured using a 0–10 numbers scale (10 = extremely anxious).^{36, 37}

On completion of the study, parents were shown two consent documents representing two extremes of the written strategies (i.e. 8^th grade, high processability, short, and graphics versus 12^th grade, low processability, long, no graphics) and asked which they preferred (see eAppendix).

Statistical Analysis

Data were analyzed using PASW (Chicago, IL) and R software. Prior to the principal analyses, one-way ANOVA was used to determine if randomization was successful in creating equivalent groups. Because the threshold definitions of understanding e.g., ≥3 correct answers out of 4 are considered discrete data, they did not satisfy the usual assumptions for ANOVA and were thus, analyzed using logistic regression techniques. Although these definitions have been used previously,^17,19 we performed sensitivity analyses to ensure that different threshold definitions produced qualitatively similar results. For example, the number of correct responses for verbatim understanding ranged from 0–4. Comparisons were thus made between 0–3 vs 4 correct answers; 0–2 vs 3–4; and 0–1 vs 2–4. Similar analyses were conducted for the other outcomes and confirmed that different thresholds produced qualitatively similar results.

The final models were determined using the Akaike information criterion (AIC) model-selection criterion.³⁸ AIC selects the best fitting model from other candidate models in minimizing information loss. Inter-rater reliability for the assessors' scores of understanding was measured using the Kappa statistic. Kappa scores for each of the elements of consent ranged from 0.76 to 1.0 (P < 0.001) revealing very good/excellent agreement. Data are described as mean differences and odds ratios with 95% confidence intervals (CI).

Sample size was based on previous data comparing parental understanding of consent information using different individual communication strategies e.g., use of different graphics and high versus low processability.^{7, 20} Based on these data, we considered it important to detect an effect size of 0.5 between levels of each factor. This estimate required at least 640 subjects i.e., 40 subjects per cell in the 16-cell FFD (α = 0.05, β = 0.1, 2-sided).

Results

A total of 871 parents were approached to participate in this study of whom 209 declined and 22 withdrew. Data are thus presented for 640 parents. There were no differences in demographics between the 16 consent groups thus; table 2 provides a summary of the entire sample.

Table 2.

Parental demographics

	N = 640
Parent age (yrs)	35.9 ± 8.8
Gender (M/F %)	27.7/72.3
English as first language	600 (94.0)
Race/ethnicity:
Caucasian	496 (77.6)
African American	83 (13.0)
Hispanic	25 (3.9)
Asian	15 (2.3)
Other	20 (3.1)
Highest level of education:
Grade school	38 (5.9)
High school graduate	112 (17.5)
Some College/Trade school	192 (30.0)
College graduate	205 (32.1)
Graduate school	92 (14.4)
REALM score (range 0–66)	63.8 ± 5.3
Numeracy score (range 0–48)	36.4 ± 9.4
Parent anxiety (0–10)	6.01 ± 2.9
Parent prior research participant	89 (14.0)
Child prior research participant	112 (17.6)

Data are presented as n (%) and mean ± SD; REALM = Rapid Estimate of Adult Literacy in Medicine

Overall, 182 (28.5%) parents reported that they had read “most” and 441 (69.0%) “all” of the consent form to which they were randomized. Short forms were more likely to be read completely compared with long forms (OR, 95% CI = 1.75, 1.23–2.49). Of those who did not read the entire form, 33 (5.2%) skipped the study purpose, 31 (4.9%) the procedures, 30 (4.7%) the benefits, 84 (13.2%) privacy issues, and 108 (17.0%) contact information.

Tables 3 and 4 describe the logistic regression analyses for verbatim and gist understanding, respectively. As shown in table 3, high processability and inclusion of graphs had significant positive effects on verbatim understanding as indicated by their positive coefficients. The odds ratio for inclusion of graphs, for example, showed that the odds of verbatim understanding were increased by 50% if graphs were used. Longer consent forms, on the other hand, had a negative effect on verbatim understanding. For gist understanding (table 4) high processability, inclusion of graphs, and verbal disclosure had significant positive effects, whereas readability (12^th grade) had a negative effect. Logistic regression analyses for understanding of the individual elements of consent showed that high processability had significant positive effects on understanding of the study's purpose, procedures, risks, benefits, and study duration. Use of graphs had positive effects on the understanding of risks and alternatives and supplemental verbal disclosure had a positive effect on understanding of the procedures. In all regression models high numeracy and Caucasian race predicted improved understanding of the consent information.

Table 3.

Final Logistic Regression Model for Verbatim Understanding^* of Risks and Benefits

	Coefficient	S.E.	z value	OR	95% CI lower	95% CI upper
Processability (high)	0.297	0.091	3.262	1.35	1.13	1.61
Graph (yes)	0.408	0.091	4.451	1.50	1.26	1.80
Length (long)	−0.351	0.091	−3.83	0.29	0.59	0.84
Minority Race	−0.486	0.231	−2.107	0.39	0.39	0.97
Numeracy (high)	0.065	0.011	5.823	1.07	1.04	1.09
Age (yrs)	−0.023	0.011	−2.065	0.023	0.96	1.00

SE = Standard error, OR = Odds ratio, CI = Confidence interval

^*Verbatim understanding = ≥ 3 correct answers out of 4

Table 4.

Final Logistic Regression Model for Gist Understanding^* of Risks and Benefits

	Coefficient	S.E.	z value	OR	95% CI lower	95% CI upper
Verbal (yes)	0.179	0.094	1.906	1.20	0.99	1.44
Processability (high)	0.351	0.096	3.663	1.42	1.18	1.71
Readability (12th grade)	−0.29	0.096	−3.01	0.25	0.62	0.90
Graph (yes)	0.268	0.094	2.836	1.31	1.09	1.57
Gender (M)	−0.559	0.213	−2.616	0.43	0.38	0.87
Minority Race	−1.106	0.227	−4.855	0.67	0.21	0.52
Numeracy (high)	0.058	0.01	5.451	1.06	1.04	1.08
Age (yrs)	−0.022	0.011	−1.969	0.02	0.96	1.00
Processability:Readability†	−0.33	0.096	−3.428	0.28	0.60	0.87

SE = Standard error, OR = Odds ratio, CI = Confidence interval

^†= synergistic interaction between low processability and 12^th grade reading level

^*Gist Understanding = > 5 correct answers out of 7

Based on the results of the regression models, we assigned positive attributes to: 8th grade reading, high processability, graphics, verbal disclosure, and shortened forms. We then examined the effect of the number of positive attributes (1, 3, or 5) per consent form on parents' understanding and perceptions (Table 5). Results showed that although inclusion of all 5 positive attributes resulted in a trend towards greatest understanding, forms with at least 3 positive message attributes were sufficient to improve understanding significantly over those containing only one positive attribute. The number of positive attributes had no effect on understanding of voluntariness, freedom to withdraw, confidentiality, study duration, and contact information. Given that 3 positive attributes were sufficient to improve understanding, the effect of combining the 3 messages with the strongest main effects i.e., 8^th grade reading, high processability, and graphics was examined. Forms containing these specific attributes resulted in significantly greater gist and verbatim understanding compared with forms without these attributes (mean difference, 95% CI = 0.57, 0.26–0.88 correct responses out of 7 and 0.54, 0.20–0.88 correct responses out of 4 for gist and verbatim, respectively). Similarly, understanding of the study purpose (0.22, 0.07–0.36 out of 2 correct responses), the risks (0.22, 0.07–0.38 out of 2), and benefits (0.17, 0.005–0.33 out of 2) were better understood when these 3 attributes were included.

Table 5.

Effects of the Number of Positive Message Attributes on Parents' Understanding and Preferences

	Understanding by the Number of Positive Message Attributes
	1 (n = 200)	3 (n = 400)	5 (n = 40)
Informationa:
Quality	7.78 ± 1.8	8.16 ± 1.7	7.9 ± 2.1
Layout	7.34 ± 2.3	8.02 ± 1.9*	8.35 ± 2.0*
Clarity	7.64 ± 2.0	8.27 ± 1.7*	8.03 ± 1.9
Risk/benefita:
Effectiveness	7.29 ± 2.4	7.83 ± 1.9*	8.28 ± 1.8*
Ease of interpretation	7.46 ± 2.3	8.07 ± 1.9*	8.43 ± 1.9*
Clarity	7.59 ± 2.2	8.16 ± 2.0*	8.33 ± 2.0*
Understanding:
R/B Gistb	5.35 ± 1.7	5.88 ± 1.5*	6.41 ± 1.2*
R/B Verbatimc	1.54 ± 1.3	2.21 ± 1.4*	2.69 ± 1.2*
Purposed	1.33 ± 0.61	1.37 ± 0.61	1.63 ± 0.49*
Protocold	1.10 ± 0.63	1.24 ± 0.64*	1.38 ± 0.67*
Risksd	1.17 ± 0.72	1.34 ± 0.64*	1.65 ± 0.53*†
Benefitsd	1.45 ± 0.75	1.49 ± 0.74	1.70 ± 0.61
Alternativesd	1.56 ± 0.76	1.49 ± 0.79	1.38 ± 0.89

R/G = Risk/benefit

Scores are mean ± SD

^a0–10 scale (where 10 = maximum response),

^b0–7 scale (# of correct responses out of 7),

^c0–4 scale (# of correct responses out of 4),

^d0–2 scale (where 2 = complete understanding)

^*P< 0.025 vs 1,

^†P< 0.025 vs 3 (Bonferroni corrected)

When shown the two documents with contrasting attributes (#8 vs #10, see table 1 and eAppendices 1 and 2), 460 (74.3%) parents preferred the consent form with the positive attributes (#8). Comments included “less words, easier to understand, simpler, like the bullet points, key points more obvious.” Those who preferred form #10 reported that they preferred more information and detail.

Discussion

Previously, Flory et al. conducted a systematic review of interventions designed to improve understanding of informed consent for research.⁸ This review highlighted several individual interventions used in the present study that improved understanding of consent including enhanced consent forms (i.e., improved readability, reduced length, and formatting), and verbal disclosure. ^{9, 10, 12, 23} However, this review also noted that these interventions were not consistently effective suggesting that any single intervention cannot be “all things to all people.” To date, there have been limited data with respect to the effectiveness of combining interventions, thus, our study is important in identifying specific strategies, that when combined, enhance understanding on a more consistent level.

The present study identified several active communication strategies that improved parental understanding of consent information. Furthermore, data showed that consent forms containing at least 3 active communication strategies resulted in enhanced understanding. In particular, consent forms written at 8^th grade level with high processability and use of pictographs provided the best format for understanding risk/benefit statistics. Document processability was a strong and consistent predictor of understanding in our study. Improved processability was achieved simply using larger font, more white space and by bolding and underlining important information. Previously, we showed that consent/assent forms with high processability significantly improved understanding of research information and were preferred by 82% of parents and children compared with a standard form.^{7, 39} The use of risk/benefit graphics also resulted in improved understanding of risks. We chose pictographs as the graphic of choice as previous work suggests that this format is both preferred and better understood compared with text and other graphical formats.^{17, 19, 20} Graphical depiction of risks and benefits may work because they have greater visual salience, require less cognitive effort and thus, are easier to understand.^40–43

It was surprising that health literacy was not an independent predictor of understanding given that previous studies have shown the importance of health literacy in understanding health information. A likely reason for this was that our population was skewed towards those with good health literacy and this likely confounded the results. Despite this, the consent forms written at an 8^th grade level were better understood compared with those written at the 12^th grade level and thus, reemphasizes the importance of reducing the grade reading level of these forms.

Numeracy, however, was a significant predictor of understanding. Given that innumerate individuals are more likely to make risk/benefit trade-offs based on emotion or trust rather than numbers,⁴⁴ these results are significant in emphasizing the importance of improving the presentation of risk/benefit statistics to those with low numeracy. Overall, Caucasian parents had higher numeracy and literacy abilities compared with minority parents and were more likely to have been educated beyond high school. However, even when controlling for these factors, Caucasian parents had better understanding compared with minority parents. These results are important in that a lack of understanding may contribute to greater distrust or disparities in research participation.^{45, 46}

There are limitations to our approach that must be recognized. First, this study was based on a hypothetical study and as such may not fully replicate the real-life situation. To obviate this concern, the consent forms and the process of consent for this study mimicked our routine research practice. Furthermore, there is precedence for the use of simulated methodologies and strong evidence showing that behaviors based on real and simulated methodologies are highly correlated.^{47, 48} Second, while the supplemental verbal disclosure used in this study was standardized we recognize that, in practice, more or less information may be given depending on the investigator-subject interaction.

This study highlights the importance of employing several different active communication strategies in the design of clinical consent forms. Results suggest that incorporation of at least 3 of the identified active strategies, serves to significantly improve understanding of the information and perceptions of the message delivery. Although these results are based on parents' understanding of pediatric consent documents it is reasonable to assume that they would also be relevant to consent forms for adult research subjects and for patients who consent to treatment. We therefore recommend that investigators, clinicians, IRBs, and study sponsors consider inclusion of these active strategies as a simple, effective, and inexpensive means to optimize the consent message and enhance subject understanding and participation.