A Pilot Study of Simple Interventions to Improve Informed Consent in Clinical Research: Feasibility, Approach, and Results

Abstract

Background

Informed consent is intended to ensure that individuals understand the purpose, risks, and benefits of research studies, and then can decide, voluntarily, whether to enroll. However, research suggests that consent procedures do not always lead to adequate participant understanding and may be longer and more complex than necessary. Studies also suggest some consent interventions, including enhanced consent forms and extended discussions with patients, increase understanding, yet methodologic challenges have been raised in studying consent in actual trial settings. This study aimed to examine the feasibility of testing two consent interventions in actual studies and also to measure effectiveness of interventions in improving understanding of trials.

Methods

Participants enrolling in any of eight ongoing clinical trials (“collaborating studies”) were, for the purposes of this study, sequentially assigned to one of three study arms involving different informed consent procedures (one control and two intervention). Control participants received standard consent form and processes. Participants in the 1^st intervention arm received a bulleted fact-sheet providing simple summaries of all study components in addition to the standard consent form. Participants in the 2^nd intervention arm received the bulleted fact-sheet and standard consent materials and then also engaged with a member of the collaborating study staff in a feedback Q&A session. Following consent procedures, we administered closed and open ended questions to assess patient understanding and we assessed literacy level. Descriptive statistics, Wilcoxon-Mann-Whitney and Kruskal-Wallis tests were generated to assess correlations; regression analysis determined predictors of patient understanding.

Results

144 participants enrolled. Using regression analysis participants receiving the 2^nd intervention, which included a standard consent form, bulleted fact sheet and structured question and answer session with a study staff member, had open-ended question scores that were 7.6 percentage points higher (p=.02) than participants who received the control arm (standard consent only), although unadjusted comparisons did not reach statistical significance. Eleven clinical trial investigators agreed to participate and 8 trials provided sufficient data to be included, thereby demonstrating feasibility of consent research in actual settings.

Conclusions

Our study supports the hypothesis that patients receiving both bulleted fact sheets and a question and answer session have higher understanding compared to patients receiving standard consent form and procedures alone. Fact sheets and short structured dialog are quick to administer and easy to replicate across studies and should be tested in larger samples for effectiveness.

Introduction

U.S. regulations and international guidance documents for the ethical conduct of research emphasize that informed consent is required for most clinical research.^1-3 Grounded in the ethical principle of respect for persons, informed consent is intended to ensure that individuals understand the purpose, risks, and benefits of proposed investigations and then can decide, voluntarily, whether to enroll. While Institutional Review Boards (IRBs) are charged with ensuring the overall ethical acceptability of research studies,¹ the informed consent process allows potential participants to make decisions in accordance with their own beliefs and preferences. The requirement of informed consent rests on an assumption that individuals considering research participation have adequately understood the information provided to them. Studies over more than three decades, however, have repeatedly shown that some participants are unaware of being enrolled in research,^4-7 have poor understanding of research benefits and side effects,^8-13 with the concept of randomization particularly difficult to understand.^14-18 Higher education is often correlated with greater understanding.^5,14,19-23 While consent forms are used both to explain research to prospective subjects and to document willingness to participate, evidence suggests consent forms are long and highly complex;^2,24-37 a recent study found adult HIV trial forms had median length of 27 pages.³⁸ Another study found that IRB review of consent forms resulted in forms becoming significantly longer and more complex.³⁹

Various interventions have been tested to improve informed consent, including modifications of forms, multimedia presentations, monetary incentives, communication training programs for clinicians, and follow-up or modified consent interactions.^40-49 While studies have had mixed results, some evidence suggests that simpler and shorter consent forms ^{19,22,44,50-57} and increased dialogue between potential participants and study team members may improve understanding,^58-63 while video and computer presentations have had mixed results.^{43,45,55,64-66} A systematic review of 54 informed consent interventions suggests that enhanced consent forms and extended discussions with subjects best improved understanding.⁶⁷

Studies that have experimentally measured the effects of consent interventions have often had methodological limitations.^41,42 Most intervention studies have tested individuals' understanding of alternative consent approaches under simulated research conditions, rather than in actual trials. Actual subjects, however, often learn about studies through many sources, including informal conversations with study staff and family members, information on the internet, or asking questions during the consent process.^68-71 Consent studies in simulated settings typically do not incorporate these real-world supportive interactions and therefore may overestimate the value of the specific enhancements under study by minimizing the impact of other influences on understanding.

Evaluating new consent interventions in simulated settings is understandable, as experimental consent research in ongoing studies poses methodological, logistical, and regulatory challenges. Randomizing subjects to different consent interventions cannot easily be masked, potentially leading to bias or contamination in consent delivery and data collection. Parent trial investigators may think collaborations are too burdensome, as they require modification to IRB- and perhaps sponsor-approved consent procedures and may worry that embedding experimental informed consent research into their studies may confuse subjects by collecting data for two studies at once, or may slow down recruitment.⁷² Further, IRBs may not be comfortable altering the information provided in consent interactions. Nonetheless, the large body of evidence that many subjects leave consent interactions with inadequate understanding suggests that more rigorous, experimental informed consent work in actual clinical trials is warranted.⁷³ Further, as more studies move into “pragmatic” clinical settings,⁷⁴ with fewer research staff and an emphasis on collecting data efficiently with minimal burden for clinical staff, identifying streamlined ethically acceptable consent procedures is additionally important.^75-77

Intervention studies also have been criticized for how they measure understanding. Many have used true/false or multiple choice assessments that may measure recall more than comprehension.^41,44,78,79 Further, many studies have used the same tests repeatedly as both intervention and assessment until subjects' scores significantly improve.^79-81 This raises whether understanding improved or subjects memorized correct answers.

This study developed and pilot tested two simple, easily transferable informed consent interventions and an original instrument to measure participants' understanding under experimental vs. standard conditions. These interventions were implemented in eight ongoing clinical trials of the Johns Hopkins Schools of Medicine (SOM) and Public Health (JHSPH) to determine both feasibility and effectiveness. The study potentially addresses some of the methodological pitfalls and challenges that previous informed consent intervention research has encountered.

Methods

Informed Consent Interventions

Informed consent interventions were designed to a) improve participants' understanding; and b) be simple and inexpensive to create and use.

Bulleted Fact Sheet

Intervention #1 was a bulleted fact sheet for each collaborating study. This intervention was based on literature indicating that summaries of information, simplified text, and shorter sentences improve comprehension. Our preliminary focus groups with former research participants suggested study summaries would be useful. Fact sheets were created for each collaborating study using bullets that highlighted in 1-2 sentences key study information. Fact sheets drafts were reviewed with collaborating study investigators for accuracy and to ensure key information was included. Fact sheets were 1-3 pages long, while standard collaborating study forms were 6-15 pages; Fleisch-Kinkaid readability scores of fact sheets were, on average, 1.4 grades lower than standard forms (6.7 vs. 8.3). Supplementary Appendix 1 contains a sample bulleted fact sheet.

Verbalization of Informed Consent Essentials (VOICE)

Intervention #2 was an original short question/answer tool called “Verbalization of Informed Consent Essentials” (VOICE) designed for use within the consent process. Literature suggests that understanding improves when subjects must verbalize information,^61,62,67 and suggests quizzes with correction can be helpful.⁸² The VOICE tool consisted of six open-ended questions relevant to almost all research studies, with an additional two questions added for the randomized studies (Figure 1). Collaborating study staff members were asked to use VOICE after going through consent information but before agreement to participate was documented. Each question was asked individually. The study staff member listened to each response and discussed and corrected any inaccuracies.

Figure 1. VOICE instrument.

Collaborating studies

We sought to work with 11 studies from the Johns Hopkins Schools of Medicine and Public Health. We sought studies with diverse research topics, designs, and subjects, as consent interventions were designed to be transferable to most study settings. Thus, while we deliberately sampled several randomized studies, given the literature that randomization is a difficult concept to understand, we also included observational studies and included studies both with healthy individuals and patients, and with complex and simpler research information (Figure 2). We refer to these as “collaborating studies (or staff)” and to our own study as “consent study (or staff)”.

Figure 2. Summary of collaborating studies.

Ethical review

This study was approved by the JHSPH IRB. Since our study altered IRB-approved consent procedures of collaborating studies, we worked with the two relevant IRB offices to identify procedures for enrolling actual participants using modified consent procedures. This resulted in a decision to test the feasibility and efficacy of our interventions in addition to standard consent forms and procedures rather than as an alternative, avoiding any regulatory concerns from IRBs if the full standard consent form was not used.

Collaborating investigators submitted protocol amendments to their IRBs for approval to enroll patients into their studies using any of our three consent approaches. Amendment requests included a) a cover letter, written by our staff, explaining the amendment was related to our consent study; b) a request to audio record consent discussions; c) a request to use the interventions for some patients.

Sequential allocation design

This study compared participants' understanding of the collaborating study in which they were enrolling under three conditions:

Arm 1: Standard consent (control arm)

Consent forms were developed by collaborating study investigators. Participants in Arm 1 were enrolled using the standard, IRB-approved consent form and process.

Arm 2: Bulleted fact sheet + Standard consent

The bulleted fact sheet was used by collaborating study staff to explain key information to participants. Collaborating staff then administered the usual consent form and process.

Arm 3: Bulleted fact sheet + Standard consent + VOICE tool

The bulleted fact sheet was used by collaborating study staff to explain key information to participants. Collaborating staff went through their usual consent form and process; finally, collaborating study staff asked participants each open-ended VOICE question, one by one, listening to answers and discussing and correcting inaccurate responses.

To mimic consent interactions in actual research settings, all three arms were implemented by collaborating study staff, with no involvement by consent study staff. The three arms were implemented sequentially (rather than randomly) to avoid contamination if collaborating staff were asked to continuously alter consent procedures. We sought to enroll 12-24 participants from each collaborating study (over three arms), depending on their projected sample size. We hypothesized that participants in Arm 3 would receive higher CUE score than participants in Arm 2; and that both Arms 2 and 3 would have higher scores than participants in the control (Arm 1).

Informed consent for the pilot consent study

Prior to starting a consent process with a potential participant, collaborating staff briefly explained that they were working with colleagues who wanted to learn more about how researchers explain research studies to participants. The collaborating staff person then asked if they could audio record the consent process and refer them to a consent study team member afterwards by phone. After consent (in any arm) was completed, the collaborating study staff member asked participants again about their willingness to speak by phone with our consent study team. Willing participants called our dedicated study phone number, generally from the clinic. Consent study staff obtained oral consent to be interviewed, for audio recording of the interview and consent discussion, and for a short literacy assessment by phone.

Data collection

See Figure 3 for study flow.

Figure 3. Informed consent study arms.

Audio recording of consent interactions

Collaborating study staff were asked to audio record consent interactions for participants in all three consent study arms. We wanted data on how standard consent interactions were implemented and whether the two experimental arms were implemented as we envisioned, especially the VOICE tool, which required staff discernment.

Consent Understanding Evaluation (CUE)

We developed a structured interview tool for use by consent study staff to measure participants' understanding. Development of our Consent Understanding Evaluation (CUE) tool was partially informed by other instruments used in previous consent studies ^22,83-86 and through input from experts in low literacy communication, consent understanding evaluation, survey methodology, and research ethics. We conducted three focus groups with current and former research participants and community advisory board members to hear which study elements they thought most critical for participant understanding. Ten cognitive interviews and ten pilot interviews were conducted with current and former research participants to ensure appropriate interpretation and flow. We sought to create an instrument administrable via telephone which we knew would be important for some future experimental and multi-site informed consent studies.

The CUE instrument (Supplementary Appendix 2) consists of approximately 50 open-and closed-ended questions, taking approximately 20 minutes to administer.

• 25 questions related to study purpose, voluntariness, risks and design; 3 additional questions were added for studies that were randomized. Approximately 2/3 of all questions were identical across all collaborating studies; approximately 1/3 focused on understanding of the specific collaborating study but were consistent across studies in the structure of the question and the general topic (e.g., study purpose or study risks)

• 8 questions attitudinal questions about research and consent

• 8 questions about consent for this collaborating study

• 11 demographic/background questions

CUE interviews were conducted by phone and audio recorded.

Rapid Estimate of Adult Literacy in Medicine (REALM)

The Rapid Estimate of Adult Literacy in Medicine (REALM) was used to assess health literacy of participants.⁸⁷ REALM is a well validated, interviewer-administered word recognition test taking 2-3 minutes to complete. It includes 22 health related words of increasing difficulty. Previously, REALM had only been validated for administration in person. We tested the feasibility of administering REALM by telephone among participants in an urban outpatient clinical setting. The phone equivalence and feasibility sub-study revealed no significant obstacles to phone administration; REALM scores correlated with years of reported education and were comparable to REALM administration in person.⁸⁸ We administered and audio recorded REALM by phone in this study, immediately following each CUE interview.

On-site study materials

Consent study binders with color-coded, easy to follow tabs, were placed at each study site and given to participants to use during phone interviews. For example, one binder section contained the list of words participants read for REALM.

Each participant received a $20 gift card.

Data entry & analysis

Audio recordings were transcribed and quality checked. Raw CUE and REALM data were double entered into a master database and cleaned. Responses to open-ended questions were scored as 1 for correct or 0 for incorrect (Figure 4). Two investigators, blinded to the study arms, independently scored open-ended responses and reconciled any discrepancies. Each participant's scores on open-ended questions were averaged, creating an open-ended score for each participant ranging from 0 to 1. The same was done for closed-ended questions. An overall summary measure for each participant averaged their open- and closed-ended scores. Measures also were created for open-, closed-ended, and overall summary scores on only the questions related to randomization, given the challenges, historically, in understanding that concept. Average REALM readability scores and health literacy levels were calculated for each participant.

Figure 4. Example of scoring criteria for open-ended CUE questions.

Data analysis was performed using STATA 12.⁸⁹ Descriptive statistics (Table 1) and statistical comparison of means and medians in scores across study arms using Kruskal-Wallis tests (Table 2) were conducted. We also compared the proportion of correct answers on each CUE item by study arms (Supplementary Appendix 3). Based on our descriptive analysis, we then performed OLS regression analysis to control for associations among independent variables and identify any predictors of greater patient understanding for three outcome variables: open-ended, closed-ended and overall CUE scores (Table 3).

Table 1. Demographic characteristics of participants by study arm.

		Arm 1 n=49		Arm 2 n=45		Arm 3 n=50		Total n=144
Gender
Male		20	40.8%	14	31.1%	13	26.0%	47	32.6%
Female		29	59.2%	31	68.9%	37	74.0%	97	67.4%
Age
Mean (SD)		52.4	11.99	48.6	14.02	53.14	15.07	51.5	13.8
Median		53		50		54		52
Ethnicity
Hispanic		2	4.1%	2	4.4%	1	2.0%	5	3.5%
Non-Hispanic		46	93.9%	43	95.6%	49	98.0%	138	95.8%
No response		1	2.0%	0	0.0%	0	0.0%	1	0.7%
Race
White		26	53.1%	24	53.3%	28	56.0%	78	54.2%
Black		23	46.9%	17	37.8%	20	40.0%	60	41.7%
American Indian		0	0.0%	1	2.2%	1	2.0%	2	1.4%
Asian		0	0.0%	3	6.7%	0	0.0%	3	2.1%
Other		0	0.0%	0	0.0%	1	2.0%	1	0.7%
Income
<$25,000		18	36.7%	18	40.0%	23	46.0%	59	41.0%
$25,000 - $49,999		10	20.4%	12	26.7%	12	24.0%	34	23.6%
$50,000 - $74,999		7	14.3%	3	6.7%	6	12.0%	16	11.1%
$75,000 - $100,000		7	14.3%	4	8.9%	5	10.0%	16	11.1%
>$100,000		6	12.2%	6	13.3%	4	8.0%	16	11.1%
No response		1	2.0%	2	4.4%	0	0.0%	3	2.1%
Employment status
Full-time		15	30.6%	13	28.9%	13	26.0%	41	28.5%
Part-time		10	20.4%	6	13.3%	7	14.0%	23	16.0%
Not employed		24	49.0%	26	57.8%	30	60.0%	80	55.6%
Reading level (REALM)
1	3rd grade or below	1	2.0%	1	2.2%	0	0.0%	2	1.4%
2	4th - 6th grade	3	6.1%	1	2.2%	1	2.0%	5	3.5%
3	7th-8thgrade	4	8.2%	10	22.2%	13	26.0%	27	18.8%
4	high school	41	83.7%	33	73.3%	36	72.0%	110	76.4%
Self-reported health status
Excellent		6	12.2%	6	13.3%	7	14.0%	19	13.2%
Good		28	57.1%	19	42.2%	25	50.0%	72	50.0%
Fair		12	24.5%	17	37.8%	16	32.0%	45	31.3%
Poor		3	6.1%	3	6.7%	2	4.0%	8	5.6%
Had private or public medical insurance
Yes		39	79.6%	43	95.6%	42	84.0%	124	86.1%
Private		20	40.8%	18	40.0%	20	40.0%	58	40.3%
Public		18	36.7%	25	55.6%	22	44.0%	65	45.1%
Unspecified		1	0.0%	0	0.0%	0	0.0%	1	1.0%
No		10	20.4%	2	4.4%	8	16.0%	8	5.6%
Received consent form in advance
Yes		16	32.7%	12	26.7%	17	34.0%	45	31.3%
No		33	67.3%	33	73.3%	33	66.0%	99	68.8%
Read consent form on own
Yes		38	77.6%	36	80.0%	37	74.0%	111	77.1%
No		11	22.4%	9	20.0%	13	26.0%	33	22.9%
Showed consent form to others
Yes		7	14.3%	2	4.4%	5	10.0%	14	9.7%
No		41	83.7%	43	95.6%	45	90.0%	129	89.6%
No response		1	2.0%	0	0.0%	0	0.0%	1	0.7%
Past enrollment
Yes		26	53.1%	21	46.7%	30	60.0%	77	53.5%
No		23	46.9%	24	53.3%	18	36.0%	65	45.1%
Don't know		0	0.0%	0	0.0%	2	4.0%	2	1.4%

Table 2. Understanding scores across study arms.

	Arm 1	Arm 2	Arm 3	Kruskal Wallis p-value (with ties)
Open Ended Score
Mean	0.74	0.77	0.80	.25
Standard Deviation	0.23	0.18	0.21
Median	.83	.83	.83
IQR	.17	.25	.33
One sample median test (p>z)	0.13	0.83	0.27
Closed Ended Score
Mean	0.75	0.75	0.76	.82
Standard Deviation	0.19	0.19	0.18
Median	.83	.71	.77
IQR	.35	.30	.35
One sample median test (p>z)	0.80	0.85	0.89
Overall Score
Mean	0.75	0.76	0.78	.43
Standard Deviation	0.17	0.19	0.16
Median	.79	.79	.79
IQR	.17	,24	.15
One sample median test (p>z)	0.35	0.71	0.20

Table 3. Regression analysis.

	Dependent variable:
	(1) Open score β (SE)	(2) Closed score β (SE)	(3) Overall score β (SE)
Study Arm 2	0.064 (0.04)	0.014 (0.04)	0.039 (0.04)
Study Arm 3	0.076** (0.02)	0.010 (0.06)	0.043 (0.03)
Age	0.000 (0.00)	0.000 (0.00)	0.000 (0.00)
Female	0.044 (0.05)	0.008 (0.03)	0.026 (0.03)
Income $25,000-$49,999 (Base: Income <$25,000)	0.086 (0.04)	-0.012 (0.0)	0.037 (0.02)
Income >=$50,000 (Base: Income <$25,000)	0.101* (0.04)	-0.057 (0.06)	0.022 (0.04)
Employed part-time (Base: Employed full time)	0.034 (0.04)	-0.061 (0.05)	-0.013 (0.04)
Not employed Base: Employed full time	-0.034 (0.05)	-0.048 (0.04)	-0.041 (0.04)
Race – White (Base: Race – non-White)	0.059 (0.05)	0.029 (0.06)	0.044 (0.05)
Good or excellent health (Base: Poor or fair health)	-0.004 (0.04)	0.062 (0.03)	0.029 (0.04)
High school reading level (REALM 4) (Base: Below high school reading level (REALM 1, 2, or 3))	0.083 (0.04)	0.063 (0.03)	0.073* (0.03)
Received consent form in advance	0.033 (0.04)	-0.020 (0.07)	0.006 (0.05)
Read consent form on own	0.055 (0.04)	0.044 (0.03)	0.050 (0.05)
Showed consent form to others	0.024 (0.04)	0.039 (0.03)	0.032 (0.03)
Past enrollment in a medical study	0.057 (0.03)	0.085 (0.04)	0.071* (0.03)
Has medical insurance	-0.129 (0.06)	-0.056 (0.03)	-0.093* (0.04)
Constant	0.598*** (0.04)	0.681*** (0.09)	0.639*** (0.04)
R-squared	0.283	0.151	0.246
Obs.	136	136	136

^*p<0.05

^**p<0.01

^***p<0.001

Standard errors, in parentheses, are clustered at collaborating study level.

Results

Participants

From 2009 to 2011, 11 studies agreed to collaborate. Three did not reach the minimum enrollment criteria given their slow accrual. Eight collaborating studies ultimately were included in this analysis (Figure 2). We aimed to enroll 4-8 participants for each study arm for every collaborating study. In total, 145 adults agreed to participate. One individual requested their data be removed, resulting in a sample size of 144 participants. Closed-ended CUE questions were available from 144 participants and open-ended questions were available for 140. Participants' mean age was 51.5 years, 67% were female, 54% were white and 42% African American (Table 1). About one-third (31.3%) of participants reported receiving the consent form at least one day in advance of the study, 77.1% reported reading the form on their own, and 53.5% reported having enrolled in research in the past, with similar rates across arms.

Demographic characteristics varied somewhat across study arms; participants in Arm 3 had lower incomes, lower literacy, and were less likely to be employed than participants in Arm 1 (Table 1), though differences were not statistically significant. Across all three study arms, significantly higher numbers of women served as study participants than men (chi-square goodness of fit p <.00).

Using audio recordings from three studies, we compared length of time required for each consent approach. On average, consent procedures in Arm 1 and Arm 2 took the same amount of time (10:00 and 10:05 minutes, respectively). As expected, Arm 3 took longer (12:11 minutes).

Understanding scores

Table 2 presents average and median summary scores across study arms of open-ended and closed-ended items, as well as overall scores. Distributions of open-ended and overall CUE scores were skewed to the left (open-ended skewness=-1.12, p<.00, overall skewness=-.92, p<.00). Supplementary Appendix 3 presents the proportion of correct responses on each item across study arms.

Without controlling for other variables, participants in Arm 3 on average scored 6 percentage points higher on open-ended questions than participants in the control group (study Arm 1) and 3 percentage points higher than participants in Arm 2 (p=0.25). Participants in Arm 3 scored an average of 2 percentage points higher on closed-ended questions than participants in both Arms 1 and 2 (p=0.82). Participants in Arm 3 on average scored 7.3 percentage points higher on open-ended questions related to randomization than participants in the control group and 18.2 percentage points higher than participants in Arm 2 (p = .06) (Supplemental Appendix 3). Although there were significantly more female participants across all study arms, this gender distribution did not appear to affect scores (two-sample rank-sum test, z=-1.34, p=0.18). Uncontrolled for other variables, the demographic and background characteristics associated with statistically significant higher open-ended scores were being a white participant (z=-3.02, p=.00), being employed full time (p=.03), having a higher income (p=.00), having a high reading level (REALM level 4) (p=.00, z= -3.99), and receiving the consent form in advance (z=-2.83, p=.00). The characteristics associated with significantly higher closed-ended scores were reporting good or excellent health (p=.03, z=-2.20) and previously enrolling in medical studies (p=.00, z=-3.22). The characteristics associated with greater overall scores were a high reading level (REALM level 4) (p=.00, z=-2.85) and having previously enrolled in medical studies (p=0.00, z=-3.38).

Regression analysis

We employed OLS linear regression to predict: 1) open-ended CUE score, 2) closed-ended CUE score and, 3) overall CUE score. The sample included only participants for whom we had both open- and closed-ended scores (n=136).ⁱWe created collapsed-category dummy variables for income, race, health status, and REALM reading level (Table 3). “Don't know” responses were considered as missing. Standard errors are robust and clustered at the collaborating study level.

In regression analysis, being in Arm 3 predicted a 7.6 percentage point increase in open-ended CUE score compared to being in Arm 1 (p=.02, Table 3) and a 1.2 percentage point increase compared to being in Arm 2 (p=.81). Higher income level also predicted higher open-ended CUE scores, with participants who reported an annual income of $50,000 or more scoring 10 percentage points higher than participants who earned less than $25,000 yearly (p=.05).

No demographic or behavioral characteristics were significant predictors of closed-ended CUE scores in regression. While the model accounted for 28.3% of the variation in open-ended score, it only accounted 15.1% in closed-ended scores.

Regression analysis of overall CUE scores showed a trend of higher scores for those in study Arm 2 or 3 compared to control, but not with statistical significance. High reading level (REALM) predicted a 7.3 percentage point increase in overall score (p=.03). Past enrollment in medical research predicted a 7.1 percentage point increase in overall CUE score (p=0.04) while having private or Medicaid insurance led to a 9.3 percentage point decrease in overall score (p=0.02).

Discussion

This study tested the feasibility and effectiveness of introducing two consent interventions into eight real world medical and public health studies. A significant methodological goal was to demonstrate that research testing new informed consent interventions can be conducted in ongoing medical and public health studies. Despite modest sample size goals, this may be one of the largest studies to date testing novel consent interventions in multiple ongoing clinical studies.

Despite its modest sample size, this study found a statistically significant improvement in participants' open ended scores on items measuring understanding when adding a bulleted fact sheet and having researchers ask participants a few questions, compared to using standard consent procedures alone; similarly, it found that our interventions improved scores on open-ended questions related to randomization at the significance level of p = 0.06. Given that this study was designed as much to demonstrate the feasibility of introducing and evaluating experimental consent approaches into actual clinical trials as it was to measure effectiveness, further research with larger numbers of subjects and of studies will determine if these interventions ultimately are helpful in improving participants' understanding. Nonetheless, even our modest findings are consistent with the two largest review papers of informed consent interventions concluding that more conversation between researchers and participants and shorter and simpler forms may be the two most promising strategies to improve informed consent.^41,67

It is unclear why such approaches seem to be effective, but we speculate three reasons. First, literature from multiple settings^90-92 concludes there is a limit to the amount of information most people can easily process; consent may be another case where “less is more”. Second, standard consent forms often are not only long, but also do not highlight which study information is most important. The number of pages forms devote to particular sub-topics does not necessarily correspond to their relative importance. We previously reviewed 124 HIV consent forms and found forms spent an average of two pages on confidentiality protections but only an average of 53 words explaining the study was randomized and what that meant.³⁸ Bulleted fact sheets or other summaries allow researchers to identify and highlight key elements for participants. Third, evidence from different arenas^63,67,93 suggests that verbalizing information not only reveals what individuals do and do not know, but also allows more active than passive learning. While federal regulations require only disclosure, and not assessment of understanding, even simple measures like asking participants what they think a study involves may increase the likelihood participants are adequately “informed”.

Multivariate analysis revealed improvement in understanding from our interventions only within open-ended outcome measures. Given one's ability to guess on multiple choice questions, or to be cued by multiple choice responses, many studies from quite different contexts have found that responses to open-ended questions may better reveal true understanding than multiple choice questions.^93-95 Nonetheless, it is important to determine whether closed-ended measures of understanding improve when sample sizes are larger.

Research of this sort is methodologically and logistically challenging. To validly test the feasibility, acceptability, efficiency, and effectiveness of consent interventions, they must be nested within other studies that themselves face tight timelines, recruitment goals, IRB approvals, and staff demands. We are fortunate that so many investigators were willing to collaborate, were eager to learn whether additional strategies could improve their own participants' understanding, and were willing to submit additional amendments to their IRBs for our study. We recognize that not all investigators will be equally willing.

Collaborating with eight different studies ensured that the overall study sample was diverse in terms of design, eligibility, and purpose. This variety not only increased the relevance of our findings, but increases the likelihood that our interventions will be useful in varied settings. This study suggests it is feasible to write a short, bulleted fact sheet for studies ranging from a randomized trial of diets to randomized COPD trial to a genetic registry; similarly, it is feasible to administer the VOICE questions – asking participants to verbalize a study's purpose, risks, and voluntariness and have collaborating staff discuss and correct as needed – in quite different studies. At least in this sample of eight studies, interventions were easily adapted to each study setting.

Consistent administration of consent interventions was important in this study, and we believed there was a high possibility for contamination bias if we pursued purely random assignment across study arms. For example, we did not want study staff unwittingly incorporating elements from the other two arms, such as using short phrases from the bulleted fact-sheet, within the control arm. We aimed to minimize this risk through a sequential allocation design so staff would not need to shift back and forth between study arms. Our hope and expectation was that that the sequentially allocated arms would have achieved the same balance among arms as a randomized trial. Since, however, the arms turned out to not be comparable in other ways, it became necessary for us to also conduct regression analyses. Indeed, Arm 3 had a larger proportion of participants with lower literacy, employment, and income than participants in other arms, which may help to explain why no differences in understanding were seen in bivariate analyses that compared Arm 3 to the other arms. We acknowledge that our failure to randomize may have skewed our results; we attempted to address this through regression analysis that controls to some degree for non-random allocation to study arms.

Relatively little information is available on how research staff implement IRB approved consent procedures; this study aimed to capture this, as well as whether interventions were implemented as intended, through audio recording consent. When multiple collaborating research staff reported that they changed how they implemented standard consent procedures when the recorder was on, however, we discontinued audio recording, given the importance of measuring interventions' effectiveness against a true standard. Indeed, our findings may be conservative since some proportion of our “standard” consent interactions may have been enhanced due to the presence of the recorder.

Our original proposal described testing a streamlined consent form against standard consent (with VOICE added as the second intervention). That our IRB required all participants to receive the full, approved consent form for collaborating studies was a limitation, but perhaps understandable in our current regulatory climate where institutions have been sanctioned for what OHRP considered omissions in consent.^96-100 Such requirements, however, must be reconciled with evidence that participants' understanding is often limited when exposed to current consent approaches^4,6-8,9,20 and that shorter forms do better or are equivalent to longer ones.^{19,22,44,50-54,57,101,102} As more trials move from settings with dedicated research staff to clinical settings with fewer research supports, it will be important to measure how patients' understanding of a study, including whether it is randomized, compare when exposed to either a fact sheet or other streamlined approach vs. longer consent forms and procedures. Nonetheless, since long forms and legal boilerplate may be required by IRBs at least in the short term, knowing we can improve participants' understanding – even to a small degree --by adding a fact sheet and a few conversational questions may be valuable. Importantly, these took only an additional two minutes on average. Interviews with collaborating staff after data collection suggest that many collaborating studies requested continued use of the bulleted forms when our study was over.

Conclusion

Studies repeatedly document that participant understanding of research is incomplete, including of important concepts such as its purpose or that participants are randomized. This study suggests that a simple and widely transferable intervention combining a bulleted fact sheet with a few open-ended conversational questions as supplements to traditional consent is a promising approach for improving participants' understanding in multiple, varied research projects compared to standard consent. We fail to honor the foundational goals of U.S. human subjects regulations if the mechanisms by which we conduct informed consent are inadequate.