Improving the Proficiency of Research Consent Administrators

Abstract

Objective:

To describe the development and testing of a module to improve consent administrators’ skills when obtaining research consent from culturally and linguistically diverse and low literacy populations.

Design:

Development and psychometric testing of video module including community vignettes.

Methods:

Following initial content, face, and construct validity testing by experts, a field trial was conducted with pre‐ and postknowledge tests and satisfaction surveys completed by 112 consent administrators.

Results:

Mean score out of a possible 10 on pretest was 8.6 (±standard deviation [SD], 1.55) and on posttest was 9.1 (±SD, 1.2; paired t‐test 95% confidence interval of difference: –0.18 to –0.88; two‐tailed p = 0.003). The average years of experience with obtaining consent was 6.42 years (range: 0–35), but years of experience was not significantly associated with either pre‐ or posttest scores (p = 0.82 and 0.44, respectively). Most user evaluations were positive, although suggestions for improvements were made.

Conclusion:

Although pretest scores were relatively high, training needs of research consent administrators for consenting diverse and low literacy populations may be unmet. We urge that institutional review boards, researchers, policymakers, educators, and bioethicists address the training needs of research consent administrators and we offer this training module as one potential resource and adjunct to such training.

Introduction

Despite increasing regulatory efforts to protect individuals who participate in research and to ensure that they are fully informed, data have consistently demonstrated that participants are often unable to recall important elements of the research such as its purpose, the potential risks, or what is expected of them, even after they have signed consent forms and discussed the protocol with research staff. The misunderstanding worsens when working with immigrant populations where low literacy and limited English proficiency further hinder individuals’ understanding of the research process. In a recent systematic review, Nishimura et al. found that while a number of interventions to improve participant comprehension were effective, results were inconsistent.1 A common theme, however, seemed to be the importance of the one‐on‐one interaction of a potential participant with the person obtaining consent (i.e., the consent administrator).

Several years ago, we developed and tested a formal training program for research personnel who obtain consent from individuals to participate in a research study (i.e., consent administrators). Although the program was successful in improving the consent process, it was expensive, time consuming, and not practical in many busy settings.2 Therefore, as part of a larger National Institutes of Health (NIH)‐funded study, we developed a short online video training module for research personnel designed to assess participant understanding with an emphasis on hearing from the community about their thoughts, beliefs, and feelings about research and incorporating principles of health and cultural literacy and the use of the “playback” method designed to assess participant understanding. This focus was of particular importance to us as we serve a predominantly immigrant Latino population with low levels of health literacy and a high percentage of participants with limited English proficiency. The aims of this paper are to describe the development and testing of the module to enhance the knowledge of research personnel regarding the importance of health and cultural literacy in the consent process.

Methods

Conceptual underpinnings

Using the Precede–Proceed model of health promotion education as the framework,3 we undertook the development of an educational module to improve culturally appropriate, health literate, and comprehensive informed consent for research (Figure 1). We focused on assuring that the training module considered predisposing (knowledge about the fundamental required elements of informed consent), enabling (communication skills, with specific focus on health literacy and cultural understanding), and reinforcing (feedback techniques) factors that would provide consent administrators with the tools essential for a successful informed consent interaction. The purpose was to increase the user's knowledge of both health and cultural literacy and to provide instruction on how to assess comprehension as a critical part of the informed consent process.

Figure 1.

Components of the Precede–Proceed model³ used as conceptual underpinnings to train consent administrators.

Development of the module

The project to develop and test the module was approved by the Columbia University Institutional Review Board. In several team meetings, we determined that a video format would be the most feasible and efficient method to educate consent administrators, and defined the following criteria as essential characteristics of a training tool: interactive, self‐directed, possible to complete in one short session, and available online at the convenience of the user. First, we reviewed the current literature on communication, health and cultural literacy, and informed consent for research. We used the principles identified in the literature as well as the Institute of Medicine's Building the Foundation for Integrating Health Literacy, Disparities Reduction and Quality Improvement in Health and Health Care,4 and the Agency for Healthcare Research and Quality's Health Literacy Universal Precautions Toolkit,5 which guided the health literacy content of the video presentation. To guide the communication and comprehension component, we used the Kalamazoo Consensus statement6 and the relevant literature,7, 8 which described the essential elements of communication, with special consideration of vulnerable and minority populations. We also reviewed the literature on video module development and self‐paced learning.

Because we were concerned that teaching cultural competence in a video format might risk reinforcing the stereotyping of people and cultures, we recorded a series of vignettes in which community members discussed research experiences and attitudes regarding research, and cultural considerations about participation from their individual perspectives. These vignettes were integrated into the training module.

The module had four learner objectives: to (1) understand the roles that health literacy, language, and culture play in the informed consent process; (2) provide practical tools to bridge the gap in health literacy, language, and culture during informed consent; (3) recognize when a miscommunication has taken place during consent; and (4) learn how to assess participant understanding using the playback technique. The module was organized into the three steps identified as critical for a comprehensive informed consent: Assessing readiness, Conveying information, and Ensuring understanding, and used the acronym “ACE” throughout the video presentation to help participants remember each step.

The explanation of each step was then broken into three parts; (1) a slide framing the issue, (2) vignettes of community members explaining the importance of the issue from their own perspective, and (3) a set of practical suggestions for improving consent. Following initial development, the draft slides were presented to a group of graduate students enrolled in a health literacy course at the study institution's School of Public Health. The slides were then evaluated by 12 members of the institutional review board (IRB) staff to examine content and construct validity and clarity. Several comments and suggestions were made and the slides were modified based on this feedback.

Finally, an evaluation tool, the Process and Quality of Informed Consent,8 previously developed and tested by two of the authors, was adapted and used interactively by the viewer to rate two scripted consent encounters at the end of the module. Using a split screen, the users watch the video on the left side of the screen and select answers from the rating tool on right. The tool has 20 items identified as critical to a comprehensive consent encounter and a click circle for “done,” “not done,” or “not applicable.” A sample item, for example, was: “The consent administrator uses nonmedical language when describing the study.” The scripted videos were purposefully designed to include and exclude critical features of informed consent so that they contained some aspects of the consent that were done well and some aspects not done or done poorly. The entire module could be completed in about 25 minutes.

Field test of module

Following initial development and testing of the module, a field trial was conducted at a large, research‐intensive academic health center in Manhattan with six IRBs responsible for the oversight of approximately 5,100 active protocols. Participants who tested the module were individuals who conducted informed consent sessions at the study institution or individuals attending a workshop at a regional IRB conference. Inclusion criteria for the first group were: a member of a research team who was responsible for obtaining consent but had no other formal education in research ethics beyond the required human subjects and Health Insurance Portability and Accountability Act training. Eligible participants were consent administrators, clinical research coordinators, project managers, or principal investigators. Participants attending the workshop were research staff, IRB administrators, members, or chairs primarily from the New York City metropolitan region and environs.

Recruitment and study procedures

Initially a generic email was sent from the IRB staff to invite consent administrators at the study institution to express interest in participating in research regarding the consent process. Those who expressed interest were contacted by a member of the IRB staff, who discussed the study with them and obtained verbal consent. Workshop participants were provided with an information sheet, which was reviewed with the group at the beginning of the session. All volunteers watched the video module and completed the pre‐ and postknowledge test and satisfaction survey (described later).

Evaluation of video module

Before and after watching the video, participants completed a five‐question knowledge test which was based on the objectives of the training module and focused specifically on health literacy components of a written consent form (Table 1). Participants at the study institution and the workshop were also asked to rate the module in three domains: length of the video (too long, too short, just right), the amount of content presented (too much, too little, just right), and how interesting the video was (boring, interesting, just right). They were also asked whether they learned anything new and whether they felt any content was missing (if so, what).

Table 1.

Pre‐ and postknowledge test items (maximum score: 10).

Item
Which of the following is NOT a component of health literacy? (select one):
1. Able to obtain health information
2. Able to understand health information
3. Able to make decisions
4. Able to read *
Recommendations to improve health literacy in written documents include (check all that apply):
1. 5th grade reading level *
2. Use of graphics *
3. Use of different fonts and clip art
4. Use of plain language *
People might feel comfortable speaking one language but reading in another as it relates to health information.
1. True *
2. False
Which of the following elements might have an impact on a participant's willingness to enroll in research? (check all that apply):
1. Level of trust in the research process *
2. Level of trust in the researchers *
3. Religious practices *
4. Cultural beliefs *
Which of the following is the most accurate evidence that a participant understands the consent process? (select one):
1. Signs and dates the consent form
2. Is able to describe the main components of the research study *
3. States they have no questions about the study
4. Nods and gives verbal agreement that they understood the study

*Correct response.

Descriptive statistics, using SPSS (Chicago, IL, USA) software were calculated. The Student's t‐test, Spearman's correlation coefficients, paired analysis of variance, and nonparametric statistics were used as appropriate.

Results

A total of 112 participants completed the pre‐ and posttests and video evaluation. Mean score out of a possible 10 on pretest was 8.6 (±standard deviation [SD], 1.55) and on posttest was 9.1 (±SD, 1.2) (paired t‐test 95% confidence interval of difference: –0.18 to –0.88; two‐tailed p = 0.003). The average years of experience with obtaining consent was 6.42 years (range: 0–35), but years of experience was not significantly associated with pre‐ or posttest scores (p = 0.82 and 0.44, respectively).

The majority (79.4%) of participants reported that the length of the module was “just right”; 3.9% rated it as too short and 16.7% as too long. In terms of the amount of information in the module, 60% rated it as “just right,” 12% as too much, and 28% as not enough. About one‐fourth (27.9%) of participants listed one or more things they had learned from the module: in rank order these included use of the playback technique (i.e., asking the potential subject to state in their own words what they understood about the study), elements and importance of health literacy and culture, and assessing potential subject's readiness to hear about and understand the research. A similar number of individuals suggested items they thought were missing from the video, again in rank order: emphasizing the difference between clinical care and clinical research, dealing with families, and importance of data confidentiality. An additional one‐third of participants made other comments. Most were positive, but five (4.1%) reported that some component of the video was boring or that they did not feel it appropriate to do this activity during a workshop.

Discussion

Overall, scores of pretest knowledge regarding the consent process were high. Nevertheless, despite the fact that many individuals who agreed to test the module had considerable experience with obtaining research consent, an average of more than 6 years, there was not only a significant improvement in their knowledge after taking the module, but there was also no correlation between years of experience and pre‐ or posttest scores. Hence, it is clear that simply repeating the consent process over time in the absence of additional training will be unlikely to improve its quality. Over the past decade, we have examined the consent process and institutional review board policies in a number of institutions, finding that the process continues to be nonstandardized and inefficient, despite increased regulatory scrutiny.9, 10

Ironically, although about 16% of participants reported that the module was too long, more than one‐fourth also reported that it did not contain sufficient information. Educators face major challenges to accurately assess learning needs and effective teaching strategies, always with time constraints. While this module incorporated several important educational principles such as including vignettes from community members and providing opportunity for participant interaction and feedback, the concepts of health literacy seemed to be somewhat new to a number of viewers and their suggestions related primarily to the need to focus on those aspects of the consent process specific to that topic. This is extremely important if we are to ensure understanding of the research process among vulnerable populations and individuals with low health literacy and limited English proficiency, and aim to close the gap in disparities of health.

In this project, we also found that individuals obtaining consent for research may be lacking in some basic, straight forward knowledge regarding how to enhance comprehension in potential participants, and how to assess their understanding and readiness to give truly informed consent. We urge that IRBs, researchers, policymakers, educators, and bioethicists address the training needs of research consent administrators and we offer this training module, despite its limitations, as one potential resource and adjunct to such training. This module will be made available open access through Columbia University's clinical and translational institute, Irving Institute for Clinical and Translational Research (http://irvinginstitute.columbia.edu/), and on our Clinical Trials Office Web site, is being included as a training module in our institution's human subjects research training center, and is available to readers upon request.