A Cognitive Approach for Design of a Multimedia Informed Consent Video and Website in Pediatric Research

Abstract

Objective

Poor participant comprehension of research procedures following the conventional face-to-face consent process for biomedical research is common. We describe the development of a multimedia informed consent video and website that incorporates cognitive strategies to enhance comprehension of study related material directed to parents and adolescents.

Materials and methods

A multidisciplinary team was assembled for development of the video and website that included human subjects professionals; psychologist researchers; institutional video and web developers; bioinformaticians and programmers; and parent and adolescent stakeholders. Five learning strategies that included Sensory-Modality view, Coherence, Signaling, Redundancy, and Personalization were integrated into a 15-minute video and website material that describes a clinical research trial.

Results

A diverse team collaborated extensively over 15 months to design and build a multimedia platform for obtaining parental permission and adolescent assent for participant in as asthma clinical trial. Examples of the learning principles included, having a narrator describe what was being viewed on the video (sensory-modality); eliminating unnecessary text and graphics (coherence); having the initial portion of the video explain the sections of the video to be viewed (signaling); avoiding simultaneous presentation of text and graphics (redundancy); and having a consistent narrator throughout the video (personalization).

Discussion

Existing conventional and multimedia processes for obtaining research informed consent have not actively incorporated basic principles of human cognition and learning in the design and implementation of these processes. The present paper illustrates how this can be achieved, setting the stage for rigorous evaluation of potential benefits such as improved comprehension, satisfaction with the consent process, and completion of research objectives.

Conclusion

New consent strategies that have an integrated cognitive approach need to be developed and tested in controlled trials.

Graphical abstract

Introduction

For research studies that would enroll children as participants, the conventional informed consent process for adult participants is replaced by a combination of parental permission (similar to the conventional informed consent process) plus children’s assent (their active agreement to participate after receiving basic study information they can understand) [1]. Both parental permission and children’s assent must be obtained before the child can participate in any research activities and either the parent or child can end the child’s participation in the research by withdrawing their permission or assent. The conventional process typically involves the parent reading a paper consent form document followed by a face-to-face discussion with the researcher to answer questions about the research and signing the consent document.

A cornerstone of the parental permission and assent process is the provision of key elements of informed consent (Table 1) that enable the parent to make a fully informed and voluntary decision about permitting their child to participate in the research based on careful consideration of the study purpose and procedures, the foreseeable risks and potential direct benefits of participation, and the protection of their children’s privacy and confidentiality. Institutional human subjects protection committees review informed consent documents diligently to ensure that the key elements of informed consent are presented accurately, fairly and in plain understandable language. Yet, substantial research shows that most prospective research participants do not read these documents carefully, do not comprehend their contents completely, and forget what they have read and formally acknowledged [2–5]. Inadequate comprehension or retention of key elements of research informed consent may be associated with poorer enrollment rates, subsequent noncompliance with the research protocol, and decisional regret when study procedures or time commitments are more complex than originally expected at the time consent was provided [6, 7].

Table 1.

Basic Elements of Informed Consent^*

A statement that the study involves research, an explanation of the purposes of the research and the expected duration of the subject’s participation, a description of the procedures to be followed, and identification of any procedures which are experimental;

A description of any reasonably foreseeable risks or discomforts to the subject;

A description of any benefits to the subject or to others which may reasonably be expected from the research;

A disclosure of appropriate alternative procedures or courses of treatment, if any, that might be advantageous to the subject;

A statement describing the extent, if any, to which confidentiality of records identifying the subject will be maintained;

For research involving more than minimal risk, an explanation as to whether any compensation and an explanation as to whether any medical treatments are available if injury occurs and, if so, what they consist of, or where further information may be obtained;

An explanation of whom to contact for answers to pertinent questions about the research and research subjects’ rights, and whom to contact in the event of a research-related injury to the subject; and

A statement that participation is voluntary, refusal to participate will involve no penalty or loss of benefits to which the subject is otherwise entitled, and the subject may discontinue participation at any time without penalty or loss of benefits to which the subject is otherwise entitled.

^*From: Health and Human Services 45 CFR 46:116 General requirements of informed consent (https://www.hhs.gov/ohrp/regulations-and-policy/guidance/checklists/)

Efforts to improve outcomes of the informed consent (for adults) and parental permission and assent (for children) processes include a wide range of efforts (teach-back methods, extended discussion, and modified documents with simplified language or revised layout) to improve the consent forms or discussion, yet these efforts have not led to robust or lasting improvements in participants’ comprehension or retention of basic study information [7–10]. Similarly, the recent emergence of electronic multimedia consent documents has led to improved satisfaction with the research consent process, but it has less consistent impact on participant comprehension or retention of information [10–17].

One reason for the disappointing outcomes of the typical process for obtaining informed consent or parental permission and child assent for research participation may be that the conventional consent documents, whether in hard copy or electronic multimedia formats, as well as the characteristics of social interaction during the consent process, may not have been developed to actively incorporate or capitalize on basic principles of human cognition and learning. Systematic reviews have found little evidence that learning principles have been intentionally incorporated into consent form development [12, 18]. Careful planning and construction of the research consent process with deliberate application of basic learning principles to the design, structure, and flow of the research consent interaction may yield better outcomes of the consent process and, consequently, better engagement of research participants in the completion of their commitments to the researchers and the research project [7, 12, 18].

The project reported in this paper had basic learning principles systematically incorporated into the practical design and implementation of a multimedia research informed consent process for an ongoing study of parents and adolescents (12 to 17 years old) who are considering participation in an actual asthma clinical trial. Development of the multimedia consent was based primarily on Mayer’s principles for multimedia learning [19, 20] and was intended to enhance learning of the study related information while reducing cognitive load [21–23] in our research participants and their parents, who were expected to have varying levels of health literacy. The goal for developing a theory-driven multimedia consent tool was for assessing participant comprehension of consent information when delivered within a multimedia platform versus conventional written consent documents. To this end, we incorporated five learning principles, Sensory-Modality view, Coherence, Signaling, Redundancy, and Personalization, that we could reasonably incorporate into a multimedia 15-minute consent video and study website.

The objectives of this paper, then, were: 1.) To describe the procedures used in designing and building an electronic multimedia platform for obtaining parental permission and child assent for an ongoing, controlled trial evaluating asthma medication regimens; and 2.) To illustrate how the five basic learning principles were actively incorporated into the multimedia consent platform.

Material and Methods

This project was designed to develop an informed consent video and related website material suitable for parents and adolescents aged 12 to 17 years who are considering participation in an actual asthma clinical trial. Our goal was to evaluate parent and adolescent comprehension of the research study and satisfaction with the consent information after reviewing the video and website material. A multidisciplinary team of researchers, human subjects professionals, psychologists, bioinformaticians and programmers, web designers, and video producers were assembled for development of the video and website. Nemours is a pediatric health system and all team members had expertise in working with families and adolescents for medical issues. The project was approved by the Nemours Institutional Review Board and all parents and adolescents provided permission and assent, respectively.

Design Principles

The multimedia consent video and website was grounded in five specific theoretical principles of multimedia learning [19, 20] that are difficult or impossible to employ in paper consent forms. These principles are described below.

The Sensory-Modality view

Consistent with a learner-centered approach, this strategy involves the use of two or more sensory systems and takes information processing into account. Sensory-Modality is consistent with the cognitive theory of learning [24, 25] which stipulates that people have separate information processing channels for auditory and visual processing. In other words, learning is not simply a one-channel process. Sensory-Modality is learner-centered when applied to multimedia design and is supported by an underlying model of working memory [26] which allows learners to manipulate new information for learning through sensory input, such as, auditory and visual [20].

Coherence

This principle posits that learning improves when irrelevant words and pictures are excluded [21]. By streamlining information, unnecessary graphics, words, or sounds are excluded from the learning process, thus, avoiding competition for cognitive resources [21]. Extraneous information, such as background sounds or other graphics, contribute to the demand on working memory, which can lead to cognitive overload, especially in novice learners [19]. Using the Coherence strategy minimizes the demands on working memory and is especially helpful for learners who have less knowledge about the topic [27] or have lower capacity for processing information [28].

Signaling

This principle operates on the premise that individuals learn better when cues that highlight the organization of the essential material are added [21]. For complex learning such as clinical trial consent information, supplementary details must be included (e.g., side effects of study-related medications). Because eliminating supplementary information is not always possible, signaling provides a method to organize information and direct the learner as needed. Signaling reduces unnecessary processing by guiding attention to key elements and it may facilitate review by enabling users to search more efficiently for information that was previously read. This strategy is considered useful when the learner has lower literacy and when the lesson may contain a significant amount of material. Signaling may also include pre-training, which has been shown to increase retention toward complex tasks [29].

Redundancy

This principle asserts that individuals learn better from graphics and narration than from graphics, narration, and printed text (printed text is thus, redundant to narration or graphics) [19]. Redundancy creates unnecessary and excessive processing to accommodate and compare incoming printed and spoken word, therefore potentially increasing the cognitive load for learners [21, 30]. However, redundancy can be helpful by integrating information in certain situations, such as when spoken text is either presented before printed text (not concurrently), or when no graphics are present and verbal segments are relatively brief [31].

Personalization

The Personalization principle maintains that learning may be enhanced with use of a visible author or on-screen character/narrator who can guide the learner through the information [19]. Personalization allows multimedia learning to occur within a conversational style between the person providing the information (e.g., visible author) and the prospective research participant, which can ease the cognitive load for the learner. This strategy may employ social learning cues, similar to the standard consent process which involves a face-to-face conversation between a research team member and the potential research participant. While the use of a pedagogical agent or narration to aid learning has been debated [32–34], there is evidence to suggest that an agent, when used within other established learning principles, likely will not negatively impact and may benefit learning [35].

Conceptual Design of the Video and Website

Our team spent a considerable amount of time at the outset reviewing learning theory concepts that are directly related to multimedia learning [19, 20]. Multimedia and e-learning applies cognitive [25] and social learning theory [24] toward multimedia learning strategies. We focused primarily on principles or strategies that would apply to our anticipated patient population (parents and their adolescent child) and meet our goal of educating families about the clinical trial. We anticipated our “learners” to be unfamiliar with research procedures, to be of varying age, and to be of varying health literacy or reading ability.

We selected empirically supported multimedia learning strategies based upon what would intuitively work within our model of presenting a learning video and associated website material. Each learning concept was discussed with the ethicists, psychologists, bioinformatics, video production, and web-team individuals during development and was incorporated as a blend of theory and practicality. In addition, some design decisions were not strictly based upon learning principles but were requirements following our Office of Human Subjects Protection (OHSP) and Institutional Review Board (IRB) review. This project focused primarily on five learning strategies that included Sensory-Modality view, Coherence, Signaling, Redundancy, and Personalization.

Ethical and Human Subjects Protection Considerations

The initial step with our institutional OHSP was to discuss how the standard, in-person parental permission and assent process [1, 36] could be achieved acceptably with an electronic multimedia platform.

This format change required IRB review and approval of a waiver allowing certain departures from the conventional parental permission and assent process. These changes included replacing the paper consent document with the video and multimedia website; having a person-to-person telephone interaction versus a face-to-face interaction with the researcher; and documenting informed consent electronically rather than by providing an ink signature on paper. The latter had to meet federal regulatory requirements for electronic informed consent [37] and ensure the security of protected health information as required by the Health Insurance Portability and Accountability Act (HIPAA). Our institution already had in place an electronic medical record (Epic Systems, Verona, WI) and patient portal (MyChart®) that could be used for obtaining the electronic consent signature that is compliant with federal regulations.

We met with OHSP informally on several instances for feedback as the ideas for the video and script were being developed. Specific suggestions during these meetings included using quizzes to engage the participant throughout the video, using an image of a baby boy and baby girl for the concept of randomization, keeping the video short, and editing language to achieve readability ≤ 8th grade level. While these features were not necessarily directly related to the learning principles we elected to use, they reflected the experience of the OHSP on appropriate parental permission and pediatric assent processes. The OHSP also suggested that a convened meeting of IRB members review the consent video script prior to beginning production to ensure that any IRB required changes would be included. The IRB had previously reviewed and approved studies that proposed electronic informed consent processes, but this was the first instance in which IRB members actively collaborated with a research team that proposed to develop and build a multimedia electronic platform for parental permission and assent. The material was submitted to the IRB iteratively over 15 months. However, most of this period was spent on development and editing the multimedia website as the IRB response to each submission was typically less than 10 days. The submissions and timeline included: a description of the telephone consent process and PowerPoint presentation of the overall study concept (month 1); manual of procedures for consent process (month 6); a rough cut of an example video to present the narrator tone of voice, pacing, and scene selection; the video sidebar material which supplemented the video content on the website; the video script with scene descriptions; participant flow for consent process through Epic and MyChart; HIPAA language; draft of website screen shots (month 10); revised consent video script with scene descriptions; quiz questions and answers included in the consent video (month 11); parental permission and assent documents that are sent via Epic and MyChart (month 12); final video and website submission (month 15).

Website Structure

The bioinformatics team developed the architecture that housed the video. The first step was to create the website wireframes which is a schematic diagram of the website components and function. A programmer was hired specifically to work on the website development.

The multimedia format was designed to convey complex study information in an understandable and visually appealing manner that was suitable for promoting learning in low health literacy adolescent participants and their families. In order to achieve this, the web team and author, HA (a pediatric psychologist researcher), suggested dividing the video into five short 3- to 4-minute segments each embodying a single concept (About this Study, Participating in the Study, Possible Benefits and Risks, Payment for Participating, and Protecting your Privacy) to facilitate assimilation of material. These short durations would be familiar to adolescents as the average length of a YouTube video. At the end of each section, 2 to 3 multiple-choice questions were presented which must be answered before proceeding to the next section. We included quizzes because adolescents prefer quizzes as a means for interaction [38], quizzes provide an opportunity to “teach to goal” [39] or improve comprehension of consent information, and quizzes are another suggested e-learning strategy [19].

To ensure that information was presented in a logical progression, each video section must be viewed in sequence before the viewer can progress to the next section. Research Electronic Data Capture database (REDCap) was used to electronically capture data from the video and actions on the website that included for example, who watched the video, how long an individual watched a section, and what sections of the website were accessed [40]. The researchers were able monitor each participant through REDCap to ensure all sections of the video were watched before the person-to-person telephone interaction. This design ensured that all the necessary material was reviewed by the potential participant in order to be knowledgeable about the study prior to the decision to participate in the trial.

Because the video was intentionally limited in length to appeal to the attention span of the viewer, it could not contain all the detailed information that is typically present in a paper consent document. In order to provide the opportunity for the viewer to access additional information, the bioinformatics team created a sidebar on the left of the video screen with tabs containing pertinent study information to supplement the video, such as, “When to get emergency care” and “What is expected if you decide to join the study?”.

Video Script and Scene Development

The principal investigator (author, KB) met with an institutional video expert who regularly developed videos and other content for pediatric patients and their parents. The principal investigator initially drafted a conventional written parental permission document (a 13-page single spaced typed document). The video expert re-wrote the document into a conversational script to be read by a narrator used in the video. Reviews and revisions offered by the team members from the start of editing to the final IRB approved script took approximately five months. The video expert arranged for and worked closely with an outside professional video director for the video production, assisted in selection of the narrator (a local actor talent which would be a consistent feature as the “an agent / visible author” in the video), actors (parent and adolescent), shoot locations and props, and editing the video. Care was taken to select a narrator who was an attractive young adult of mixed racial descent that was expected to appeal to the anticipated racial and ethnic diversity of the adolescent and parent viewer. Selection of a racially and ethnically mixed narrator was encouraged by our OHSP to reflect the racial and ethnic diversity in the geographic areas in which the clinical trial was being conducted.

Focus Group Feedback

Prior to launching the video and website, the institution’s Marketing and Communication expert conducted in-depth one-on-one observation and interview with two parents and their adolescents (ages 12 and 13 years) using the video and website. The purpose was to: 1.) uncover usability issues; 2.) understand how participants would use the site to understand the study information; 3.) learn if the website design is effective in engaging the viewer; and 4.) make final recommendations on site design and functionality. The expert was specifically looking for how much effort the parent and adolescent expended to navigate the information presented with the expectation that the webpages should be self-evident and self-explanatory. The expert observed the parent and adolescent and asked questions about their choices as they navigated the web pages. Based upon these interviews, the expert suggested changes to text content; reduction in text content on each web page; enhancement of color contrasts for better visualization; renaming sidebar tabs to improve comprehension of material contained within the tab; and shortening the horizontal width of windows to make them easier to read.

Final Testing

After all content changes were made, the bioinformatics team tested the video and website on different web browsers, operating systems, and mobile devices to ensure complete functionality.

Results

A formal evaluation is in progress to assess the effects of the multimedia process for obtaining parental permission and assent for participation in an ongoing pediatric asthma clinical trial. We present below the products of this collaboration to design and build the platform and will report the results of our outcome evaluation in a future paper.

Operation of the Video and Website

A link to the multimedia video and website and a cut of the video are available in the supplementary material. The video and website were accessed through a login site that was only available to parents and adolescents who were emailed a link to the website from the study team; the live multimedia website was housed on a private YouTube site and, thus was not freely searchable on the internet. The login feature enabled electronic tracking for who watched the video, how long each section was watched, sidebar tabs accessed, and results of the quiz questions. The website login screen provided the title of the study (and a short user-friendly study title), study contact information, and an image of active adolescents. After logging in, the viewer encountered a summary (Figure 1) that informed the viewer that the video had five sections, the video can be watched repeatedly, there were quizzes after each section, the sidebar contained additional information, and what to do after watching the video. Radio buttons were present to capture who was watching the video (parent/guardian, child, or other) and must be selected before the video would start. A frequently asked questions tab contained information to troubleshoot watching the video. We intended viewers to watch the video in one sitting, which would facilitate understanding of study participation.

Figure 1.

Welcome page of the consent video website.

The welcome page orients the viewer to the layout and steps for viewing the video and website material.

Tabs for the five sections of the video were presented above the video screen and changed to a darker color when viewing that section. The layout of the five video sections was identical, the sidebar material was on the left and the final tab of the sidebar was “Test yourself”. The tab content was specific for each section of the video. In addition, the tab changes color when a relevant section of the video was reached to prompt the viewer to select the tab to read additional information. The video had a bar at the bottom to indicate the amount of time the video segment had been watched and the total time of the video segment.

Learning Theory and Principles Used in Multimedia Website Development

Learning theory and principles were systemically employed with a practical approach to achieve our aim of improving study comprehension by parents and adolescents. We were mindful to avoid selection of theoretical concepts that overlapped conceptually. Specific concepts that we intentionally applied included Sensory-Modality view, Coherence principle, Signaling principle, Redundancy principle, and Personalization principle. Table 2 presents these principles along with specific examples of how they were incorporated into the video consent website.

Table 2.

Learning theory principles used in multimedia consent website development^*

Learning Theory Principle:	Examples from the Multimedia Video and Website:
Sensory-Modality View	Website includes visual information as well as a video with auditory narration to engage visual as well as auditory learners.
Coherence Principle	Narration is simply stated and script was assessed for comprehension on a 6th grade level. Simple graphics are used to enhance learning.
Signaling Principle	The consent video provides an overview/preview of sections included in the video. Sidebar material is organized according to various sections of consent/assent. Each sidebar title signals what information is included. The narrator is a human signaling device, providing cues as to what information is included in the video and where to obtain answers to questions. The sidebar lights up when the viewer comes to a point in the video that is relevant for the sidebar material.
Redundancy Principle	Narration and printed text (e.g., sidebar material) are used, yet kept separate in order to reduce burden of overlapping visual and auditory input. When text and/or graphics are used in combination with auditory information, the author is narrating the printed text or graphic information.
Personalization Principle:	Actor with no notable accent was hired to narrate script with conversational language.

^*Adapted from References 19, 20, and 30.

The Sensory-Modality view

This strategy was applied by combining visual aids and narrated text to teach potential study participants and their parents the necessary information for human-subjects informed consent. The sensory-modality view allows the learner to use visual and auditory channels to comprehend information and is consistent with the cognitive theory of learning, noting the separate channels for processing visual and auditory information [20]. An example within our consent video is the use of a verbal information (provided by the visible author) along with visual text or graphics. This process was employed throughout the consent video, including describing the study purpose, protocol and benefits to patients.

Coherence principle

We anticipated many potential participants would be novel learners with no prior history of engaging in medical research. We were also aware that individuals of varying age, educational, socioeconomic, and cognitive capacity status would be participating in the study. In order to maximize the likelihood of pediatric patients and their parent/caregivers retaining the consent/assent information, we employed the use of coherence to streamline information to avoid unnecessary text/graphics; no additional graphics or animations were included in order to reduce the processing required (Figure 2). As presented in Figure 2, we had a title in the video screen to orient the viewer to the information below, only two items were presented, and the sentences were kept short. To maintain the coherence principle throughout the video, we presented a simple visual layout, kept our visible author/narrator in one physical location within our medical center throughout the video shoot, and avoided background music/sounds during instruction.

Figure 2.

Coherence principal.

The Coherence principal as depicted in the consent video was used to avoid unnecessary text and graphics in order to maximize the likelihood for participant and parent retaining study information.

Signaling principle

In the consent video, signaling was used in several ways (Figure 3). The first segment of the consent video, “About this study” provided an overview/preview of sections included in the video. The sidebar material was organized according to various sections of consent/assent. Each sidebar title provided a signal for what information was included within the tab. The sidebar served as a signaling device, which consequentially created organization and coherence to the substantial research consent information, therefore potentially reducing the cognitive load for our novice learners. We considered the narrator as a human signaling device, who provided families with cues as to what they can expect to learn from the video and how they can find answers to their questions. Additionally, signaling was employed by having the sidebar button light up (or blink for the quizzes) when the viewer comes to a point in the video that was relevant for the sidebar material.

Figure 3.

Signaling principle.

The Signaling principle is represented by the visible author and sidebar for reinforcement of information. The visible author guides the viewer through the video information. The sidebar tab is highlighted when a relevant section of the video is reached and additional information can be accessed in the sidebar. The video tabs are highlighted for the section the participant/parent are currently viewing.

Redundancy Principle

We incorporated the redundancy principle recognizing that consent/assent information is complex. To reduce cognitive load burden on families, we relied primarily on narration and images, being mindful in video development to keep video segments brief (approximately 3 to 4 minutes for each of the five sections). Striking a balance between the Sensory-Modality view and the Redundancy principle, we used joint text and narration in limited areas (for example, information in the sidebar tabs is in written format but also may be narrated), when reiterating key elements of consent.

Personalization Principle

We incorporated the use of a “visible author” [19] via a professional actor who narrated throughout the consent video (Figure 3). We adhered to the personalization principle to guide selection of our hired actor by selecting an actor who was of mixed racial and ethnic descent, who spoke with no notable accent, and was provided a script developed by our web team that allowed conversational language in order to appeal to the audience of parents and adolescents. Our visible author referred to viewers as “you “or “your child,” which is also recommended in lieu of a less personal approach (e.g., “the participant”). The female actor stood on a main floor of our medical center, spoke clearly and concisely based upon script and looked directly at the viewer. Great care was taken to ensure that the author’s spoken word was not only clear and concise but also void of unnecessary complex terminology. Our goal with our “visible author” was to use a conversational style and language that would be consistent with our intended audience, which was to include parents and adolescents of varying racial and ethnic, socioeconomic, and health literacy backgrounds.

With regard to the overall website and video design, we used moderated control, course map, and web-links to provide additional learner control [19]. Based upon the expected wide educational level of the parent and adolescent, we made choices to allow control to viewers by permitting the viewer to watch the video repeatedly in its entirety or sections. We incorporated a course map through a combination of the author’s script and associated on-screen text as well as similarly phrased tab’s (links) at the top of the video. Each of these strategies are deemed useful for novel learners [19] which fits our target audience.

Additional multimedia learning strategies were applied to the script content and scenes to promote comprehension of the narrator’s verbal information. We applied the concept of segmenting by dividing the consent information into five video sections and also by having tabs in the sidebar to break down information in each video section into smaller portions. Another concept, rehearsing in working memory, was used by having a brief multiple-choice quiz at the end of each section to review the material just presented and the correct response consistently provided to reinforce learning [19]. Regardless of their response, the correct answer appears in a text block, therefore teaching to the goal [39] of consent comprehension or rehearsing depending on the viewer’s need. While we included visual information to reinforce learning, graphics were used selectively and in simplistic form. For example, to demonstrate how randomization is conducted within a clinical trial, we used a simple graphic of a boy and girl baby, which was accompanied by an audio explanation (“Randomization is like the sex of a baby for a woman who is pregnant. The possibility of a boy is the same as the possibility of a girl.”). We incorporated the personalization principle by use of a pedagogical agent (described above). However, our agent/narrator was also an application of voice principle, which encourages human versus machine voice [21]. We also we chose to keep words visible, rather than briefly flashing, on screen in the video and avoided redundant “on screen” text. Avoiding superfluous text is an example of the redundancy principle, whereby learners can be confused and retain less information when both text and graphics are on screen.

The incorporation of the above principles was enhanced by the inclusion of several interactive components as described below.

Interactive Components

We included two interactive components in order to promote comprehension, the use of short multiple choice quizzes at the end of each of the five video sections, and a person-to-person telephone interaction between the parent and adolescent and the researcher to discuss study related questions and assess comprehension of study participation.

Video Quizzes

In order to reinforce learning after each video segment, the viewer (parent or adolescent) was required to answer 2 to 3 multiple-choice questions before the next video section became available. If questions were answered incorrectly, a short phrase of encouragement was given along with the correct answers. Correctly answered questions were followed by praise and the correct answer was also shown. Thus, all responses, whether incorrect or correct, were provided with the correct answer to reinforce learning.

Comprehension Assessment

To assess the study comprehension following watching the video and interacting with the information on the website, the researchers created a 17-item questionnaire (adapted from a previously created comprehension instrument used in federally funded trials [NCI R03CA133442, NCI R03CA133419]) (Table 3). The questionnaire included open-ended questions that probed for knowledge of the research study as required for the informed consent process and was administered to the parent and adolescent during a scheduled telephone interview with the researcher.

Table 3.

Consent Comprehension Questionnaire.

Each item began with “Please tell me…”

1. the researchers’ reasons for doing this study.

2. how much of your child’s (your) time is required while you are in this study.

3. the main things that your child (you) will need to do at each study visit.

4. about the study treatments that are being tested.

5. what the chances are that your child (you) will get one kind of treatment or another

6. how many other people will be in this study.

7. what bad things or risks there could be from being in this study.

8. what the good things or benefits there are from being in the study.

9. what other choices your child has (you have), aside from being in this study.

10. how the researchers will protect your child’s (your) privacy while being in this study.

11. who’s responsible for your child’s (your) medical costs if your child gets (you get) hurt or sick while your child is (you are) in the study.

12. who you (your parent) should call if your child has (you have) questions about the study.

13. what your child (you) should do if your child wants (you want) to stop being in the study.

14. if and why the researchers could take your child (you) out of the study without your permission.

15. why researchers would give your child (you) new information about this study while your child is (you are) in it.

16. what type of payment or rewards your child (you) will get for being in the study.

17. why your child is (you are) being asked to be in the study.

The telephone interview occurred within 4 days of watching the video. The dual purpose of the call was to capture data on comprehension of the study information from the multimedia website, and to ensure the parent and adolescent had all questions answered prior to providing consent to participate in the trial. The parent and adolescent were informed at the start of the call that the purpose of the interview was to help researchers improve the way information was conveyed to families for a research study and that it was not a test to determine if they can be in the study. They were told that any questions they had about the research would be reviewed with them at the conclusion of the questionnaire administration.

The questionnaire was administered before any discussion in order to capture the parent’s and adolescent’s understanding of the research as a result of viewing the information on the multimedia website. The phone conversations were audio recorded with the verbal permission of the parent and later scored by trained coders, the study psychologist (author, HA) and a research assistant. The researchers were trained by the study psychologist on interview techniques including how to deliver prompts without leading (e.g. “tell me more about that”), how to reassure parents and adolescents if they did not know the information, and how to allow for silent pauses while the parent/adolescent were thinking. The parent and adolescent were interviewed separately; each was asked to not be present while the other was on the call so that the parent and adolescent could be interviewed independently. The comprehension assessment was repeated 12 weeks later (at the end of the adolescent’s participation in the clinical trial) to measure retention of study information.

The two coders listened to the audio recording and scored responses as incorrect, partially correct, or correct, scored as 0, 1, or 2, respectively. Discrepancies between the coders were resolved by mutual agreement with input from the principal investigator, as needed. The Newest Vital Sign was administered to capture the level of health literacy [41–46].

Data from the video and website analyzed for effects on the comprehension score includes, time spent watching the video, who watched the video (parent, child, other), answers to quiz questions, and which sidebar tabs were selected while watching the video.

Satisfaction with the Consent Process Questionnaire

The day after the comprehension assessment, the parent was sent a link to complete an online satisfaction questionnaire. The questionnaire was sent to the parent to ask about their satisfaction with the video and website in providing them with the necessary information to make a well-informed decision about their adolescent’s participation in the clinical trial. Responses were in a Likert scale (5 choice) format from “not at all satisfied” to “completely satisfied”. In addition, a slider scale was provided to determine preference for the video consent and website and phone call at one extreme versus a conventional written consent with in-person meeting with the study staff at the other extreme.

Evaluation of Comprehension in a Controlled Clinical Trial

Evaluation of the results from the 17-item comprehension questionnaire is ongoing in a controlled clinical trial in which participants provide comprehension assessment following the interactive multimedia consent process or the conventional consent process (a paper consent document followed by a face-to-face discussion with the researcher). Enrollment and study visits will continue through 2017. Briefly, these are two simultaneously conducted trials in which adolescents (ages 12 to 17 years, 60 adolescents in each trial) are enrolled in an actual asthma treatment study. The experimental group (Trial name: Use of Mobile Devices and the Internet to Streamline an Asthma Clinical Trial; [NIH grant 101HL114899 to K. Blake; ClinicalTrials.gov Identifier: NCT02061280]) receives the interactive multimedia consent process with the incorporated learning principles as described above and the control group (Trial name: Long-acting Beta Agonist Step Down Study [NCT01437995]) receives the conventional consent process with the paper consent document. The clinical study procedures (spirometry, questionnaires, diary cards) are the same in the experimental and control trials. A full description of the study design, inclusion and exclusion criteria, rationale, commonalities, and differences between the experimental and control groups has been previously published [47].

Discussion

This article describes the design, development, and planned evaluation of a multimedia consent video and website for parents and adolescents considering participation in an actual asthma clinical trial. The consent video and website were intentionally developed based upon learning theory principles applicable to multimedia electronic learning [19, 20]. Experts involved in the development included: OHSP, which evaluated content and appearance to ensure that regulatory and ethical requirements for informed consent were met; psychologist researchers, who evaluated the literature on learning theory and proposed specific principles appropriate for the electronic learning format of the video and website; institutional video and web developers, who drafted the narrative script and video scenes and ensured an appealing appearance of the website; the bioinformatics team and programmers, who created the infrastructure for the function of the video and website; and the parent and adolescent stakeholders, who provided user feedback prior to launch. Throughout the iterative process in developing the video and website, the team psychologists and principal investigator ensured any modifications were consistent with the learning principles specifically selected to promote comprehension of the research study.

Certain multimedia learning strategies were intentionally not incorporated within the video or website design. For example, we did not use signaling principle of pre-training, which would include presenting key terms, definitions, etc., in advance of the actual “lesson”, the “lesson” being the consent information. Pre-training was not included primarily due to time constraints with the informational video. We aimed to keep the video as brief as possible (approximately 15 minutes) to maintain parent and adolescent attention and concentration. We also did not incorporate simulations and gaming despite evidence of benefit for e-learning, as these features were not within the model of our multimedia video and website. Additionally, some learning principles offer opposing strategies with evidence for either approach. In these situations, we chose according to the method that fit best within our overall design, or was supported by the literature for a specific section of the video. For example, within the redundancy principle [19], there is evidence for and against the concept of on screen text to accompany another modality (e.g., narration, visual guide). We chose to allow the viewers the option to read and listen to a narration of the information within the sidebar tabs as it fit within criteria that includes giving the learner options for how to process information, expecting that the information could be difficult for the learner to process text only.

Multimedia components are incorporated into the consent process typically when information is particularly complex [17, 48–53]. While multimedia tools in the consent process are promising, enhanced comprehension compared to the conventional consent process has not been consistently reported [10–12, 18, 54]. Lack of effect has been posited due to the relative absence of conceptual learning models or theory used in development of the multimedia consent aids [12]. Notably, in a review of 20 multimedia consent studies [12] only three studies [13, 55, 56] included any cognitive learning principles for the multimedia tool that was expected to enhance consent comprehension. In the study by Jeste, et al. [55] many of the same multimedia principles [20] as in the present study such as, Coherence, Personalization, and Signaling were used to create a DVD to explain key study information. The study population included 128 adults with schizophrenia and 60 adult healthy normal controls who were presented a hypothetical protocol with the primary outcome to determine decisional capacity, rather than comprehension of study information. In another study, Bickmore, et al. [13] developed a computer agent that incorporated nonverbal behavior animations in explaining a consent document. The trial compared study comprehension when the information was delivered by the computer agent, by an experienced researcher, or by the participants reading a consent document at their own pace (the control). The participants were all adults, the sample size was relatively small (9–11 participants) in each group, and the hypothetical study was noninterventional (enrollment in a genetic repository). Campbell, et al. [11] evaluated an enhanced print version that was modified to have simplified text, bold-type headings, white space, and pictures; a video with voice narration; and a computerized version that combined narration and still pictures with simplified, bulleted text for two actual ongoing clinical trials. The trial included 29 to 30 parents in each group who were asked to “visualize an actual child and pretend that the relevant medical condition was affecting him or her” as they participated in the consent process. The primary outcome was how well the parent comprehended the study information through a free recall and prompted recall interaction with the researcher. Thus, none of these studies embodied all the critical elements incorporated in the present study which included, evidenced-based learning principles for a multimedia consent format, an actual ongoing interventional (and thus, medically complex) clinical trial, and inclusion of both a parent and child for the consent assessment which would enroll the child in the study. It is anticipated that the role for multimedia in the consent process will become more common as nearly 90% of the United States population regularly uses the internet and media-based technology costs continue to decrease and become more portable [18, 57]. Therefore, it is critical that the development of new electronic consent tools be integrated with a cognitive approach to improve learning and rigorously tested in clinical trials to evaluate effects on comprehension [58].

The application of cognitive approaches or learning theory (e.g., learning principles, cognitive load theory, working memory) is also understudied with regard to its use within pediatric biomedical research trials. Because pediatric research often requires both parental permission and child assent, refinement of consent comprehension tools based upon empirically-supported cognitive learning strategies is warranted. The literature includes development of tools for assessing children’s competence to consent for research [59], the application of learning theory in multimedia consent in hypothetical trials involving adult specialty populations [55], and modification of consent information targeting lower-literacy parents [56]. Research has also been conducted to assess consent satisfaction and comprehension using an animated computerized agent [13]. However, Bickmore and colleagues’ [13] research included a simulated consent process for adult research. Results from adult studies and hypothetical trials are difficult and inappropriate to generalize to actual pediatric clinical trials. More recently, O’lonergan, et al. [17] assessed parent and child consent comprehension with a multimedia consent tool. However, the underlying trials were hypothetical and the tool was created with an “innovative learning-objective approach,” which was not further described by the authors. Therefore, much work is yet to be done on the use of cognitive learning theory as applied to the development of multimedia consent to improve consent comprehension for parents and children who are being asked to participate in an ongoing clinical trial.

Including a cognitive informatics approach in future research evaluating learning theory principles underpinning novel electronic web-based consent tools, is critical to validate web-based informed consenting and move the process into mainstream clinical research. Additional cognitive informatics research could include evaluation of the effect of multimedia learning principles on consent comprehension based upon health literacy as well as digital and media literacy. Determining which principles of multimedia learning are essential to improve consent comprehension for different types of learners would allow for an individualized approach to the informed consent process.

Our project has several limitations. Our target audience had inherent diversity as we were including both parents and adolescents (12 to 17 years), with potentially widely varying cognitive and learning abilities, health literacy, and experience with an electronic learning format (parents potentially being less experienced with an electronic learning format than adolescents). Health literacy as measured by the Newest Vital Sign scores is a planned covariate for the comprehension assessment [41]. In addition, because we used five theoretical principles of multimedia learning concurrently for the video and website, we will not be able to determine which learning principle had a specific effect on the comprehension score. Future studies would have to be designed with tools to intentionally measure the effect of a specific learning principle on comprehension and to test differences in individual cognitive capacity on consent comprehension. As the intent of this study was to develop and evaluate a novel method of consenting that was based on specific learning principles, we recognize that there is imprecision in the relationship between the applied theories and the practical means we used to implement them. However, it is likely that this issue will be problematic for any research attempting to combine multiple learning strategies in a real-life clinical trial. Also, we had sufficient funding from the budget for our NIH-funded clinical trial to hire a professional actor for the video and to cover salary for professionals involved in the project including bioinformaticians and web designers. Thus, the costs to develop a similar video and website may be cost-prohibitive for a single-site study but may be suitable for multi-center trials where the cost could be distributed among many centers. We were also fortunate to work with an IRB that was open to assisting in the video content and design and had a rapid turn-around response for submitted modifications. While IRB delays were not a limitation for our site, it could be for other sites with less accommodating IRBs. Our study is currently in the data collection phase, but we will compare comprehension, utilization costs, and satisfaction between our multimedia consent and the conventional consent processes.

Conclusions

Previous publications on multimedia consent have typically addressed only a single or few dimensions of a learning theory, models, or rationale for the design or have not used any specific principles at all. We have shown how use of Sensory-Modality view, Coherence principle, Signaling principle, Redundancy principle, and Personalization principle can be incorporated into a cognitive approach to develop an electronic multimedia content to enhance comprehension of medical information. While our consent video and website is designed for an actual research study, the learning principles we employed are translatable to other venues in which complex medical information needs to be communicated in a multimedia manner to a diverse patient population [60, 61].

Highlights.

Manuscript entitled, “A Cognitive Approach for Design of a Multimedia Informed Consent Video and Website in Pediatric Research”

• Use of cognitive learning theory in informed consent documents is uncommon.

• A multidisciplinary team was assembled to develop a video consent website.

• Cognitive learning theory was intentionally applied to the video consent website.

• Examples of the applied learning principles to enhance comprehension are presented.