Abstract
Introduction
Standard print and verbal information provided to patients undergoing treatments is often difficult to understand and may impair their ability to be truly informed.
This study examined the effect of an interactive multimedia informational program with in-line exercises and corrected feedback on patients’ real-time understanding of their cardiac catheterization procedure.
Methods
151 adult patients scheduled for diagnostic cardiac catheterization were randomized to receive information about their procedure using either the standard institutional verbal and written information (SI) or an interactive iPad-based informational program (IPI). Subject understanding was evaluated using semi-structured interviews at baseline, immediately following catheterization, and 2 weeks after the procedure. In addition, for those randomized to the IPI, the ability to respond correctly to several in-line exercises was recorded. Subjects’ perceptions of, and preferences for the information delivery were also elicited.
Results
Subjects randomized to the IPI program had significantly better understanding following the intervention compared with those randomized to the SI group (8.3 ± 2.4 vs 7.4 ± 2.5, respectively, 0–12 scale where 12 = complete understanding, P<0.05). First-time correct responses to the in-line exercises ranged from 24.3% – 100%. Subjects reported that the in-line exercises were very helpful (9.1 ± 1.7, 0–10 scale, where 10 = extremely helpful) and the iPad program very easy to use (9.0 ± 1.6, 0–10 scale, where 10 = extremely easy) suggesting good clinical utility.
Discussion
Results demonstrated the ability of an interactive multimedia program to enhance patients’ understanding of their medical procedure. Importantly, the incorporation of in-line exercises permitted identification of knowledge deficits, provided corrected feedback, and confirmed the patients’ understanding of treatment information in real-time when consent was sought.
Keywords: Health information technology, Multimedia, patient comprehension
Background
In order for patients to make informed treatment decisions they must be provided with pertinent information presented in a manner that promotes understanding.[1] Typically, this information is imparted either through a discussion with the physician, a paper consent form, or a combination of both. Unfortunately, the literature is replete with examples showing that patients often do not read conventional consent forms and/or have difficulty understanding the information provided.[2–10] Reasons for this include patient anxiety, insufficient time, poor communication, cognitive abilities, age, limited education, poorly constructed and text-heavy consent forms, medical jargon, and low literacy/numeracy abilities.[11–13] Although several interventions such as shorter consent forms with lowered grade reading levels have been employed to help improve understanding, not all are uniformly effective.[11, 14–19] Recently, however, novel techniques employing interactive computer graphics have been shown to be effective in the communication of both medical and research information.[20–26] However, despite this, most empirical research has measured patients’ recall of information at various times after the information has been given rather than in real-time. The primary aims of this study therefore, were to evaluate the effect of an interactive multimedia program with in-line exercises and corrected feedback on patients’ real-time understanding and post-procedure recall of information regarding their cardiac catheterization.
Methods
Population
This study was approved by the University of Michigan’s IRB with a waiver of written informed consent. Consecutive patients (>18 yrs) scheduled for diagnostic cardiac catheterization were enrolled. Patients who had undergone catheterization within the previous 5 years, those requiring emergency catheterization or stenting, and those who could not read were excluded. Patients were approached to participate at minimum one hour prior to their procedure at a time when consent is normally sought. Patients were randomized to receive information about their procedure using either our standard (written and verbal) institutional consent protocol (SI) or an interactive informational iPad (Apple Corp. Cupertino CA) program (IPI). Consent for the actual procedure was obtained for all subjects separately by a cardiology team member, not involved in the study and who was blinded to the group assignment.
Program development
Based on information from medical textbooks, media, and expert opinion, graphic artists created 2 and 3D computer models of individual body parts. Each model was then combined to create a virtual patient complete with dynamic visualization of anatomical and physiologic functions (e.g., beating heart, blood flow). The principle software programs utilized in the creation of these models were: Maya®, 3-D Studio Max®, and Adobe After Effects®, Photoshop®, and Macromedia Flash®. Once the content and the visual models/simulations were created they were merged to form 9 modules entitled “Your heart,” “Your condition,” “Before your procedure,” “Your procedure,” “After your procedure,” “Risks and benefits,” “Results” and, “Alternatives.” A summary module was also included. Each module was presented sequentially so that relevant information could not be skipped (figure 1 example). Visual effects were supplemented by informational text inserts and reinforced with a narrative (voice-over). Additional information could also be accessed by tapping on any “?” icon on the iPad. This feature allowed the amount of information to be tailored to those individuals who needed more.
Figure 1.
Screenshot with visual and written information. A voice-over reinforced the information.
Embedded between each module were 7 brief interactive exercises (26 items) designed to test real-time comprehension of the information. These required the subject to provide answers to specific questions regarding risks, alternatives, etc., or to perform tasks to indicate the subjects’ understanding of the information at the time it was given (figure 2 example). For example, in one exercise, subjects were required to locate and tap on the catheter and indicate its intended path by dragging it from the groin up through the aorta, heart, and coronary arteries. Each exercise included corrected feedback to indicate whether the subject had correctly performed a task or answered a question. Correct responses were highlighted in green and incorrect responses in red with audio alerts. In some cases, correct responses initiated a “pop up” with additional information. Although the program was extremely intuitive, trained research assistants were available to help the subjects navigate the program, if required. Prior to final implementation, several iterations of the interactive program were reviewed by the study team (expertise in interventional cardiology and informed consent) for accuracy of content and tested for usability by 3 lay individuals (non-physicians/scientists). This included an assessment of how the information flowed, ease of navigation, and formatting of the questions in the on-line quizzes. The entire program including all exercises took approximately 10–12 minutes to complete.
Figure 2.
Screenshot depicting one of the in-line exercises. Subjects were required to drag each response to the correct arm of the risk/benefit scale.
Evaluation
Evaluation of the subjects’ understanding of the SI or IPI information occurred at three time-points: 1) initial baseline assessment prior to receiving the SI or IPI information (pre-test); 2) following cardiac catheterization when the patient had fully recovered and prior to discharge (post-test 1), and; 3) two weeks following the procedure (post-test 2). Each evaluation used the same semi-structured open-ended interview format to assess understanding of 6 core-elements i.e., the purpose, procedures, risks (major and minor), benefits, alternatives to cardiac catheterization, and alternatives for treatment following an abnormal result. The process for interviewing subjects and scoring understanding was based on the Deaconess Informed Consent Comprehension Test [27] and has been described previously.[7, 18, 25] Briefly, trained researchers interviewed the subjects and their responses were written down verbatim. Each response was scored independently by two assessors, blinded to the group assignments. Scores of 0, 1, and 2 were assigned based on the subject having no, partial, or complete understanding of each core element, respectively. Thus, a maximum understanding score of 12 could be obtained for all 6 core-elements. Criteria for scoring were determined a priori. In addition to the interviews, the ability of the subject to correctly answer each quiz item was recorded. Subjects also completed a short questionnaire to determine their perceptions of the risks and benefits, and clarity of the information provided. At the end, subjects randomized to SI were shown the IPI and asked which they preferred. Subject demographics including race/ethnicity, level of education, age, and gender were also recorded. Finally, the subjects’ medical literacy and numeracy were measured using the validated Rapid Estimate of Adult Literacy in Medicine (REALM) instrument [28] and Subjective Numeracy Scale (SNS), [29, 30] respectively.
Statistical Analysis
Statistical analyses were performed using SPSS© statistical software (IBM Corp., New York). Comparisons between the within-group pre and post-tests were conducted using paired t tests. Between group pre and post-test comparisons were conducted using unpaired t tests. Comparison between groups with respect to categorical variables such as gender and education were analyzed using chi-square and Fisher’s Exact test, as appropriate. Inter-rater agreements between the two assessors were performed using the kappa (κ) statistic for each of the consent elements (risks, procedures, etc.). Kappa values of ≥ 0.7 were considered to represent acceptable levels of agreement. Results ranged from 0.76–0.97 (P<0.001).
Sample size determination was based on a previous study which showed that understanding of a standard consent form was 8.1 ± 2.3 (scale of 0–12) and that understanding using a computer program was 9.3 ± 2.2. (26) Accepting this difference as the smallest that we believed to be important to detect, we required a sample of 74/group (α =0.05, β =0.1, two-sided).
Results
A total of 189 subjects were approached to participate in this study, of which 25 declined. Of those who declined, 52% were female, 92% Caucasian and the average age was 59.8 years suggesting that the non-respondents were not substantively different from the respondents. Six subjects withdrew following randomization (3 IPI and 3 SI). Of those in the SI group, 2 subjects were withdrawn because the procedure was canceled and 1 did not want to continue with the interviews. In the IPI group, 1 subject was “too tired,” and 2 did not want to see all the information. One subject felt that they already knew enough and another was upset that she had not been made aware that she would be awake during the procedure. Complete data for 151 subjects were thus available for analysis. Figure 3 describes the flow of subjects through the phases of the study.
Figure 3.
Flow diagram of subject progress through the phases of the study.
Table 1 describes the demographics of the study sample by group and showed no differences.
Table 1.
Demographics by Message Delivery
| Standard (SI, n = 77) | Interactive (IPI, n = 74) | |
|---|---|---|
| Age, yrs | 61.6 ± 14.6 | 62.2 ± 12.1 |
| Gender (F/M) % | 42/58 | 47/53 |
| Race/ethnicity: | ||
| Caucasian | 64 (84.2) | 59 (80.8) |
| African American | 9 (11.8) | 10 (13.7) |
| Other | 3 (4.0) | 4 (5.5) |
| Level of Education: | ||
| ≤High school graduate | 25 (33.3) | 17 (23.3) |
| Some college/trade school | 20 (26.7) | 23 (31.5) |
| ≥ Bachelor’s degree | 30 (40.0) | 33 (45.2) |
| English as primary language | 75 (98.7) | 72 (97.3) |
| Pre-procedure anxiety (0–10 scale) | 5.3 ± 3.1 | 5.4 ± 3.2 |
| REALM (literacy) | 63.3 ± 6.8 | 63.5 ± 8.2 |
| SNS (numeracy) | 38.4 ± 8.1 | 38.7 ± 7.9 |
Data are mean ± SD and n (%)
REALM = Rapid Estimate of Adult Literacy in Medicine (0–66 scale, where 66 = High literacy). SNS = Subjective Numeracy Score (0–48 scale, where 48 = High numeracy)
The subjects’ perceptions of the risks of cardiac catheterization were not different between the SI and IPI groups (4.2 ± 2.3 vs 4.8 ± 2.7, respectively, 0–10 scale where 10 = extremely risky). However, significantly fewer subjects rated the information as “very clear” in the SI group compared with the IPI group (88.3% vs. 97.3%, respectively, P < 0.05). Of note too was that, given an option, significantly fewer subjects in the IPI group (5.4%) reported that they would prefer standard print information compared with those that received the SI (26.7%, P < 0.05). Approximately, one quarter (24%) of subjects in the SI group reported that they would prefer future medical information presented using a computer only format compared with 29.7% in the IPI group and 46.7% and 63.5% would prefer both computer and print information, respectively. Overall, subjects randomized to receive the iPad program found it very easy to use i.e., 9.0 ± 1.6 of 10 (0–10 scale, where 10=extremely easy). None of the subjects reported difficulties in viewing the images or hearing and reading the information.
Table 2 describes the subjects’ understanding of the information and showed that those randomized to the IPI had significantly better Post-test 1 and 2 recall of the information compared with those who received the SI. There were also clinically significant trends towards greater understanding in the IPI group compared with the SI for the different elements of the consent message. As shown in the table, there were also relevant trends towards improved understanding in the IPI group for subjects regardless of literacy and numeracy abilities. Subjects in both groups with high literacy had significantly better overall understanding compared with those with low literacy. Subjects in the IPI group with a greater than high school education had significantly better understanding than those in the SI group.
Table 2.
Understanding of the Information by Message Delivery
| Standard (SI) | Interactive (IPI) | Mean difference (95% CI) | |
|---|---|---|---|
| Baseline understanding (Pre-test) | 6.47 ± 2.5 | 6.57 ± 2.4 | 0.1 (−0.7, 0.9) |
| Post-test 1 (after intervention) | 7.35 ± 2.5 | 8.28 ± 2.4*† | 0.93 (0.1, 1.7) |
| Low literacy | 5.14 ± 2.7 | 6.40 ± 2.6 | 1.26 (−2.2, 4.8) |
| High literacy | 7.83 ± 2.2‡ | 8.55 ± 2.3‡ | 0.7 (−0.6, 1.5) |
| Low numeracy | 6.75 ± 2.5 | 7.63 ± 2.5 | 0.8 (−0.6, 2.4) |
| High numeracy | 7.87 ± 2.3 | 8.72 ± 2.3 | 0.8 (−0.1, 1.8) |
| ≤ High school education | 6.80 ± 2.8 | 6.76 ± 2.3 | −0.04 (−1.7, 1.6) |
| > High school education | 7.66 ± 2.3 | 8.77 ± 2.3*$ | 1.1 (0.2, 1.9) |
| Post-test 2 (2 wks after intervention) | 6.69 ± 2.5 | 7.60 ± 2.2* | 0.91 (0.08, 1.7) |
| “Complete” Post-test 1understanding of: | |||
| Indications for cardiac catheterization | 79.2 | 83.8 | |
| Procedure | 33.8 | 47.3 | |
| Risks | 57.1 | 70.3 | |
| Benefits | 59.7 | 64.9 | |
| Alternatives to cardiac catheterization | 9.1 | 25.7* | |
Data are presented as % and mean ± SD (based on a scale of 0–12, where 12 = Complete understanding). SI = Standard information, IPI interactive iPad information. CI = Confidence intervals
Low numeracy = 0–35, High numeracy = >35 on the Subjective Numeracy Scale
Low Literacy = 0–60 (3rd – 8th grade equivalence), High Literacy = >60 (9th grade equivalence or higher)
P< 0.05 vs Standard,
P< 0.05 vs Pre-test,
P<0.05 vs Low literacy,
P < 0.01 vs ≤ High school
Table 3 describes the ability of patients to correctly respond to each of the exercise items. As shown, although most subjects were able to respond correctly to questions related to the risks, benefits and alternatives, only a third of patients were initially aware that they would be awake during the procedure and might experience some discomfort. Subjects who performed better than the median split (22) for the number of correct responses were significantly more likely to have greater Post-test 1 understanding compared with those below the median split (9.4 ± 2.0 vs. 7.9 ± 2.2, respectively, P =0.01; 0–12 scale, where 12 = complete understanding). Overall, subjects with lower numeracy and literacy performed less well on the quizzes compared with those with high numeracy and literacy (20.1 ± 2.7 vs 22.0 ± 2.1 and 18.3 ± 4.2 vs 21.6 ± 2.1, respectively, P<0.01). None of the subjects reported frustration when giving an incorrect answer. Although, completion of the in-line exercises added less than 2 minutes to the program’s length, subjects found them to be very helpful (9.1 ± 1.7, 0–10 scale, where 10 = extremely helpful).
Table 3.
Results of the In-line interactive Exercises
| 1st attempt correct n (%) | 2nd attempt correct n (%) | |
|---|---|---|
| Exercise 1: Before the procedure | ||
| Required to select from options of yes, no, or maybe (correct answers in parentheses) | ||
| a) Allowed to eat before the procedure (no) | 72 (97.3) | 2 (2.7) |
| b) Awake during procedure (yes) | 27 (36.5) | 47 (63.5) |
| c) Feel anything during procedure (maybe) | 23 (31.1) | 51 (68.9) |
| Exercise 2: During the procedure | ||
| Required to tap or tap and drag on correct answer | ||
| a) Identify catheter insertion site | 56 (77.8) | 18 (22.2) |
| b) Trace catheter path | 67 (90.5) | 7 (9.5) |
| c) Identify spread of contrast dye | 47 (68.1) | 27 (31.9) |
| Exercise 3: After the procedure | ||
| Required to select from options of yes, no, or maybe | ||
| a) Allowed to drive after procedure (no) | 72 (97.3) | 2 (2.7) |
| b) Allowed to eat/drink after procedure (yes) | 35 (47.3) | 39 (52.7) |
| c) Ability to resume normal activities (maybe) | 18 (24.3) | 56 (75.7) |
| Exercise 4: Risks of procedure | ||
| Required to select from options of yes or no | ||
| a) Bruising (yes) | 68 (91.9) | 6 (8.1) |
| b) Infection (yes) | 69 (93.2) | 5 (6.8) |
| c) Rash (no) | 61 (82.4) | 13 (17.6) |
| Exercise 5: Benefits/risks of cardiac catheterization | ||
| Required to move statements to correct arms of the risk/benefit scales (Fig. 2) | ||
| a) Short recovery time (benefit) | 68 (91.9) | 6 (8.1) |
| b) May cause discomfort (risk) | 66 (93.0) | 8 (7.0) |
| c) Bruise/infection at the insertion site (risk) | 69 (93.2) | 5 (6.8) |
| d) Helps diagnose heart disease (benefit) | 70 (98.6) | 4 (1.4) |
| Exercise 6: Alternative diagnostic tests | ||
| Required to select most accurate way to identify coronary artery disease from Echocardiogram, Stress test, Cardiac catheterization, CT/MRI | ||
| a) Cardiac catheterization (correct) | 64 (86.5) | 10 (13.5) |
| Exercise 7: Benefits of cardiac catheterization | ||
| Required to select “all that apply” as the main benefits of cardiac catheterization | ||
| a) Evaluate how well the heart is pumping (correct) | 69 (93.2) | 5 (6.8) |
| b) Determine weight of heart (incorrect) | 68 (91.9) | 6 (8.1) |
| c) Assess condition of heart valves (correct) | 72 (97.3) | 2 (2.7) |
| d) Identify congenital heart disease? (correct) | 72 (97.3) | 2 (2.7) |
| e) Cure coronary artery disease (incorrect) | 57 (77.0) | 17 (23.0) |
| f) Identify blockages in coronary arteries (correct) | 74 (100) | 0 (0.0) |
Comments regarding use of the interactive program were generally very positive and included:
“iPad is very helpful. Would like to see this before undergoing a procedure. It can also be helpful in explaining procedures to family members.”
“Like how visual iPad is, can see what will happen during procedure.”
“If I were to see that I would have been less worried.”
One individual, however, thought that the IPI was too graphic: “That needle in the video scared me to death, it’s like a turkey baster.”
Discussion
Findings from this study confirmed those of others that interactive informational computer-based programs are well accepted and result in enhanced understanding and recall compared with traditional print-based medical information. [20, 25, 26, 31] Although there may be several explanations for these findings, one possible advantage of interactive programs is that they provide active participation in learning such that individuals become more “engaged” in the information compared with the traditional print-based approach. Indeed, data from the educational literature confirm the importance of interactive multimedia on student satisfaction and performance over traditional learning. [32–34] Another important advantage is that high resolution computer-graphics provide greater visual salience compared with simple text. This increased visual salience is best explained by the so called pictorial superiority effect (PSE) which is grounded in the theory that information is better received and retained when presented as pictures rather than just words. [35–37] Indeed, PSE appears to be particularly relevant in helping vulnerable individuals understand medical information such as those with low literacy and numeracy skills and the young and old. [38, 39] At a time when traditional informed consent documents are getting longer and more complicated, [40] the greater visual saliency offered by dynamic computer-based programs provides new opportunities to enhance the comprehension and retention of complex information.
A primary aim of this study was to examine the effectiveness of using in-line interactive exercises with corrected feedback as a means to evaluate the patients’ real-time understanding of the information. Although several studies have shown that recall and understanding of treatment information is often sub-optimal, these measures are typically made after the information has been given and consent obtained. One could argue, therefore, that it is immaterial whether or not the patient can recall information “after the fact” since the decision to consent (or decline) has already been made. Use of real-time quizzes is thus important in that they allow for an assessment of the patients’ understanding at the time the information is given and when most consents are obtained Furthermore, knowledge deficits can immediately be identified and. corrected feedback provided to encourage further discussion and ensure understanding of the material. Overall, subjects found the computer program easy to use and the in-line exercises extremely helpful, suggesting good clinical utility. As shown, good performance on the in-line exercises was also associated with improved recall of information following the IPI intervention.
While the concept of testing with corrected feedback as a means to enhance the retention of information is not new, [41] there is a relative paucity of studies evaluating this approach in the medical and research settings. [42–45] In the educational field, Kang et al. showed that undergraduate students given intervening short answer tests with corrected feedback did better on the final exam compared with those without feedback. [41] In an older study, Taub et al. [45] showed that testing with corrected feedback among elderly patients resulted in improved comprehension of an informed consent procedure. More recently, Sudore et al. [46] showed that a “teach to goal strategy” wherein information was re-read to patients was helpful in improving overall comprehension of study information and, in another study, Festinger et al. [43] showed that subjects randomized to a post-consent corrected feedback group had greater long-term retention of study information compared with those that received no feedback.
Overall, the IPI approach was well received. Not only did subjects find the IPI information to be clearer and easy to use but when asked about alternatives for message delivery, only 16% reported that they would choose a written consent form for their future medical information needs. Indeed, the majority of subjects reported that they would prefer incorporation of computer-based media as a vehicle for presenting medical information. Given the subjects’ stated preferences for computer-based information and apparent ease of use, one potential advantage would be to have patients view the program well before arrival at the hospital and perhaps again on the day of surgery.
Although we have used cardiac catheterization as our treatment example, our primary purpose was to evaluate the technology and as such, it is likely that this approach would be equally effective for a variety of medical treatments and indeed, research. To this end, development of the program incorporated several techniques that have been known to successful in both the communication and education literature. Notably, in addition to the in-line exercises, the presentation of risks in this study utilized animated pictographs rather than text. Recent studies suggest that risk graphics, particularly pictographs, result in greater comprehension compared with numerical text and are particularly effective among individuals with low numeracy. [47–49] In support of this, the percentage of patients with “complete” understanding of the risks and benefits of cardiac catheterization in this study (with pictographs) was much greater than in a previous study that used text [26] (70.3% vs 53.6% and 64.9% vs 49.3%, respectively).
The potential limitations of this study are recognized. First, although use of the interactive program resulted in greater understanding of the information, it is not known how this greater knowledge would have affected the initial decision to undergo cardiac catheterization since this was likely made in the clinic before arrival at the interventional cardiology suite. However, given that several subjects arrived on the day of their procedure lacking basic knowledge suggests that the enhanced understanding achieved through this interactive program would be critical in the informed consent process. Second, although the computer-program was commercially developed, the study was funded independently by the National Institutes of Health and evaluated by individuals with no commercial or financial associations. Third, this study represents one intervention at a single center and thus, may not be generalizable to all situations. Finally, although the sample size was sufficiently powered for the primary objective of this study, we recognize that it may have been insufficient for all subgroup analyses.
Results from this study reinforce the promise of interactive multimedia programs as a means to enhance patients’ understanding of their medical treatments. Importantly, this study demonstrated the ability of in-line interactive exercises to assess patients’ real-time understanding of medical information with the opportunity to correct and re-educate the patient at the time information is given and the patient’s consent is sought. In light of the increasing societal use of computer technology and the gradual move away from paper to electronic medical information systems, these results provide evidence to support the use of interactive multimedia applications as a means to enhance patient education and informed consent for both treatment and research.
Research Highlights.
The interactive program resulted in greater understanding compared with the standard method.
Acceptance and utility of the interactive program were high.
In-line exercises with corrected feedback allowed real-time assessment of understanding.
Patients reported a preference for interactive programs for future medical information needs.
Summary points.
What is known about this topic
Many patients, parents, and research subjects have difficulty understanding information regarding treatments and research participation and, as such, are unable to make informed decisions.
Traditional paper consent forms are often difficult for the lay person to understand and provide no opportunity for participatory learning.
A lack of understanding can result in the misinterpretation of risks and benefits and the inability to follow a treatment regimen or research protocol.
What this study added
Use of an interactive multimedia consent program resulted in greater patient understanding of medical treatment information compared with the standard written and verbal consent process
The interactive program offered the ability to tailor the amount of information to the individual patient’s preferences,
Patients found the in-line exercises with corrected feedback to be extremely helpful and the interactive program, easy to use.
The majority of patients reported a preference for interactive programs for their future medical information needs
Acknowledgments
Supported by a grant from the National Institutes of Health; NHLBI (2R42HL087488) to Dr. Levine. Dr. Tait was also supported by a grant from the National Institutes of Health, Bethesda MD (NICHD: R01HD053594). The sponsors had no involvement in the study design, analysis, interpretation, manuscript preparation, or decisions to submit the manuscript for publication. Many thanks to Sarah Zyzanski, B.S., Katherine Bill, B.S., Adam Eickmeyer, Lamira Ray, Roneil Jackson, Cortney Segmen, for help with recruitment and data collection.
Footnotes
Author contributions
All authors qualify for authorship by substantial contributions to the research design and manuscript preparation. Dr. Tait led the evaluation of the program, analyzed the data, and wrote the manuscript. Drs. Voepel-Lewis and Chetcuti contributed to the design of the study, interpretation of the results, and reviewed and revised the manuscript drafts. Ms. Brennan-Martinez contributed to the design of the study, facilitated data collection, and reviewed drafts of the manuscript. Dr. Levine contributed to the development of the multimedia program and reviewed and revised drafts of the manuscript.
Conflict of interest statement
Dr. Levine is the President and Chief Medical Officer and President of ArchieMD, Inc. but was funded independently for this project by a grant from the National Institutes of Health. Dr. Levine was responsible for the development of the interactive program and read the final manuscript but had no involvement in subject recruitment, data collection, scoring, statistical analysis, or interpretation of the data. None of the other investigators have any financial, commercial, or other interests in ArchieMD, Inc. to declare.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Department of Health and Human Services. . Protection of Human Subjects. 1991;56(117):28001–32. 45 CFR 46. [Google Scholar]
- 2.Barrett R. Quality of informed consent: measuring understanding among participants in oncology clinical trials. Oncol Nurs Forum. 2005;32:751–5. doi: 10.1188/05.ONF.751-755. [DOI] [PubMed] [Google Scholar]
- 3.Chan T, Eckert K, Venesoen P, Leslie K, Chin-Yee I. Consenting to blood: what to patients remember? Transfusion Med. 2005;15:461–6. doi: 10.1111/j.1365-3148.2005.00622.x. [DOI] [PubMed] [Google Scholar]
- 4.Engel K, Heisler M, Smith D, Robinson C, Forman J, Ubel P. Patient comprehension of emergency department care and instructions: Are patients aware of when they do not understand? Ann Emerg Med. 2009;53:454–61. doi: 10.1016/j.annemergmed.2008.05.016. [DOI] [PubMed] [Google Scholar]
- 5.Makaryus A, Friedman E. Patient’s understanding of their treatment plans and diagnosis at discharge. Mayo Clin Proc. 2005;80:991–4. doi: 10.4065/80.8.991. [DOI] [PubMed] [Google Scholar]
- 6.McCormack D, Evoy D, Mulcahy D, Walsh M. An evaluation of patients comprehension of orthopaedic terminology: implications for informed consent. J Royal Coll Surg Edin. 1997;42:33–5. [PubMed] [Google Scholar]
- 7.Tait AR, Voepel-Lewis T, Malviya S. Do they understand? (Part I): Parental consent for children participating in clinical anesthesia and surgery research. Anesthesiology. 2003;98:603–608. doi: 10.1097/00000542-200303000-00005. [DOI] [PubMed] [Google Scholar]
- 8.Tait AR, Voepel-Lewis T, Malviya S. Do they understand? (Part II): Assent of children participating in clinical anesthesia and surgery research. Anesthesiology. 2003;98:609–14. doi: 10.1097/00000542-200303000-00006. [DOI] [PubMed] [Google Scholar]
- 9.Vallance J, Ahmed M, Dhillon B. Cataract surgery and consent; recall, anxiety, and attitude toward trainee surgeons preoperatively and postoperatively. J Cataract Refract Surg. 2004;30:1479–85. doi: 10.1016/j.jcrs.2003.11.050. [DOI] [PubMed] [Google Scholar]
- 10.Rosique I, Perez-Carceles M, Romero-Martin M, Osuna E, Luna A. The use and usefulness of information for patients undergoing anaesthesia. Med Law. 2006;25:715–27. [PubMed] [Google Scholar]
- 11.Flory J, Emanuel E. Interventions to improve research participants’ understanding in informed consent for research. JAMA. 2004;292(13):1593–601. doi: 10.1001/jama.292.13.1593. [DOI] [PubMed] [Google Scholar]
- 12.Campbell F, Goldman B, Boccia M, Skinner M. The effect of format modifications and reading comprehension on recall of informed consent information by low-income parents: a comparison of print, video, and computer-based presentations. Patient Educ Counsel. 2004;53:205–16. doi: 10.1016/S0738-3991(03)00162-9. [DOI] [PubMed] [Google Scholar]
- 13.Jackson L. Information complexity and medical communication: The effects of technical language and amount of information in a medical message. Hlth Commun. 1992;4:197–210. [Google Scholar]
- 14.Björn P, Rossel P, Holm S. Can the written information to research subjects be improved? - an empirical study. J Med Ethics. 1999;25:263–7. doi: 10.1136/jme.25.3.263. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Dresden G, Levitt M. Modifying a standard industry clinical trial consent form improves patient information retention as part of the informed consent process. Acad Emerg Med. 2001;8:246–52. doi: 10.1111/j.1553-2712.2001.tb01300.x. [DOI] [PubMed] [Google Scholar]
- 16.Rogers C, Tyson J, Kennedy K, Broyles R, Hickman J. Conventional consent with opting in versus simplified consent with opting out: an exploratory trial for studies that do not increase patient risks. J Pediatr. 1998;132:606–11. doi: 10.1016/s0022-3476(98)70347-6. [DOI] [PubMed] [Google Scholar]
- 17.Tait AR, Voepel-Lewis T, Malviya S. Presenting research information to children: A tale of two methods. Anesth Analg. 2007;105:358–64. doi: 10.1213/01.ane.0000270326.44507.11. [DOI] [PubMed] [Google Scholar]
- 18.Tait AR, Voepel-Lewis T, Malviya S, Philipson S. Improving the readability and processability of a pediatric informed consent document: effects on parents’ understanding. Arch Pediatr Adolesc Med. 2005;159:347–52. doi: 10.1001/archpedi.159.4.347. [DOI] [PubMed] [Google Scholar]
- 19.Young D, Hooker D, Freeberg F. Informed consent documents: Increasing comprehension by reducing reading level. IRB: Ethics Hum Res. 1990;12:1–5. [PubMed] [Google Scholar]
- 20.Hopper K, Zajdel M, Hulse S, Yoanidis N, TenHave T, Labuski M, et al. Interactive method of informing patients of the risks of intravenous contrast media. Radiology. 1994;192:67–71. doi: 10.1148/radiology.192.1.8208968. [DOI] [PubMed] [Google Scholar]
- 21.Morris D, Rothera M. The application of computer-enhanced imaging to improve preoperative counselling and informed consent in children considering bone anchored auricular prosthesis surgery. Int J Pediatr Otorhinolaryngol. 2000;55:181–6. doi: 10.1016/s0165-5876(00)00382-7. [DOI] [PubMed] [Google Scholar]
- 22.Rossi M, Guttmann D, MacLennan M, Lubowitz J. Video informed consent improves knee arthroscopy patient comprehension. Arthroscopy. 2005;21:739–43. doi: 10.1016/j.arthro.2005.02.015. [DOI] [PubMed] [Google Scholar]
- 23.Rossi M, McClellan R, Chou L, Davis K. Informed consent for ankle fracture surgery: Patient comprehension of verbal and videotaped information. Foot & Ankle Int. 2004;25:756–62. doi: 10.1177/107110070402501011. [DOI] [PubMed] [Google Scholar]
- 24.Tait AR, Voepel-Lewis T, Brennan-Martinez C, McGonegal M, Levine R. Using animated computer-generated text and graphics to depict the risks and benefits of medical treatment. Am J Med. 2012;125:1103–10. doi: 10.1016/j.amjmed.2012.04.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Tait AR, Voepel-Lewis T, McGonegal M, Levine R. Evaluation of a prototype interactive consent program for pediatric clinical trials: A pilot study. JAMIA. 2012;19:e43–e5. doi: 10.1136/amiajnl-2011-000253. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Tait AR, Voepel-Lewis T, Moscucci M, Brennan-Martinez C, Levine R. Patient comprehension of an interactive, computer-based information program for cardiac catheterization. Arch Intern Med. 2009;169:1907–14. doi: 10.1001/archinternmed.2009.390. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Miller C, O’Donnell D, Searight H, Barbarash R. The Deaconess Informed Consent Comprehension Test: an assessment tool for clinical research subjects. Pharmacotherapy. 1996;16(5):872–8. [PubMed] [Google Scholar]
- 28.Davis T, Long S, Jackson R, Mayeaux E, George R, Murphy P, et al. Rapid estimate of adult literacy in medicine: A shortened screening instrument. Fam Med. 1993;25:391–5. [PubMed] [Google Scholar]
- 29.Fagerlin A, Zikmund-Fisher B, Ubel P, Jankovic A, Derry H, Smith D. Measuring numeracy without a math test: Development of the subjective numeracy scale (SNS) Med Dec Making. 2007;27:672–80. doi: 10.1177/0272989X07304449. [DOI] [PubMed] [Google Scholar]
- 30.Zikmund-Fisher B, Smith D, Ubel P, Fagerlin A. A validation of the subjective numeracy scale: effects of low numeracy on comprehension of risk communication and utility elicitations. Med Dec Making. 2007;27:663–71. doi: 10.1177/0272989X07303824. [DOI] [PubMed] [Google Scholar]
- 31.Hermann M. 3-dimensional computer animation - a new medium for supporting patient education before surgery. Acceptance and assessment of patients based on prospective randomized study - picture versus text [German] Chirug. 2002;73:500–7. doi: 10.1007/s00104-001-0416-y. [DOI] [PubMed] [Google Scholar]
- 32.Schrand T. Tapping into active learning and multiple intelligences with interactive multimedia: A low-threshold classroom approach. College Teaching. 2008;36:78–84. [Google Scholar]
- 33.Schank R. Active learning through multimedia. IEEE Multimedia. 1994;1:69–78. [Google Scholar]
- 34.Zhang D. Interactive multimedia-based E-learning: A study of effectiveness. Am J Distance Learn. 2005;19:149–62. [Google Scholar]
- 35.Ally B, Gold C, Budson A. The picture superiority effect in patients with Alzheimer’s disease and mild cognitive impairment. Neuropsychologica. 2009;47:595–8. doi: 10.1016/j.neuropsychologia.2008.10.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Cherry K, Park D, Frieske D, Smith A. Verbal and pictorial elaborations enhance memory in younger and older adults. Aging Neuropsychol Cognition. 1996;3:15–29. [Google Scholar]
- 37.Hockley W. The picture superiority effect in associatve recognition. Memory Cogn. 2008;36:1351–9. doi: 10.3758/MC.36.7.1351. [DOI] [PubMed] [Google Scholar]
- 38.Rissenberg M, Glanzer M. Picture superiority in free recall: the effects of normal aging and primary degenerative dementia. J Gerontol. 1986;41:64–71. doi: 10.1093/geronj/41.1.64. [DOI] [PubMed] [Google Scholar]
- 39.Winograd E, Smith A, Simon E. Aging and the pictorial superiority effect in recall. J Gerontol. 1982;37:70–5. doi: 10.1093/geronj/37.1.70. [DOI] [PubMed] [Google Scholar]
- 40.Schreiner M. Can we keep it simple? JAMA Pediatr. 2013;167:603–5. doi: 10.1001/jamapediatrics.2013.152. [DOI] [PubMed] [Google Scholar]
- 41.Kang H, McDermott B, Roediger H. Test format and corrective feedback modify the effect of testing on long-term retention. Eur J Cogn Psychol. 2007;19:528–58. [Google Scholar]
- 42.Coletti A, Heagerty P, Sheon A, Gross M, Koblin B, Metzger D, et al. Randomized, controlled evaluation of prototype informed consent process for HIV vaccine efficacy trials. J Acquir Immune Defic Syndr. 2003;32:161–9. doi: 10.1097/00126334-200302010-00008. [DOI] [PubMed] [Google Scholar]
- 43.Festinger D, Dugosh K, Croft J, Arabia P, Marlowe D. Corrected feedback: a procedure to enhance recall of informed consent to research among substance abusing offenders. Ethics & Behav. 2010;20:387–99. doi: 10.1080/10508422.2010.491767. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Stiles P, Poythress N, Hall A, Falkenbach D, Williams R. Improving understanding of research consent disclosures among persons with mental illnesses. Psychiatr Svcs. 2001;52:780–5. doi: 10.1176/appi.ps.52.6.780. [DOI] [PubMed] [Google Scholar]
- 45.Taub H, Baker M. The effect of repeated testing upon comprehension of informed consent materials by elderly volunteers. Exper Aging Res. 1983;9:135–8. doi: 10.1080/03610738308258441. [DOI] [PubMed] [Google Scholar]
- 46.Sudore R, Landefeld C, Williams B, Barnes D, Lindquist K, Schillinger D. Use of a modified informed consent process among vulnerable patients: A descriptive study. JGIM. 2006;21:867–73. doi: 10.1111/j.1525-1497.2006.00535.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Hawley S, Zikmund-Fisher B, Ubel P, Jancovic M, Lucas T, Fagerlin A. The impact of the format of graphical presentation on health-related knowledge and treatment choices. Patient Educ Counsel. 2008;73:448–55. doi: 10.1016/j.pec.2008.07.023. [DOI] [PubMed] [Google Scholar]
- 48.Tait AR, Voepel-Lewis T, Zikmund-Fisher B, Fagerlin A. The effect of format on parents’ understanding of the risks and benefits of clinical research: A comparison between text, tables, and graphics. J Hlth Comm. 2010;15:487–501. doi: 10.1080/10810730.2010.492560. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Tait AR, Voepel-Lewis T, Zikmund-Fisher B, Fagerlin A. Presenting research risks and benefits to parents: Does format matter? Anesth Analg. 2010;111:718–23. doi: 10.1213/ANE.0b013e3181e8570a. [DOI] [PMC free article] [PubMed] [Google Scholar]