Using Digital Multimedia to Improve Parents’ and Children’s Understanding of Clinical Trials

Abstract

Objective

Data show that many research subjects have difficulty understanding study information using traditional paper consent documents. This study, therefore, was designed to evaluate the effect of an interactive multimedia program on improving parents’ and children’s understanding of clinical trial concepts and participation.

Methods

Parents (n = 148) and children (n = 135) were each randomized to receive information regarding clinical trials using either a traditional paper format (TF) or an interactive iPad program (IP) with in-line exercises. Participants’ understanding of the information was assessed using semi-structured interviews prior to (pre-test) and after (post-test) receiving the information. Participants also completed a short survey to assess their perceptions of the information delivery and satisfaction with the process.

Results

Regardless of the mode of information delivery, all participants demonstrated improved pre- to post-test understanding. While there were no statistical differences in parents’ post-test understanding between the TF and IP groups, children in the IP group had significantly greater post-test understanding compared with children in the TF group (11.65(4.1) vs 8.85(4.1) [2.8, 1.4,4.2] 0–18 scale where 18 = complete understanding). Furthermore, the IP was found to be significantly “easier to follow” and “more effective” in presenting information compared with the TF.

Conclusions

Results demonstrated the importance of providing information regarding clinical trial concepts to parents and children. Importantly, the ability of interactive multimedia to improve understanding of clinical trial concepts and satisfaction with information delivery, particularly among children, supports this approach as a novel and effective vehicle for enhancing the informed consent process.

INTRODUCTION

Informed consent is central to the bioethical principle of respect for persons, yet many studies have shown that research participants do not understand the information they are given and, as such, have difficulty making informed decisions.[1–4] In one cancer trial, less than half of participants were able to recall the risks and the unproven nature of the trial,[5] and in a neuro-oncology study, 29% of participants were unable to recall any risks of the trial drug.[6] Clinical trials should neither create false hopes nor a sense of futility[7] yet, therapeutic misconception (belief that the study is an extension of standard treatment) is common.[8, 9] Furthermore, many central concepts of clinical trials such as randomization and blinding are often misunderstood.[8–11]

Pediatric clinical trials may pose greater challenges compared with adult trials. Parents and adolescents, for example, may have different appreciation and understanding of the risks and benefits of a clinical trial and differ in opinions regarding decision-making authority and physician influence.[12] In one study, adult oncology decision-makers were more informed and engaged by their physicians compared with parent decision-makers.[13] Parents and children also struggle with concepts such as randomization and often confuse the different phases of trial development.[13,14] This is important given that a lack of understanding of these important concepts and confusion between research and treatment undermines the central basis for informed consent. Furthermore, subjects who do not understand study information may misinterpret the risks and benefits, be unable to follow a research protocol, and may ultimately regret participating.[2]

Growing evidence suggests that interactive computer-based digital information may provide greater patient comprehension of medical information compared with the more traditional paper formats[15–18] but there is little data to support this approach for research. This study therefore, was designed to compare parents’ and children’s understanding of clinical trial information delivered using either an interactive multimedia program or a traditional paper format.

METHODS

Approval by the University of Michigan’s Institutional Review Board with a waiver of written consent/assent was obtained. Participants consisted of parents (>18yrs) and children (10–17yrs) attending one of several pediatric clinics. Exclusion criteria included non-English speakers and those with cognitive impairment. Demographics including age, gender, education, and race/ethnicity were recorded. Subject literacy was measured using the Slosson Oral Reading Test-revised (SORT-R3). [19]

Digital program development

Computer-visualizations were modeled using 2 and 3-D graphics software and merged into modules that described the clinical trial concepts. Using an iPad platform, information was presented in both visual and written formats together with a narrative “voice-over.” Screens were presented sequentially and could not be skipped. An example screenshot that uses a pictograph to describe the chance of receiving the experimental treatment in a clinical trial is shown in figure 1. The program also included five interactive exercises that participants were required to complete in order to advance. These exercises required participant to “point and touch” or “touch and drag” on the screen to select their responses. One exercise required participants to solve a jigsaw puzzle by matching a term with the correct answer. Each exercise employed corrective feedback (colored icons or sound) to inform the participant as to whether or not they had correctly answered the questions.

Figure 1.

Screenshot highlighting the probability of being randomized to the treatment or control arms of a clinical trial.

The paper version contained text only but the content was identical to that presented in the digital version, however, no exercises were included. Written information in both versions was written at the 7^th –8^th grade reading level (Flesch-Kincaid reading formula).[20]

Evaluation

Consecutive parents and children were recruited in the outpatient clinic and each given a short pre-test to elicit their understanding of 9 basic research concepts i.e., clinical trial, participation, protocol, randomization, placebo, blinding, double-blinding, effectiveness, and informed consent. Participants’ responses were written down verbatim by a trained research assistant who had no vested interest in the study. Participants were then randomized (computer-generated) to receive information about these clinical trial concepts and participation using either a traditional paper format (TF) or the digital interactive program (IP). Parents and children were evaluated separately to ensure individualized responses.

Once the participants completed the TF or IP, they were interviewed a second time to determine their “new” understanding of the information (post-test). This post-test was identical to the pre-test. In addition, participants were also interviewed to test their knowledge of different aspects of clinical trial participation (7 items e.g., “what are the possible good things that might happen to you (or your child) by being in a clinical trial?” All tests were scored independently by 2 blinded assessors based on criteria established prior to the study and using a 3-point scale of “no understanding” (0), “partial understanding” (1), and “complete understanding” (2). Thus, scores on the tests ranged from 0–18 (concepts) and 0–14 (participation). In scoring, children were not expected to provide the same amount of detail as parents. This process of interviewing and scoring has been described previously.[4, 21, 22]

For the IP group, the ability to correctly answer questions (first or subsequent attempts) in the in-line exercises was recorded. Additionally, a survey related to the participants’ perceptions of, and satisfaction with the quality and effectiveness of the information was conducted using a combination of 0–10 numbers scales and Likert scales. At the end of the study, participants were shown both the TF and IP information and asked which they preferred.

Statistical analysis

Statistical analyses were performed using SPSS^© (IBM Corp, New York, v 21.0) software. Within and between group pre and post-tests were analyzed for both parametric and non-parametric data. Categorical variables were analyzed using Chi-square or Fisher’s Exact tests. A linear regression analysis was performed with forced variable entry to identify independent predictors of post-test comprehension. Inter-rater agreements between assessors’ scores were performed on approximately 12% of the population using the kappa (κ) statistic for all the concepts tested. Kappa values of ≥0.7 were considered acceptable. Results ranged from 0.77–1.0 (P<0.001). Data are m(SD) and n (%).

Sample size

a) Parents

Power analysis was based on a previous study in which parental understanding of a standard consent was 8.1(2.3) (scale of 0–12) compared with 9.3(2.2), when using a computer program.[18] Accepting this difference as the smallest difference deemed clinically important, we required a sample of 73/group (α=0.05, β=0.1, two-sided).

b) Children

Sample size was based on previous studies which showed that children’s understanding of standard assent information was 3.7(3.3) (scale of 0–10) and that understanding of research information using an interactive consent program was approximately 50% greater.[17–23] Accepting this difference as clinically important, we required a sample of 67/group (α=0.05, β=0.1, two-sided).

RESULTS

A total of 278 parents and 186 children were approached to participate. Of these, 130 parents and 51 children declined participation. The primary reason for declining was a perceived lack of time to complete the assessments (46.9%). Other reasons included a lack of interest, poor English, and “no reason.” One child was deemed ineligible after enrollment (age criterion) and was withdrawn from analysis. Complete data were thus, available for 148 parents and 135 children.

Participant demographics are described in table 1. As shown, there were no differences in socio-demographics between the TF and IP groups for both parents and children.

Table 1.

Parent and Child Demographics

	TF	IP
Age (yrs):
Parents	40.1 (11.8)	42.0 (10.9)
Children	13.2(1.8)	13.4(2.3)
Gender (F):
Parents	46 (62.2)	52 (71.2)
Children	34 (50.0)	33 (50.0)
Parents’ race/ethnicity:
White	57 (77.0)	53 (74.6)
African American	13 (17.6)	13 (18.3)
Hispanic	3 (4.1)	0 (0.0)
Other	1 (1.4)	5 (7.0)
Education:
Parents
≤ High School	23 (31.1)	18 (24.7)
Some College	23 (31.1)	18 (24.7)
≥ College graduate	28 (37.8)	37 (50.6)
Children school grade: median (range)	8 (4–11)	8 (4–12)
Previous research participation
Parents	23 (30.7)	15 (20.5)
Children	6 (8.7)	13 (19.7)
SORT-R3:
Parents	184.5(27.9)a	182.5(26.9)a
Children	155.8(29.1)b	155.9(35.1)b

Data are presented as mean (SD) or n (%)

TF = Traditional paper format, IP = Interactive digital program,

SORT-R3 = Slosson Oral Reading Test Revised:

^aapproximates 10^th grade reading ability,

^bapproximates 7.0 grade reading ability

Table 2 describes the parents’ and children’s pre- and post- comprehension of clinical trial concepts and their understanding of clinical trial participation. As shown, parents’ and children’s baseline understanding was generally poor.

Table 2.

Pre and Post-understanding by Information Delivery

	TF	IP	Mean difference (95% CI)
Children	n = 68	n = 67
Pre-test understanding of concepts a	3.97 (2.7)	3.76 (2.6)	−0.2 (−1.1, 0.7)
Post-test understanding of concepts a	8.85 (4.1)	11.65 (4.1)	2.8 (1.4, 4.2)*
Δ Pre-Post understanding of concepts a	4.88 (3.4)	7.84 (3.1)	3.0 (1.8, 4.1)*
Post-test understanding of research participation b	12.39 (2.8)	15.86 (2.9)	3.5 (2.4, 4.6)*
Parents	n = 75	n = 73
Pre-test understanding of concepts a	7.67 (3.9)	7.90 (4.1)	0.66 (−1.1, 1.5)
Post-test understanding of concepts a	12.71 (4.0)	13.34 (3.6)	0.64 (−0.6, 1.9)
Δ Pre-Post understanding of concepts a	5.04 (3.3)	5.43 (3.5)	0.39 (−0.7, 1.5)
Post-test understanding of research participation b	16.64 (2.4)	17.36 (2.6)	0.72 (−1.5, 1.6)

Data are presented as mean (SD) and n (%). TF = Traditional paper format, IP = Interactive digital program, Δ = Pre-post change in understanding, CI = Confidence interval.

^*P< 0.001

^a0–18 scale where 18 = complete overall understanding

^b0–14 scale where 14 = complete overall understanding

There were no statistically significant differences in understanding between parents in the TF and IP groups. On the other hand, while children in both groups showed increased post-test comprehension, these improvements were significantly greater among children randomized to the IP group. Overall, children in the 8th grade (approximately 12–13 yrs) or higher had better understanding compared with children in the lower grades although this difference was only significant for the IP group [10.3(4.2) vs 6.96(3.3), 0–18 scale, P < 0.01].

A linear regression analysis using children’s post-test comprehension of clinical trial concepts as the dependent variable and age, grade level, and group assignment as the independent variables identified exposure to the IP (B=2.44, P=0.001) and being in the 8th grade or higher (B=1.60, P=0.015) as independently predictive.

Table 3 describes changes between the pre and post-test measures of understanding of the individual clinical trial concepts. These data are important in highlighting which concepts were poorly understood at baseline and which responded best to the interventions.

Table 3.

Percentage of Participants having NO understanding of Clinical Trial Concepts: Pre- vs Post-test

	Children		Parents
	TF	IP	TF	IP
Clinical Trial:
Pre	46 (66.7)	44 (67.7)	22 (29.3)	22 (30.1)
Post	19 (27.5)	14 (21.2)	11 (14.7)	5 (6.8)
Δ Pre-Post	39.2	46.5	14.6	23.3
Randomization
Pre	38 (55.1)	27 (41.5)	44 (58.7)	33 (45.2)
Post	23 (33.3)	12 (18.2)*	19 (25.3)	11 (15.1)
Δ Pre-Post	21.8	23.3	33.4	30.1
Placebo
Pre	61 (88.4)	58 (89.2)	24 (32.0)	25 (34.2)
Post	38 (55.1)	6 (9.1)*	10 (13.3)	6 (8.2)
Δ Pre-Post	33.3	80.1	18.7	26.0
Single blinding
Pre	67 (98.6)	64 (98.5)	49 (65.3)	44 (61.1)
Post	32 (46.4)	12 (18.2)*	11 (14.7)	6 (8.2)
Δ Pre-Post	52.2	80.3	50.6	52.9
Double blinding
Pre	67 (98.6)	64 (98.5)	56 (74.7)	55 (75.3)
Post	33 (47.8)	9 (13.6)*	11 (14.7)	10 (13.7)
Δ Pre-Post	50.8	84.9	60.0	61.6
Informed Consent
Pre	27 (39.1)	35 (53.8)	9 (12.0)	8 (11.0)
Post	21 (30.4)	28 (42.4)	5 (6.7)	3 (4.1)
Δ Pre-Post	8.7	11.4	5.3	6.9

Data are presented as n (%) TF = Traditional paper format, IP = Interactive digital program

Δ = Pre-post change in understanding,

^*P< 0.05 vs TF

Table 4 describes the participants’ perceptions of the information delivery. As shown, both children and parents randomized to the IP group perceived the information to be significantly easier to follow, clearer, and more effective compared to those receiving the TF.

Table 4.

Perceptions of the Information Delivery

	Children		Parents
	TF	IP	TF	IP
Information quality n (SD)	8.22 (1.6)	8.68 (1.4)	9.01 (1.2)	9.39 (1.0)*
Ability to follow information n (SD)	7.27 (1.8)	8.16 (1.8)*	8.78 (1.6)	9.27 (1.1)*
Effectiveness of presentation:
”Extremely effective” n (%)	29 (42.0)	45 (68.2)*	56 (74.7)	67 (91.8)*
Amount of information:
”Just right” n (%)	60 (87.0)	65 (98.5)*	69 (92.0)	68 (93.2)
Clarity of information:
”Very clear” n (%)	28 (40.6)	48 (72.7)*	62 (82.7)	62 (84.9)

Data are presented as mean (SD) (0–10 scale where 10 = maximum response) and n (%)

TF = Traditional paper format, IP = Interactive digital program

^*P < 0.05 vs TF

Overall satisfaction with iPad graphics and interactivity ranged from 8.59(1.5) – 9.58(0.7) out of 10 (where 10=extremely satisfied). When shown both formats, 67.9% of children and 62.4% of parents reported that they preferred the IP. When asked how they would prefer to receive information if recruited for a future study, all participants reported a preference for digital information.

Results of the in-line exercises indicated that both parents and children showed moderate to excellent real-time understanding of the information. For example, when asked to select from a number of options, 72.2% of parents and 71.2% of children correctly selected “double-blind means that neither the patient nor the doctor knows who is getting the new treatment.”

DISCUSSION

Results from this study showed that parents’ and children’s baseline understanding of clinical trial concepts was generally poor. These results are in accordance with other studies showing poor understanding of commonly used research terms such as randomization and placebo.[10, 11, 24] The reasons for this are multifactorial but are due in part to the fact that investigators either do not explain the terms and/or that consent forms have become overly complex. In one study, Kodish et al. showed that randomization was not explained by 17% of physicians involved in pediatric leukemia trials and that 50% of parents did not understand the term.[25] This lack of understanding likely plays a role in the concept of “therapeutic misconception” wherein participants confuse treatment with research and are often unaware that group allocation occurs by chance.[24, 26] Difficulties were also observed with other concepts; for example, one participant believed that a placebo was “a part of the body” and another defined double-blinding as “2 clinical trials where 2 children are blind, maybe twins.” Despite this, the results show that providing information improved both parents’ and children’s comprehension and thus, reinforce the importance of explaining not just the details of a study but also the terms used. Clearly, if these basic terms are not understood, it is not surprising that many participants have difficulty understanding more complex issues involving risk-benefit trade-offs.

Important, was the observation that children in the IP group had significantly greater comprehension of trial concepts and participation compared with children receiving the traditional paper format. The theory behind this observation is grounded in the so-called pictorial superior effect (PSE) which posits that information provided in pictorial format is easier to understand and requires less cognitive effort compared with text.[27–29] Furthermore, data suggest that interactive programs enhance understanding because they promote active learning.[27–31] However, given the known benefits of PSE, it is unclear as to why parents did not enjoy the same parallel increase in comprehension with the IP. Although the data suggest trends towards quantitatively greater understanding among parents in the IP group, these were not statistically significant. This is surprising given that recent studies show that interactive computer-based multimedia programs increase patient understanding of treatment information compared with standard text.[18, 22] One possible explanation is that because our parent population was skewed toward those with good education and literacy, any potential positive impact of the IP on understanding was less pronounced. It may also reflect the current generation’s facility with digital media. That being said, the majority of parents and children perceived the digital information to be easy to follow, very clear, and effective as a means of information delivery. Furthermore, the iPad program was deemed easy to use and satisfaction with the digital format was uniformly high. When shown both the TF and IP formats, the majority of parents and children reported that they preferred the IP, suggesting greater acceptance of this technology as a means to convey information.

The use of exercises with corrected feedback has been shown to be effective in promoting retention of information.[32, 33] Given that longer-term retention of information is typically poor, this approach allows the investigator to assess understanding at the time decisions are actually made. These exercises are also useful in identifying problem areas and in directing investigator efforts towards targeted discussion. The interactive nature of these exercises is also important as a means to promote active learning and promote understanding [22, 34–36] and may explain, in part, why children appeared to perform equally well as parents on the in-line exercises. Recently, we showed that patients who performed better on in-line exercises regarding their cardiac catheterization also had better overall understanding of the procedure.[22]

Limitations to this study are acknowledged. First, we fully recognize that the requirements for assent are different from those of parental permission and thus children would not necessarily be expected to understand all the concepts provided.[37, 38] In general, children are only required to understand the purpose of the study, what will happen to them, and that their participation is voluntary. However, studies [4, 39] suggest that some children are able to understand information beyond these basic elements and, importantly to that end, these results highlight the potential of interactive multimedia to help children do so. Second, this study represents one intervention at one institution. Given that the patient population at the University of Michigan is relatively educated and predominantly White, these results may not be generalizable to all patient populations.

In summary, this study emphasized the importance of providing parents and children with information to help them understand important clinical trial terms/concepts as well as their role as research subjects. Of note, was that among children, the use of digital information resulted in a significant increase in understanding compared with the traditional paper format. This suggests that this approach may be a better way of actively engaging children in a manner that is familiar to them and which can enhance their ability to provide assent. Although this finding was not observed to the same extent among parents, it is worth noting that all participants found the IP to be more effective in presenting information compared with the more traditional paper format and reported that they would prefer digital formats as a vehicle for receiving future research information. These results, therefore, support the use of interactive digital multimedia as a means to help research participants, particularly children, understand the fundamental aspects of clinical trials and thus, provide an important platform for shared informed decision-making.

What is already known on this topic

• As the number of pediatric clinical trials has increased, so too has the amount and complexity of the information provided.

• Many parents and children have difficulty understanding the information provided to them and often struggle to make informed decisions.

What this study adds

• To date, few alternatives to the traditional paper consent document for research and treatment exist.

• Results from this study suggest that interactive digital multimedia may be useful in helping children better understand clinical trial information and enhancing informed decision-making.