Do Illustrations Improve Children's Comprehension of Assent Documents?

Abstract

OBJECTIVE

The purpose of this study was to assess children's comprehension of a new assent booklet, (K_idS_ent), which uses pictures and written information.

STUDY DESIGN

A randomized, crossover study evaluated the comprehension of assent documents by children, 7 to 11 years of age at a local elementary school. The two types of documents tested were the standard assent form and the K_idS_ent Assent Booklet. Participants were randomized as to which test document they received first by using a cluster randomization design. Participants read the document and then took a short quiz. The process was repeated for the other document on a separate day. Study participants were assigned a percentage score and a binary perfect score for each quiz. Mixed effects logistic and linear regression models with random intercepts were applied to the continuous percent quiz scores and binary perfect quiz scores, respectively.

RESULTS

A total of 190 participants completed the standard quiz, and 195 students completed the booklet quiz. A statistically significant difference in perfect quiz scores (p=0.004) and percent quiz scores (p≤0.001) between booklet and standard form was noted.

CONCLUSIONS

The quiz scores may indicate that the style of document is not the only factor influencing participant understanding.

INTRODUCTION

Since children cannot legally consent to participate in research on their own, federal regulations generally require that parents or guardians give permission to allow a child to participate in a research study. Guidelines published in the Federal Register in 1983 mandate that research protocols that involve children must also contain methods for obtaining a child's assent in addition to parental permission.¹ The assent is defined in these regulations as “a child's affirmative agreement to participate in research” and is further specified that “mere failure to object should not, absent affirmative agreement, be construed as assent.” Although federal regulations require children's assent, there is little guidance as to how the assent requirement should be completed, therefore leaving this determination to the reviewing Institutional Review Board (IRB). In particular, the regulations do not identify which children are capable of assent but rather state only the IRBs should consider the “ages, maturity, and psychological state of the children involved.” In addition, there are no specifics in the regulations that identify the information that must be provided to children as part of the assent process nor any guidance on how investigators and review committees should implement this requirement in practice.¹ There are no templates provided by the governmental agencies that can be used to develop an appropriate assent document, and the IRB is allowed discretion as to the requirements for obtaining and documenting assent.²

Data suggest that providing written as well as verbal information to children may enhance their understanding of research participation.³ In a pediatric setting, this is usually accomplished by having the investigator talk with the study participant and giving the subject a simple handout that explains the study in a language that a child can understand. At the time of this study, the University of Mississippi Medical Center used a separate assent document for children 7 to 18 years of age in a short format containing study information in paragraphs that the investigator would read with the child. The child would then sign his or her name if he or she agreed to participate. Other studies have investigated alternate formats for children's assent documents by modifying the standard forms to include bullets, bolding, increased font size, pictures, and the use of video technology.^4,5 To better accommodate children's preferences, however, their learning materials are typically accompanied by large-print words and illustrative images which tend to capture a child's attention. Therefore, these formats should be used to improve comprehension of assent documents.⁶ To incorporate a child-friendly format into an assent document, the K_idS_ent Assent Booklet, was created which used pictures along with written information so that it resembled a storybook. Previously, in a small prospective, randomized, crossover study, the K_idS_ent Assent Booklet was evaluated for improved comprehension by children.⁷ Even though there was no significant difference between the booklet and standard form on quiz scores, 100% of participants preferred the K_idS_ent booklet, and nearly twice as many participants considered the K_idS_ent booklet easier to understand than the standard form.⁷ With this large difference in perceived ease of understanding, the authors expected a significant difference in quiz scores indicating a better understanding when the K_idS_ent booklet was used. However, the actual time and thoroughness of the interaction of the investigator in explaining either document may have been the key to the similarity of scores. To test this hypothesis, a study was designed with a larger sample size to examine the comprehension of the information presented in both styles without an accompanying investigator explanation.

MATERIALS AND METHODS

Study Design

A prospective, randomized, crossover study was conducted to evaluate children's comprehension of two different assent document formats. The study was approved by the University of Mississippi Medical Center Institutional Review Board. Parental permission and verbal assent were obtained for each child who participated.

Study Population

Participants were between 7 and 11 years of age and enrolled in a general education curriculum for second through fifth grades in a local public elementary school. All children (n= 360) in the second through fifth grade classes at the school were given a study packet to take home. The packet contained a description of the study and the parental permission and demographic data forms. A total of 217 children returned completed permission forms. The participants in this study were not being enrolled in the clinical research studies that the test assent documents represented but were only participating in the study described here.

Description of the Assent Documents

The two types of assent documents tested in this study consisted of the standard assent form and the newly developed K_idS_ent Assent Booklet. Each test document focused on a different research study derived from ongoing IRB-approved clinical trials, both of which were considered minimal risk. The standard form was for a blood pressure study, and the booklet was for a study of gastroesophageal reflux. Because the same participants would view each of the test documents, the use of documents for different studies was necessary to prevent bias due to carryover of information. The standard assent form consisted of two pages that explained all of the pertinent study information in paragraph form. The pertinent information included major procedures, risks (“bad things”), benefits (“good things”), and a signature line. The K_idS_ent Assent Booklet contained the information in sentences with pictures to go along with them. These pages were folded to form a booklet. The K_idS_ent Assent Booklet also provided information regarding purpose, procedures, and risks and was 16 pages in length. The extra pages were necessary to include illustrations and did not necessarily reflect a large increase in text.

Procedures

The testing of the two documents took place at a local public elementary school in a general classroom setting. To prevent investigator influence, each classroom teacher provided basic procedural instructions to the students, distributed the test documents, and collected the quizzes. No verbal assistance was provided about the test document in order to test the study hypothesis. If students asked questions, they were told by the teacher to do the best they could.

Two types of assent documents were given to each participant. Participants were randomized as to which test document they received first to decrease bias. Participants in test order AB (Booklet/Standard) received the Booklet first and then received the Standard form, while participants in test order BA (Standard/Booklet) received the Standard form first followed by the Booklet. The order in which the two assent documents were reviewed by the participant was determined by a cluster randomization design. Children were nested within classes, but classes as a unit of allocation were randomized to one of the test orders (AB/BA) within each grade, so all children from any given class received the same test order. The participant was provided with the first document, and a short quiz was then administered. This process was then repeated on a separate school day for the second document, followed by a study appropriate quiz. The testing day schedule was set by the principal with a 3-day lapse between testing days. The quizzes had the same six stem questions for each document, with slight variations in the multiple-choice answers to ensure equal testing of both documents (Table 1). Questions were based on the institution's suggested comprehension assessment of the consenting process. After both test documents were viewed by the child and both sets of questions were answered, the participants indicated which document type they thought they understood better. At the end of data collection, each study participant was assigned a percentage score on the booklet quiz and on the form quiz. Other information collected through a parent questionnaire included age, gender, race, grade in school, history of attention deficit hyperactivity disorder, and history of any learning disabilities.

Table 1.

Stem Questions from Quizzes

Statistical Analysis

Descriptive statistics were computed as frequency counts and percentages for categorical variables. This crossover trial comparing the effects of the booklet to the standard assent form was analyzed as a longitudinal study with two time points. Perfect scores were calculated as a binary response variable based on whether children correctly answered all six questions from the test given (perfect score = score of 6, nonperfect score = score of 1–5). Mixed-effects logistic regression models with random intercepts were applied to the binary perfect scores to account for correlations arising from the repeated-measures crossover design. Additionally, average quiz scores were calculated as the percentage of correct answers. Mixed-effects linear regression models were investigated as well as mixed-effects ordinal regression models, given the moderately categorical average scores arising from the limited potential correct answers; random intercepts were applied to both models. The results presented were from the mixed-effects logistic and linear regression models for ease of interpretation; other results are available from the authors upon request. The unadjusted models include the main effects for test type and period and their interactions. The adjusted models additionally include age, gender, and race. Sensitivity analyses examining potential carryover (ordering) and period (better scores on second test) effects were conducted. Nonsignificant carryover effect was eliminated to achieve more parsimonious models. Results from the mixed-effects linear regression models including nonsignificant age by gender interactions were also reported to be consistent with the results from the mixed-effects logistic regression models. Results were expressed as odds ratios for logistic models and regression coefficients for linear models and their corresponding 95% confidence intervals and p values. The significance level for two-sided hypothesis testing was set at 0.05. Stata version 11.2 (Stata Corp, College Station, TX) software was used to conduct statistical analyses.

RESULTS

Parents of 217 children gave their permission to participate. One participant was excluded from the analysis because this participant completed the wrong quiz on both study days. Demographic data captured from the parent questionnaire was incomplete for some of the participants. The study participants were predominantly white (89%). The gender was nearly even with 49.3% males and 50.7% females. Table 2 describes the complete demographic characteristics of the sample.

Table 2.

Baseline Demographics

Participants in test order AB (n=111) received the Booklet first and then received the Standard form, while participants in test order BA (n=105) received the Standard form first, followed by the Booklet. Quiz data were analyzed for any participant completing a quiz. Sixteen participants did not complete either quiz, resulting in a total of 200 study participants. Due to absence from school on one of the study days, 10 participants did not complete the standard quiz, and 5 did not complete the booklet quiz.

The percentage of children who had a perfect score from the booklet form was 22.1% (95% CI, 16.2–27.9), whereas, 34.7% (95% CI, 27.9–41.6) of children had a perfect score from the standard form (Figure 1). A higher percentage of females had a perfect score from both standard and booklet tests compared to males (Figure 1). As shown in Figure 2, the likelihood of receiving perfect scores increased by age. In addition, the difference in percentages of perfect scores between females and males also changed by age. The percentage of females who had a perfect score were lower at age 7 but higher at age 10 than those of males for both standard and booklet forms (Figure 2). The percentage of perfect scores was highest for 10-year-old females from the standard form, 70.8% (95% CI, 52.2–89.5). Figure 3 shows the relationship between average quiz scores as a continuous outcome and test type, period, gender, and race. Similar relationships for average quiz scores were observed with those for percentage of perfect score (Figure 4).

Figure 1.

Relationship between perfect score and test type, period, gender, and race A=K_idS_ent Assent Booklet; B=standard assent form

Figure 2.

Percentage perfect score by age and gender Solid circle, booklet-male; Hollow circle, booklet-female; Solid triangle, standard-male; Hollow triangle, standard-female

Figure 3.

Relationship between average quiz scores and test type, period, gender, and race

A=K_idS_ent Assent Booklet; B=standard assent form

Figure 4.

Average quiz score by age and gender

Solid circle, booklet-male; Hollow circle, booklet-female; Solid triangle, standard-male; Hollow triangle, standard-female

Using the mixed-effects logistic regression models, a statistically significant difference between the booklet form and the standard form was found (p=0.004). The odds of receiving a perfect score were estimated to be 2.6 times higher for the standard quiz than the booklet quiz. This difference between booklet and standard forms remained significant after adjusting for age, gender, and race (p=0.001). Results also indicated statistically significant effect of gender (p=0.023). Females were about two times more likely to have a perfect score than males. The odds of receiving a perfect score increase by age for both test types (p<0.001). There was no statistically significant effect of race (p=0.948). Findings from the mixed-effects logistic regression models also revealed statistically significant age by gender interactions (p=0.05). The odds of receiving a perfect score for females and males differed by age. A statistically significant difference between females and males for 10 years of age was found (p=0.004), but this difference was found not to be statistically significant at ages 7, 8, 9, and 11 (Table 3). The odds of receiving a perfect score were approximately 4.8 times higher for females than for males at age 10 (Table 3). No statistically significant carryover effect was found.

Table 3.

Mixed effects logistic and linear regression models results

The overall results from the mixed-effects linear regression models were similar to those from the mixed-effects logistic regression models in terms of statistically significant differences between the booklet and standard assent tests. Unadjusted mixed-effects linear regression models, including main effects of test type and period, revealed a significant difference in percentages of quiz scores between booklet and standard forms in both periods (p=0.001 for the first period and p=0.009 for the second period). The average quiz scores for all participants from the standard and the booklet tests were estimated to be 78.3% (95% CI, 74.8–81.8) and 71.6% (95% CI, 68.2–74.9) in the first period and 78.7% (95% CI, 75.3–82.1) and 72.0% (95% CI, 68.2–74.9) in the second period, respectively. The interaction term of age with gender was found not to be statistically significant, but it was kept in the model for consistency. We found borderline significant carryover effect after adjusting for age. However, it did not change the significant difference between booklet and standard tests. It should be noted that fairly categorical average scores arising from the limited potential correct answers were used to apply mixed effects linear regression models.

A statement was included on the second quiz asking the student to identify which study document they thought they understood better. Of those participants who responded (n=134), 78.4% indicated that they understood the K_idS_ent booklet form better, while 20.2% indicated that they understood the standard form better.

DISCUSSION

Even though both researchers and regulatory agencies agree that informed consent is a necessary process for any type of research, there is still controversy concerning the assent process. While there is agreement that it is important to inform study participants about research, it is unknown how much study information a minor can comprehend. Informed consent documents are required to be written in simple language that can be understood by people who are not in the medical or research fields. When used for obtaining assent, most of these documents are in a basic form with a title and paragraphs that include child-friendly language to convey the important study issues. While the information is simplified to a child's level, it is not in a format that a child is accustomed to viewing.⁶ Children generally read colorful books with large words and pictures.

This study evaluated the value of using a child-friendly assent document to improve understanding of research studies by participants. Even though the majority of students indicated that they understood the assent booklet better, the results did not support a better understanding by using a storybook format to relay medical information. These findings are similar to those reported by Tait et al⁴ in that 81.3% of children preferred the modified form used in that study.

In the previous K_idS_ent study evaluating the comprehension of these same study documents, a nonsignificant difference was found in quiz scores. However, it was postulated that the actual time and thoroughness of the investigator's interaction in explaining either document contributed to the similarity of scores.⁷ This study was therefore designed to eliminate that confounder by participants having no verbal interaction with the investigator concerning the content of study documents. Yet, this lack of verbal interaction may have contributed to the lower test scores of approximately 10 points in the current study. With only 22% and 35% of participants having perfect scores on either the booklet or standard form, respectively, in the current study, neither format met expectations of documenting thorough understanding. These results are similar to those of recently published studies in which overall assessment scores were below the normally accepted level for passing.^4,5,7 Tait et al⁴ did show a trend toward greater understanding of all informational elements by those children who were assigned the modified form. This difference in findings from the study presented here may have been due to a difference in patient population. Participants in the study by Tait et al⁴ were 7 to 17 years old and had been hospitalized after an elective surgery or medical condition, whereas our study participants were in a nonmedical environment and were 7 to 13 years old. Although O'Lonergan and Forster-Harwood⁵ reported a slight improvement in comprehension using video technology, neither the scores by the children nor their parents were at a level that would ensure acceptable comprehension. That study did not include an entire research protocol but was limited to two common pediatric research procedures.⁵ The results of this study support the adoption of the upper recommended age for assent in research studies.

Many institutions still have the age of assent as 7 years old; however, recently, organizations have published support for increasing the age to 9 years old.² In this study, children under the age of 9 performed poorly compared to their older counterparts. At age 7, 19% of the children taking the Standard test had a perfect score, while 6% of the children taking the Booklet test had a perfect score. On the other hand, at age 10, 57% of the children taking the Standard test had a perfect score, while 36% of the children taking the Booklet test had a perfect score.

Because the study was conducted in a school that was designated as a high-performing school according to the No Child Left Behind Act of 2001 and met the required yearly growth status, the lower-than-expected quiz scores cannot be attributed to the academic performance of the school. However, the impact of a nonmedical environment may have contributed to a lack of understanding of the medical terms and descriptions used in the test documents.⁶ In contrast to the previous K_idS_ent study, the study participants may not have had medical experience or exposure to relate to the pictures in the booklet.

Another possible limitation to the study may have been the inclusion of different studies for each assent format tested rather than using a four-arm crossover design. This would have allowed us to evaluate the Standard versus Booklet forms while avoiding the aliasing of the GI and BP substantive areas; however, this design would not have been logistically feasible. The authors chose the design as reported to reduce complete carryover of information that could have occurred if the same study information had been used for each document format. The chosen study design was counterbalanced by the large sample size and the cluster randomization scheme.

Even though the majority of participants reported that they understood the K_idS_ent Assent Booklet, the quiz scores may indicate that the style of the document is not the only factor influencing participant understanding. This study supports the need for appropriate communication with study participants to ensure adequate understanding of the research process. Additional research evaluating optimal communication modalities and age appropriate formats need to be undertaken.

ABBREVIATIONS

IRB

Institutional Review Board