Effectiveness of an Informational Video Method to Improve Enrollment and Retention of a Pediatric Cohort

Abstract

Objective

The Environmental Determinants of Diabetes in the Young (TEDDY), a multinational epidemiological study, is designed to identify environmental exposures triggering autoimmunity and type 1 diabetes (T1D) in children at increased genetic risk. The objective of this analysis was to evaluate the use of an informational video in the enrollment and retention of eligible participants at the Colorado TEDDY clinical center.

Study Design and Setting

Eligible participants were divided into two groups based on the inclusion of the video in the enrollment materials: the No-Video Group (n=449) did not receive the video and were contacted between 7/1/07 and 6/30/08. The Video Group (n=494) received the video and were contacted between 7/1/08 and 6/30/09. Multiple logistic regression compared the enrollment rates (percent of eligible subjects deciding to enroll) of those who received the video compared to those who did not. Kaplan-Meier survival analysis and a multivariate Cox proportional hazards model compared the differences in study retention, as defined by active participation fifteen months after the baseline visit at three months of age.

Results

Both groups were demographically similar. The enrollment rate was significantly higher for the Video Group (56.9%) compared to the No-Video Group (49.9%). Differences remained significant with adjustment for other known factors. A difference in retention between the two groups was not observed.

Conclusion

Methods and materials increasing understanding and more accurately informing participants of what is involved in participation may increase enrollment in a prospective observational study.

Introduction

The use of audio-visual technology may increase enrollment in research studies by providing a consistent delivery of information while allowing participants to control the pace of the viewed information (1, 2). Video methods have been shown to increase participant’s knowledge and positive attitude towards clinical trial participation (3, 4). Furthermore, video may be beneficial in improving participation while helping potential participants have a better understanding of the studies (5). Prior studies examining the willingness to consider future trials and the effectiveness of audio-visual interventions on increasing enrollment rates are limited to hypothetical studies using video to educate participants on a particular medical condition instead of including participants with the specific condition being studied (5, 6). Other studies are limited to the measure of attitudes associated with participation without actually measuring the impact of these methods on the study enrollment decision (3, 4).

The recruitment of pediatric participants in long term research studies can be rather challenging due to the protection of children as human participant and ensuring parents are appropriately informed of the protocol since they make a decision on behalf of the child (7, 8). It is difficult to determine how well parents comprehend the basic points of a study protocol and how well an informed consent is meeting its intended purpose. Efforts that try to improve the efficacy of this process are important to assess (9). Interventions such as multimedia technology, enhanced consent forms and discussions with research personnel have been developed to assist in this improvement (10).

The primary aim of this study is to assess the effectiveness of an 8-minute professionally produced video in improving the rate of enrollment and minimizing attrition of pediatric participants recruited at one clinical center of TEDDY conducted through the Barbara Davis Center for Childhood Diabetes at the University of Colorado Denver Anschutz Medical Campus.

Materials and Methods

The Environmental Determinants of Diabetes in the Young (TEDDY) Study is an observational study designed to identify the environmental exposures that promote or protect against autoimmunity and type 1 diabetes mellitus (T1D) in children with an increased genetic risk (11). The study is divided into two phases. The first phase is the screening of newborns for known high-risk genes. The second phase is the follow-up of those identified with the high-risk genes that were subsequently invited and decided to enroll in a 15-year observational study. It is conducted in six clinical sites, each with similar but independent methods for enrolling eligible participants into follow-up study.

Enrollment into the Follow-Up Study

In September 2004, the Colorado site of TEDDY began screening the umbilical cord blood of newborns in Denver metropolitan hospitals for genes associated with a higher risk of developing T1D. The enrollment phase into the follow-up study began when the parents of the children identified with the high-risk genes were contacted by the clinic staff within ten weeks to explain their child’s genetic risk and to present the follow-up study for which they were eligible.

A standardized phone script used by eight to ten clinic staff was structured to reflect the points made in the informed consent document signed by parents who agreed to the follow-up study at the first clinic visit. How the child was identified, the results of the genetic screening, and the details of the follow-up study protocol were all covered in the initial call, which took up to twenty minutes. A packet including information on T1D and the expectations of the study was mailed before subsequent phone calls were made two to three weeks later for a final decision to enroll in the follow-up study.

By the end of the third year of enrollment, rates were lower than originally projected. The staff involved in the enrollment efforts was challenged with an increasing number of eligible participants as a result of greater screening efforts and the amount of time the enrollment protocol required. For these reasons, several strategies were considered to better inform participants in a more efficient manner about the nature of the study in the hopes of increasing enrollment rates. Group information sessions, a strategy successfully employed in at the Finland TEDDY clinical center, were initially tried but were poorly attended. In 2008, the decision was made to produce an informational video based on the information being presented at the group information sessions and in the initial phone calls.

A high-quality DVD was professionally produced by the University of Colorado Denver Anschutz Medical Campus Media Services at a cost of approximately $5000 (12). With signed consent from the involved parents, the video included participants and their parents interacting with clinic staff during a clinic visit, as well as in their home discussing data collection elements the parents do outside of the clinic visit. An interview with the principal investigator and a professional narrator explained the study purpose and expectations. The main purpose of the video was to enhance the understanding of the study rationale while providing a visual context of the study. The DVD made it possible to standardize the information received by parents about TEDDY, was a more time-efficient method for presenting the follow-up study and provided a basis for eligible participants to ask informed questions of the clinical staff.

In July 2008, the DVD was added to the information packet mailed out to all eligible study participants who had voiced interest in receiving more information about the TEDDY follow-up study. This analysis compares the enrollment rates among eligible participants one year before and one year after the initiation of the DVD distribution to evaluate the effectiveness of the video method on improving enrollment into the follow-up study. It also examined the impact on early retention, or active participation fifteen months after the baseline visit, which occurred at three months of age for all subjects.

Participants

To examine the enrollment differences based on receipt of the video, we selected two subgroups of the overall TEDDY Colorado population that were eligible for participation in the follow-up study. The No-Video Group (n=1059) was screened, determined to be eligible, and first contacted about enrollment into the follow-up study between 7/1/07 and 6/30/08. The Video Group (n=1160) was screened, determined to be eligible and first contacted between 7/1/08 and 6/30/09. In both groups, participants were excluded if they were deemed ineligible for enrollment due to inability to complete the baseline visit before 4.5 months of age (No-Video Group: 424: Video Group: 508), mothers under age eighteen at time of child’s birth (No-Video Group: 14: Video Group: 1), and who preferred Spanish (No-Video Group: 172; Video Group: 157), since the DVD was not produced in Spanish because of budget constraints. The final sample size for this analysis included a No-Video Group of 449 and a Video Group of 494.

For the early withdrawal analysis, retention was defined as active participation in study visits beginning with the baseline visit at three months of age through eighteen months of age. Participants were considered to be disenrolled if they had actively withdrawn from the follow-up study, or had missed four consecutive visits during the 3-18 month period. All participants in this analysis had the opportunity to be enrolled for this length of time.

To assess the effectiveness of the video while adjusting for other known characteristics thought to influence enrollment and retention in this study, demographic and family factors were evaluated. Variables were collected from the hospital screening interview for all participants who agreed to the screening phase. These characteristics included the ethnicity/race of child based on self-report of the mother according to the NIH revised 1997 OMB standards (13), maternal age at the child’s birth, having a first degree relative with type 1 diabetes (FDR), and having a sibling participating in TEDDY or the Diabetes Auto-Immunity Study in the Young (DAISY), a study conducted at the same center. Season of birth was also examined because of known differences in numbers of births by season.

Statistical Analysis

Differences between the two groups were assessed using chi-square tests. Multiple logistic regression was used to examine the impact of the video on the likelihood of enrolling adjusting for other known factors described above. Kaplan-Meier survival analysis and a multivariate Cox proportional hazards model, adjusting for the previously mentioned variables, were used to assess difference in retention between the two groups. All analyses were performed using SAS Version 9.1 (14).

Results

The characteristics of the two groups are described in Table 1. The No-Video and the Video Groups were examined for differences on other factors that could affect the decision to enroll or stay in the follow-up study: gender, race/ethnicity, family history of T1D, maternal age, sibling in TEDDY or our other study, DAISY, and season of birth. No differences were found between the No-Video and Video Groups for any of these variables.

Table 1.

Demographic and Family Characteristics of the No-Video and Video Groups Colorado Center of the TEDDY Study, 2007-2008

	Pre-Video N=449	Post-Video N=494	P-Value
Gender
Female	213 (47%)	240 (49%)	0.73
Male	236 (53%)	254 (51%)
Race/Ethnicity
Non Hispanic White	314 (70%)	336 (68%)	0.65
Hispanic	116 (26%)	140 (28%)
Other	19 (4%)	18 (4%)
Family History of T1D1
No	423 (94%)	460 (93%)	0.49
Yes	26 (6%)	34 (7%)
Maternal Age at Child’s Birth
18-25	93 (21%)	94 (19%)	0.57
25-35	261 (58%)	304 (62%)
>35	95 (21%)	96 (19%)
Sibling in TEDDY/DAISY2
No	438 (98%)	477 (97%)	0.37
Yes	11 (2%)	17 (3%)
Season of birth
Winter	112 (25%)	109 (22%)	0.10
Spring	98 (22%)	136 (28%)
Summer	117 (26%)	138 (28%)
Fall	122 (27%)	111 (22%)

¹T1D: Type 1 diabetes

²TEDDY: The Environmental Determinants of Diabetes in the Young

DAISY: Diabetes Auto-Immunity Study in the Young

Enrollment into Follow-up

The results of the multivariate logistic regression analysis are presented in Table 2. In order to assess the independent contribution of the video with other factors known to affect enrollment decisions, three models are presented: 1) the unadjusted odds of enrolling associated with each covariate, including receipt of the enrollment video; 2) an adjusted model that excludes the two strongest predictors of enrollment: family history of T1D and sibling in TEDDY or our related study DAISY; and c) a final fully adjusted model with all covariates.

Table 2.

Logistic Regression Models Assessing Odds of Enrolling in a 15-Year Follow-up Study Colorado Center of the TEDDY Study, 2007-2008.

	Model 1: Univariate Model		Model 2: Sibling in DAISY/TEDDY and FDR1 Variables Excluded		Model 3: Full Model
	Odds Ratio	95% CI	Odds Ratio	95% CI	Odds Ratio	95% CI
Received Video	1.34	1.03-1.73	1.40	1.08-1.82	1.37	1.05-1.80
Female	0.93	0.72-1.21	0.96	0.74-1.24	0.95	0.73-1.24
Race/Ethnicity
Non-Hispanic White (Ref)	1.00		1.00		1.00
Hispanic	0.66	0.49-0.88	0.68	0.50-0.93	0.67	0.48-0.92
Other, Non-Hispanic	0.90	0.46-1.75	0.93	0.47-1.83	0.82	0.40-1.67
Maternal age (1 year increase)	1.03	1.01-1.05	1.02	0.99-1.05	1.02	0.99-1.05
Season
Winter (Ref)	1.00		1.00		1.00
Spring	0.85	0.59-1.23	0.80	0.55-1.16	0.84	0.57-1.23
Summer	1.20	0.84-1.73	1.19	0.83-1.72	1.25	0.86-1.82
Fall	1.57	1.08-2.29	1.63	1.11-2.37	1.66	1.13-2.45
Family History of T1D2	4.68	2.34-9.35	N/A		4.28	2.11-8.65
Sibling in TEDDY/DAISY3	11.78	2.78-49.93	N/A		11.40	2.65-49.01

¹FDR: First Degree Relative of a type 1 diabetic

²T1D: Type 1 diabetes

³TEDDY: The Environmental Determinants of Diabetes in the Young

DAISY: Diabetes Auto-Immunity Study in the Young

Of the 494 participants in the Video Group, 282 (56.9%) enrolled, compared to 224 (49.9%) of the 449 participants in the No-Video Group. The unadjusted odds ratio for enrollment comparing the two groups was 1.34 (95% CI: 1.03-1.73) (Table 2: Model 1). The covariates examined that were found, as expected, to be associated with an increased likelihood of enrollment were family history of T1D (OR =4.68, 95% CI 2.34-9.35) and having a sibling in TEDDY or DAISY (OR =11.78, 95% CI 2.78-49.93). Participants of Hispanic ethnicity were less likely to enroll compared to non-Hispanic Whites (OR =0.66, 95% CI 0.49-0.88). Age of mother was not associated with enrollment. An unexpected difference found to be associated with enrollment was season of birth, with participants born in the fall more likely to enroll than those born in spring or winter (OR =1.57 95% CI 1.08-2.29).

Although the No-Video and Video Groups were not significantly different for family history of T1D or having a sibling already enrolled in TEDDY/DAISY (Table 1), these factors can greatly influence the decision to enroll in TEDDY. We performed two logistic regression analyses (Table 2), the first excluding these variables (Model 2), and the second a fully adjusted model that included all variables examined (Model 3). The odds for enrollment associated with the receipt of the video did not change significantly when the variables indicating FDR status or having a sibling in TEDDY/DAISY were not included: (OR = 1.40, 95% CI 1.08 – 1.82) compared to the fully adjusted model that included these variables (OR=1.37, 95% CI 1.05 – 1.80). When interactions between these covariates and the No-Video/Video Groups were assessed, no differences in patterns of association were observed. These analyses demonstrate a significant and independent effect of receiving the video on increasing enrollment.

Retention in Follow-up Study

To assess the impact of the video on early study retention, the No-Video group and the Video Groups were compared with a multivariate Cox proportional hazards model and with Kaplan-Meier survival analysis. As demonstrated in Table 3 and Figure 1, those who received the video at the time of enrollment appeared less likely to have disenrolled from the follow-up study by eighteen months of age, although this difference was not statistically significant (HR for Video Group = 0.86, 95% CI 0.52 – 1.43). Participants who were non-White/non-Hispanic were more likely to disenroll when compared to those who were NHW and Hispanic (HR= 3.53, 95% CI 1.51-8.26). Increasing maternal age was protective against early withdrawal, with the risk of withdrawal decreasing by a factor of 0.9 for every one year increase in maternal age (HR=0.90, 95% CI 0.86-0.95). Stated differently, younger mothers were more likely to drop out by the time the child was eighteen months old. Having a first-degree relative with T1D was associated with staying in the study, but this association was not statistically significant.

Table 3.

Predictors of Withdrawal by the Age of 18 months, Cox Proportional Hazard Model, Colorado Clinical Center of the TEDDY Study, 2007-2008.

	Hazard Ratio	95% CI
No Video	1.00
Received Video	0.86	0.52-1.43
Female	1.15	0.68-1.95
Race/Ethnicity
Non-Hispanic White (Ref)	1.00
Hispanic	1.58	0.88-2.83
Other, Non-Hispanic	3.53	1.51-8.26
Maternal Age (1 year increase)	0.90	0.86-0.95
Family History of T1D1	0.16	0.02-1.14
Sibling in TEDDY/DAISY2	0.51	0.12-2.13

¹T1D: Type 1 diabetes

²TEDDY: The Environmental Determinants of Diabetes in the Young

DAISY: Diabetes Auto-Immunity Study in the Young

Figure 1.

Comparison of Early Withdrawal Among the No-Video and Video Groups Colorado Center of the TEDDY Study

Discussion

The introduction of this informational video to the enrollment protocol for the Colorado clinical center of the TEDDY Study provided a quasi-experimental design in which we were able to assess this as a strategy that might improve enrollment rates and support early retention. Enrolling a representative cohort and retaining participants are critical for minimizing selection and drop-out biases, therefore increasing the generalizability of prospective study findings. The intent behind using an informational video was to improve potential participants’ understanding of the study protocol so that a better informed decision to participate could be made, as well as saving staff time and presenting the information in a standard format.

In this setting, we were able to demonstrate that being in the group that received the video in addition to the other elements of the enrollment protocol was independently and positively associated with increased enrollment rates of a pediatric cohort. There was an absolute 7% higher enrollment rate in the Video Group (56.9% vs. 49.9% of the No-Video Group). While adjusting for other known factors associated with enrollment, those who received the video were 37% more likely to enroll than those who did not receive the video.

With this analysis, it was also possible to assess early drop-out risk between those who did and did not receive the video. Although there was not a significant difference in retention rates between the two groups, the survival curves show a slight difference in a positive direction, indicating that while the video did not significantly improve retention rates, it appears to have had a slightly positive impact on retention.

A recently published analysis that included all clinical centers of the TEDDY Study (n=3,757) found that country of residence, young maternal age, lack of father participation and female gender of the child to be significant predictors of early withdrawal in the first twelve months of study participation in the general population participants (15). Further models from the study-wide analysis also suggested maternal smoking and abstaining from alcohol during pregnancy, reducing their work hours or not working at all during pregnancy, and those who underestimated their child’s risk for T1D as predictors of early withdrawal (16). It was not practical to include all of these factors in this analysis because of the site specific sample size and there was only two years of enrollment being examined in our eighteen-month withdrawal assessment.

The Colorado-specific analysis reported here included an evaluation of maternal age and gender as well as the evaluation of the video and ethnicity/race. This analysis confirms the finding that younger mothers are significantly more likely to withdrawal. Ethnicity/race was particularly important for the Colorado site to evaluate because of the Hispanic representation, both Spanish-speaking preferred and English-speaking, at our site. Although the Spanish-speaking participants were not included in this analysis because the video was not produced in Spanish, an unpublished analysis has examined Spanish-speaking Hispanics compared to English-speaking Hispanics and has found a significant association with early withdrawal.

Strengths of the present study are that this was a naturally occurring comparison of one group who received an intervention and another that did not, without any misclassification of group assignment. All participants were eligible for the follow-up study and the outcomes of enrollment and retention are measurable, real outcomes rather than those of a hypothetical situation. Furthermore, the composition of the two comparison groups was similar and there was sufficient sample size to detect differences.

We hypothesized that the quality of the professionally executed video enhanced our ability to present the study protocol in a more accessible, visual and personalized format while explaining the study requirements in such a way that participants could make an informed decision to participate. Although we have demonstrated that those receiving the video benefited in this way, resulting in higher enrollment, we have no direct measures of the mechanisms of this effect. Participants receiving the video were not consistently asked whether they viewed the video and we had no measures of whether the understanding of the study protocol was better in those receiving the video compared to those who did not. Other factors that were potentially different between the two groups included staff turn-over and variation in administering the enrollment protocol over time. It was difficult to assess how these factors might have affected enrollment. However, the impact of these factors on the outcome is possibly minimized because all staff received standardized training in administration of the enrollment protocol that included standardized scripts, role playing and observation. Furthermore, the same experienced enrollment staff persons were involved over this two year time period and the number of new staff members added to this task was about the same in each year.

In longitudinal study designs, enrolling and retaining a representative cohort is essential to the generalizability of study findings. As forms of communication evolve and adapt to the available technological environment, methods of communicating essential information to study participants at enrollment and beyond must also evolve. Methods which aid in clearly describing study protocols to potential participants enable better informed decisions. A realistic and informed decision might enhance long term participation

Key factors for enhancing prospective study retention include developing and sustaining solid relationships between the participant and study staff. The strength of this relationship is based on communication. Communication strategies need to effectively reach participants through modes that are routine and familiar to a new generation of study participants, such as the use of video YouTube clips, websites, and various forms of social media that are used regularly by these participants. As a new generation of participants are being asked to participate in long-term research, such as in the TEDDY Study, it is important to think about how study designs can adapt using these innovations for the benefit of both participant and study validity.