Recruitment, Enrollment, and Response of Parent–Adolescent Dyads in the FLASHE Study

Abstract

Introduction

In 2014, the National Cancer Institute conducted the Family Life, Activity, Sun, Health, and Eating Study (FLASHE). This parent and adolescent survey examines psychosocial, generational (parent–adolescent), and environmental (home and neighborhood) correlates of cancer-preventive behaviors, with a particular emphasis on diet and physical activity. This paper describes the FLASHE data collection methods and enrollment and response rates.

Methods

FLASHE data collection methods included web-based surveys delivered to dyads of parents and their adolescent children, and deployment of accelerometers to a subset of adolescents, to achieve study goals in a nationwide study sample. The National Cancer Institute contracted with Westat, Inc. to recruit, enroll, and collect the data using a consumer opinion panel.

Results

A total of 5,027 dyads were screened for eligibility, and 1,945 (38.7%) enrolled. Of fully enrolled dyads, 85.6% of those in the Survey-Only group completed all four surveys, and 58.7% of dyads in the Motion Study group completed all surveys and were compliant with the accelerometer protocol for adolescents. The overall study response rate was 29.4%; 1,479 dyads completed all study procedures. The majority of parents were female, whereas the adolescent sample was gender balanced. Data were analyzed in 2015–2016.

Conclusions

FLASHE recruited a large sample of parent–adolescent dyads. Although challenges for research in parent–adolescent dyads include enrolling a diverse sample and having multistep enrollment and consent processes, study completion rate was high among fully enrolled dyads. Future panel studies may consider approaches used in FLASHE to encourage study enrollment and completion.

INTRODUCTION

Recruitment of parent–child dyads into scientific studies has been conducted in numerous ways, including intervention studies¹ and some household surveys.² However, few public-use resources exist that focus on health behaviors of parent–adolescent dyads and utilize a multilevel perspective.³ Collecting data on variables across the social context of parents and adolescents can be costly, as it is difficult to identify and screen eligible respondents and ensure compliance with study protocol. Web-based methodologies also tend to have lower recruitment, data collection, and data processing costs than random-digit dialing.^4,5 This issue combined with the increased penetration of the Internet has generated opportunities for web-based behavioral or health-related studies.

The National Cancer Institute (NCI) conducted the Family Life, Activity, Sun, Health, and Eating Study (FLASHE) in 2014. This parent–adolescent dyad survey examines psychosocial, generational, and environmental (home and neighborhood) correlates of cancer-preventive behaviors, with an emphasis on diet and physical activity. Although parenting, the environment, and psychosocial variables are correlates of adolescents’ health behaviors, national data sets have generally not included these variables in a survey that incorporates data from dyads.³ FLASHE used web-based methods to recruit dyads and obtain parents and adolescents’ self-reports on similar surveys.

An online consumer opinion panel was used to recruit eligible parent–adolescent dyads, and the surveys were administered via the web. In addition to the FLASHE surveys, a subset of dyads participated in a FLASHE motion study. In the motion study, adolescents wore an accelerometer for 7 days to provide an objective measure of physical activity and further validate their self-report physical activity. Given the complexity of the study design, including the “Survey-Only” protocol and the “Survey + Motion Study” protocol, the purpose of this paper is twofold: (1) to describe data collection methods, which included two web surveys and deployment of accelerometers in a nationwide sample; and (2) to describe enrollment and response rates, and implications for dyadic data collection using a web platform.

METHODS

The NCI contracted with Westat, Inc. to recruit, enroll, and collect the FLASHE study data.

Study Sample

The FLASHE dyads were recruited through the Ipsos Consumer Opinion Panel, which includes >700,000 active members. Some challenges of web-based samples are related to population coverage (i.e., non-representativeness of the general population and samples limited to Internet users).⁶ However, recruiting through Internet panels is cost efficient, provides a sample that has been screened and enrolled, and can facilitate recruitment owing to availability of large samples and information about panelists that can help recruit a specific sample.⁶ Ipsos selected a sample of their panelists for screening into FLASHE with the intention to match the U.S. population on key demographic characteristics. These panel members had indicated to Ipsos that they lived in a household with children. Using balancing techniques, the screened sample was selected to match the U.S. population on: sex of the panel member, Census division, household income, household size, and race/ethnicity.

These adult panel members were screened by Ipsos for FLASHE study eligibility via an online survey conducted in February and March 2014. A panel member was deemed eligible for FLASHE if he/she: was aged ≥18 years, was a parent or legal guardian of an eligible adolescent, and lived with the adolescent for ≥50% of the time. Eligible adolescents were: aged 12–17.5 years and lived with the panel member for ≥50% of the time. During the screening process, information on the eligible adolescents in the household was collected via a full household roster and one eligible adolescent was randomly selected until the quota for each age range (12–13, 14–15, 16–17 years) was full. Approximately one third of adolescent participants were recruited in each of the three age ranges, evenly split by gender.

Dyads invited to participate in FLASHE were randomized to different study procedures. Though all participants received two surveys (one focused on diet and the other on physical activity plus other cancer-preventive health behaviors), dyads were randomized to receive either the diet survey or the physical activity survey first. Furthermore, the FLASHE sample was randomized to participate in the Survey-Only group or the Survey + Motion Study group. Within the Survey + Motion Study group, half of the sample was randomly selected to receive a $20 incentive for participation and the other half was selected to receive a $40 incentive. Figure 1 shows the FLASHE flowchart.

Figure 1. FLASHE flowchart.

FLASHE, Family Life, Activity, Sun, Health, and Eating Study

Following randomization, dyads were invited to enroll into FLASHE in April 2014. The invitations were sent to parents’ and adolescents’ e-mail addresses and contained the URL for the study website and a personalized identification number. The study website required contact information, consent (parent consent and parent consent for adolescent), and assent (adolescent) to be fully enrolled. The invitations described the study procedures, incentives, and contacts. Consent/assent forms described data privacy and that participation was voluntary, and indicated that both dyad members were required to enroll. If assigned to the motion study, consent documents included language about wearing the accelerometer. Parents and adolescents could consent/assent for the survey portion of the study, but decline the motion study. The Westat, Inc. and NCI Special Studies IRBs approved the study protocol. Parents and adolescents that did not respond to the initial invitation were sent additional e-mails, letters, and a phone message asking them to participate. The enrollment period started on April 1, 2014 and for the Survey-Only group it remained open through the end of data collection on October 6, 2014. For the Survey + Motion Study group, the enrollment period was closed on July 14, 2014 so that the motion study activities could be completed within the field period.

Data Collection

For dyads enrolled into the Survey-Only group, study participation involved completion of four web surveys administered via the Vovici survey software: two by the parent and two by the adolescent. Both members of the dyad were invited by e-mail to complete their first survey and were required to complete it before the second set of surveys was made available. Automated e-mail reminders were sent to each participant with uncompleted surveys every 2 weeks for 6 weeks after the survey became available. Text message reminders were also sent on the same schedule for anyone who had permitted text reminders. The schedule for automated e-mails was 14, 28, and 42 days after enrollment was complete. In addition to the automated reminders, targeted reminders were sent to encourage response. The standard incentive for each completed survey was $5, which was paid in cash via mail with a “thank-you” letter addressed to the participant. To encourage completions, “bonus” incentives were offered twice during the field period. During the weeklong “bonus” period, respondents were paid $10 instead of $5.

Dyads enrolled in the Survey + Motion Study group completed the same four surveys as participants in the Survey-Only group with the addition of the adolescent wearing an accelerometer for 7 days prior to the physical activity survey. Data collection occurred between May and August 2014 with staggered data collection dates to allow for recycling of devices through the sample. Prior to the assigned wear week, each adolescent in this group was mailed an Actigraph GT3X+ accelerometer and a wear log. Accelerometers were configured to collect raw tri-axial accelerometry data at 80 Hz for the duration of the programmed data collection period. Adolescents were instructed to wear the monitors on their dominant wrist starting at 8:00PM on the Sunday of their wear week until 8:00PM the following Sunday, including while showering. The wear log was included so that each participant could record information such as the hours they were sleeping and any times that they chose to remove the accelerometer. Respondents were sent three e-mail reminders about wearing the accelerometer and returning the wear log: prior to the wear week, during the wear week, and after the wear week was completed. If needed, an e-mail was sent to parents at the end of the reminder period to let them know their adolescent still had not returned the accelerometer. Participants received the same $5 survey incentives as those in the Survey-Only group. They also received “bonus” e-mail invitations to complete surveys quickly for $10. Because there is no literature to indicate an appropriate incentive rate to return a mailed accelerometer, FLASHE included a sub-study in which half of all adolescents were compensated $20 for returning the accelerometer and the other half were compensated $40.

Calculating Completion Rates

The FLASHE study completion was measured at the dyad level for the Survey-Only and the Survey + Motion Study groups. For the Survey-Only group, a dyad was coded as “complete” if all four surveys were completed. For the Survey + Motion Study group, a dyad was coded as “complete” if all four surveys were completed and the accelerometer was worn. “Worn” was defined as having at least 1 day with ≥18 hours of wear time. This definition was determined specifically for FLASHE. As there are no established criteria for a valid “day” in wrist-worn accelerometry protocols, this definition was used to categorize participants as completing the FLASHE motion study. To determine the overall study response rate by dyad, both the enrollment rate and the survey completion rate were taken into account.

Given that inferences based on such a convenience sample such as FLASHE could be largely different from the general population, “analysis weights” were created by benchmarking the weights to the target population on key demographics. This is a process whereby weights are adjusted based on a set of variables to be more similar to the distributions of the general population, and inference bias related to frame non-coverage, sample selection, and non-response is expected to be reduced.⁶ Although weights were created, caution should be taken in the interpretation of the weighted estimates and any statistical tests as results are subject to sampling bias when generalizing to the general U.S. population. More information about calculation and use of analysis weights is available in the data users’ guide on the FLASHE study webpage (http://cancercontrol.cancer.gov/brp/hbrb/flashe.html).

Data were analyzed in 2015–2016 to calculate enrollment and completion rates and to examine participants’ demographic characteristics.

RESULTS

Following screening and de-duplication of e-mail addresses, Ipsos provided the contact information for 5,027 adult–adolescent dyads that had met the screening criteria and were invited to enroll. Of these dyads, 1,690 (33.6%) were invited to participate in the Survey + Motion Study group, and the remaining 3,337 dyads were randomized to be invited to the Survey-Only group. The distribution by demographic characteristics of this sample, compared to the American Community Survey statistics, can be found in the FLASHE methodology report on the FLASHE study webpage. Although the screened sample included a higher proportion of male versus female participants, the sample received from Ipsos was similar to the American Community Survey distribution on other demographic characteristics, including age and Census division.

Dyads could enroll at any point during the data collection period, although 97% of enrollments were completed within the first 2 months of the study. A total of 1,252 dyads (37.5%) enrolled in the Survey-Only group and 693 dyads (41.0%) enrolled in the Survey + Motion Study group (Table 1). This represents a 38.7% enrollment rate overall.

Table 1.

Enrollment Totals for FLASHE Study

Enrollment status	Survey-Only (n=3,337)	Survey + Motion (n=1,690)	Total (n=5,027)
Dyad fully enrolleda	1,252 (37.5%)	693 (41.0%)	1,945 (38.7%)
Dyad partially enrolledb	688 (20.6%)	0 (0.0%)	688 (13.7%)
Dyad refused to enroll	91 (2.7%)	115 (6.8%)	206 (4.1%)
No response	1,306 (39.1%)	882 (52.2%)	2,188 (43.5%)

^aFull enrollment was defined as receipt of parent consent for themselves, their adolescent and adolescent assent was received.

^bParent consented to the study and for their adolescent, but their adolescent did not assent to the study.

FLASHE, Family Life, Activity, Sun, Health, and Eating Study

Table 2 shows that 85.6% of fully enrolled dyads in the Survey-Only group completed all four surveys (n=1,072). Only 2.8% (n=35) of enrolled dyads did not complete any surveys and 11.6% (n=145) of dyads completed between one and three surveys. A total of 58.7% (n=407) dyads were fully compliant with the Survey + Motion Study protocol—having returned all surveys and worn the meter as instructed. Only 15.9% (n=102) of dyads that returned the accelerometers completed between one and three surveys and wore their accelerometers, and 2.9% (n=37) of dyads enrolled in the Motion Study group did not complete any surveys or return accelerometers. All dyads that returned an accelerometer (either worn or not worn) completed at least one survey.

Table 2.

FLASHE Completion Rates of Fully Enrolled^a Participants

Response rate	Survey-Only (n=1,252)	Survey + Motion (n=693)	Total (n=1,945)
Parent	1,068 (85.3%)	440 (63.5%)	1,708 (87.8%)
Adolescent	1,018 (81.3%)	350 (50.5%)	1,590 (81.7%)
Dyad	1,072 (85.6%)	407 (58.7%)	1,479 (76.0%)

^aBased on participants who fully completed the study meaning completing all four surveys and wearing the accelerometer, if applicable.

FLASHE, Family Life, Activity, Sun, Health, and Eating Study

The overall study response rate was 29.4% (Figure 1). The Survey-Only response rate was 32.1%. The Survey + Motion Study response rate (completion of two surveys for parents and adolescent and accelerometer wear) was 24.1%. The response rate for the parent surveys was 34% and the response rate for the adolescent surveys was 31.6%.

A total of 1,699 parents and 1,581 adolescents completed both the diet and physical activity surveys (Table 3). A total of 1,573 dyads completed all four surveys. There were more female than male parent respondents but similar numbers of male and female adolescents. The majority of parents were aged 35–59 years, and approximately one third of adolescents were in each of the age ranges (12–13, 14–15, and 16–17 years). The majority of participants were non-Hispanic white, and approximately half of parents had a 4-year college degree or higher. At least one dyad was from each state, except Alaska. Of caregivers, 90.0% were the parent of the adolescent and 4.2% were the step parent of the adolescent. A total of 2.4% of caregivers had other relationships with the adolescent.

Table 3.

Sample Sizes and Demographic Frequencies Across FLASHE Surveys^a

Demographics	Diet survey				Physical activity survey			Both surveys
	N	Unweighted %	Weighted %	N	Unweighted %	Weighted %	N	Unweighted %	Weighted %
Parent	1,745			1,793			1,699
Sex
Male	454	26.0	43.6	461	25.7	42.4	447	26.3	43.6
Female	1,280	73.4	55.9	1,295	72.2	55.2	1,250	73.6	56.2
Not ascertained	11	0.6	0.6	37	2.1	2.4	2	0.1	0.2
Age
18–34	196	11.2	10.0	195	10.9	9.6	189	11.1	10.0
35–44	754	43.2	42.6	765	42.7	42.2	738	43.4	42.8
45–59	733	42.0	43.5	746	41.6	42.6	721	42.7	43.7
60 +	51	2.9	3.3	50	2.8	3.2	49	2.9	3.3
Not ascertained	11	0.6	0.5	37	2.1	2.3	2	0.1	0.2
Race/Ethnicity
Hispanic	126	7.2	16.0	126	7.0	15.4	122	7.2	15.9
Non-Hispanic black alone	293	16.8	11.6	301	16.8	11.3	280	16.5	11.6
Non-Hispanic white alone	1,200	68.8	59.3	1,210	67.5	58.3	1,181	69.5	59.5
Other	100	5.7	11.9	104	5.8	11.8	99	5.8	12.0
Not ascertained	26	1.5	1.3	52	2.9	3.1	17	1.0	1.0
Education
<High school	22	1.3	1.1	22	1.2	1.0	22	1.3	1.1
HS degree/GED	292	16.7	15.3	294	16.4	14.8	285	16.8	15.3
Some college	608	34.8	30.5	623	34.8	30.1	597	35.1	30.8
College degree	806	46.2	52.2	812	45.3	51.4	788	46.4	52.3
Not ascertained	17	1.0	0.9	42	2.3	2.6	7	0.4	0.5
Adolescent	1,657			1,661			1,581
Sex
Male	810	48.9	50.0	806	48.5	49.4	781	49.4	50.6
Female	823	49.7	48.6	814	49.0	48.0	794	50.2	49.1
Not ascertained	24	1.5	1.4	41	2.5	2.6	6	0.4	0.4
Age
12–13	545	32.9	32.1	545	32.8	31.7	530	32.5	32.3
14–15	564	34.0	32.6	561	33.8	31.9	540	34.2	32.8
16–17	528	31.9	34.2	518	31.2	34.0	509	32.2	34.8
Not ascertained	20	1.2	1.1	37	2.2	2.4	2	0.1	0.1
Race/Ethnicity
Hispanic	160	9.7	15.8	162	9.8	15.7	154	9.7	16.0
Non-Hispanic black alone	272	16.4	13.5	266	16.0	13.4	255	16.1	13.7
Non-Hispanic white alone	1,037	62.6	53.7	1,033	62.2	53.3	1,009	63.8	54.3
Other	152	9.2	14.7	148	8.9	14.1	146	9.2	14.9
Not ascertained	36	2.2	2.2	52	3.1	3.4	17	1.1	1.12
Dyad	1,646			1,644			1,573

^aExcept for the last row of this table, these frequencies refer to the number of individual respondents who have data on the diet survey, the physical activity survey, and both surveys, regardless of whether their corresponding dyad member completed the survey. The final row indicates the number of dyads in which both the parent and the adolescent completed each survey.

FLASHE, Family Life, Activity, Sun, Health, and Eating Study; HS, High School; GED, General Education Development Test

DISCUSSION

The FLASHE study entailed use of a commercial consumer opinion panel to find an eligible study sample, use of a web-based survey, and a mail (versus in-person) delivery and receipt of accelerometers, which are relatively new approaches to collecting parent–adolescent dyadic data. Collection of data in dyads presented unique challenges. The inclusion of adolescent children created a more complex enrollment process because IRB requirements mandated active consent. Because the panel consisted of adults, not adolescents, a multistep enrollment and consent process was required for the dyad to receive the surveys. Relying on parental report for their adolescent’s name and e-mail address also presented challenges. The selected adolescent’s birth month and year were included in communications to enable participants to easily identify the correct adolescent. Some parents appeared reluctant to share their adolescent’s name during the enrollment; however, parents who wished to change their adolescent’s name to the correct name were able to do this by contacting Westat, Inc. Names and e-mail addresses were also screened and corrected for invalid names or types (e.g., .cm instead of .com).

The FLASHE response rates can be compared to response rates from other panel and web surveys. Response rates from other panel surveys in the U.S. are variable with recent studies reporting response rates between 45% and 70%.^7–11 Among households that received both the Centers for Disease Control and Prevention’s HealthStyles survey (for adults) and YouthStyles survey (for children), 67% completed both the parent and child surveys.¹² Although this rate is higher than the overall enrollment rate (38.7%) and response rate (29.4%) in FLASHE, the rate of study completion among the fully enrolled dyads was high, at greater than 85%. Though a primary challenge in FLASHE was enrolling dyads, most dyads complied with the study protocol once enrolled. The overall response rates for parents and adolescents responding to both FLASHE surveys (incorporating both enrollment and completion rates; 34% and 31.6%, respectively) are comparable to a meta-analysis finding that the average response rate for web surveys is 34%.¹³

Users of FLASHE data should avoid generalizing to the U.S. population. The goal of FLASHE was to examine relationships among health behavior correlates, rather than a surveillance study for which a nationally representative sample would be needed. That said, the intent was to create a sample that was similar to the U.S. population and that oversampled fathers and African Americans. However, the consumer panels did not have enough male or African American parents who had an adolescent child within the specific age range for FLASHE. Broader inclusion criteria may allow for easier recruitment of diverse samples. Parent participants in FLASHE were predominantly female, which is similar to adult samples in some panel studies, such as NCI’s Food Attitudes and Behaviors Survey¹⁴ but not all.⁹ In addition, FLASHE dyads were predominantly white, similar to households participating in the Centers for Disease Control and Prevention’s Styles panel surveys.¹² Although oversampling goals were not fully attained, the proportion of non-Hispanic black participants in FLASHE was similar to the proportion of non-Hispanic black adolescents in the U.S.¹⁵ The FLASHE sample was also predominantly of high SES, which is typical of Web surveys.¹⁶ Though the sample of panelists screened for FLASHE was balanced to U.S. population demographics, further differences between the FLASHE sample and the U.S. population were due to non-response in enrollment or study completion.

Limitations

The FLASHE motion study also had significant advantages and challenges. The inclusion of the motion study allowed for a validation study of the adolescent self-reported physical activity measure in FLASHE (the Youth Activity Profile), as detailed in this issue.¹⁷ Second, FLASHE utilized wrist-worn activity monitors to allow for better estimation of intensity-specific physical activity. However, processing these data has been challenging given the field of physical activity epidemiology is just starting to develop the appropriate metrics for wrist-worn accelerometry data processing.¹⁸ Third, the protocol tested the feasibility of collecting accelerometer data in a web-based, consumer panel framework where study administrators had no direct contact with participants and a limited number of monitors to utilize. It was unclear if participants would: (1) decline participation because of the additional requirements of the motion study protocol; (2) adhere to all aspects of the motion study protocol; and (3) return the monitor in a timely fashion. For dyads in the motion study group, the enrollment rate was actually higher than the survey only group, although the survey completion rate was lower. The majority of adolescents returned the monitors (92%) and the associated wear log (88.0%) suggesting that it is feasible to employ an accelerometer protocol remotely.

CONCLUSIONS

Using a web panel to collect survey data for FLASHE achieved study goals, but there remain challenges to enrolling a diverse sample of respondents and having the respondents complete the survey given multiple enrollment tasks. Studies considering use of web panels to collect online survey data may consider some of the approaches taken in FLASHE to encourage enrollment and survey completion.