Web-Based Recruitment and Survey Methodology to Maximize Response Rates from Followers of Popular Diets: the Adhering to Dietary Approaches for Personal Taste (ADAPT) Feasibility Survey

Abstract

Background

Although there is interest in popular diets such as vegan and vegetarian, Paleo, and other “whole food” diets, existing cohort studies lack data for these subgroups. The use of electronic data capture and Web-based surveys in nutrition research may be valuable for future studies by allowing targeting of specific dietary subgroups.

Objective

The aim was to perform a Feasibility Survey (FS) to assess the practicality of Web-based research methods to gather data and to maximize response rates among followers of popular diets.

Methods

The FS was an open, voluntary, 15-min survey conducted over 8 wk in the summer of 2015. Recruitment targeted self-identified followers of popular diets from a convenience sample, offering no incentives, via social media and e-newsletters shared by recruitment partners. Feasibility was assessed by number of responses, survey completion rate, distribution of diets, geographic location, and willingness to participate in future research.

Results

A total of 14,003 surveys were initiated; 13,787 individuals consented, and 9726 completed the survey (71% of consented). The numbers of unique visitors to the questionnaire site, view rate, and participation rate were not captured. Among respondents with complete demographic data, 83% were female and 93% were white. Diet designations were collapsed into the following groups: whole-food, plant-based (25%); vegan and raw vegan (19%); Paleo (14%); try to eat healthy (11%); vegetarian and pescatarian (9%); whole food (8%); Weston A Price (5%); and low-carbohydrate (low-carb) (4%). Forced-response, multiple-choice questions produced the highest response rates (0–2% selected “prefer not to answer”). The percentage who were willing to complete future online questionnaires was 86%, diet recall was 93%, and food diary was 75%; the percentages willing to provide a finger-stick blood sample, venipuncture blood sample, urine sample, and stool sample were 60%, 44%, 58%, and 42%, respectively.

Conclusions

This survey suggests that recruiting followers of popular diets is feasible with the use of Web-based methods. The unbalanced sample with respect to sex and race/ethnicity could be corrected with specific recruitment strategies using targeted online marketing techniques.

Introduction

Public interest in nutrition is at the forefront of health, and there appears to be growing interest in popular diets such as vegan and vegetarian (1), Paleo (2), and other “whole food” diets.

Online searches for popular diet terms have grown in the past decade, as measured by Google Trends (3), and there are a growing number of supermarket and restaurant options tailored to particular diets (4, 5). Benefits for weight loss and disease prevention have been shown for plant-based diets (6, 7), low-carbohydrate (low-carb) or Paleo diets (8, 9), and other omnivorous patterns that emphasize unrefined foods (10). Studying such individuals is of great public health interest in light of the risk of overweight associated with typical US dietary intake patterns (11). Despite the potential value of studying popular-diet followers, reaching sufficient numbers to participate in research studies is challenging. Limited nationally representative data as well as certain consumer surveys indicate a low prevalence (<4% at most) of specific popular diets such as vegetarian and vegan in the United States (12–14). Web-based recruitment and data-capture methods may be useful to narrowly target specific populations for research.

Since the beginning of the 21st century, there has been a cultural shift toward greater use of the Internet in a health care context (15–18), setting the stage for increasing use of electronic data capture in scientific research (19–21), as well as greater access to research study participants through recruitment with Web-based methods. Because consumers are now accustomed to using the Internet for health information, Web-based surveys are becoming more compatible with current consumer preferences for online interaction (16).

Web-based research is conducted online (distinct from research simply using electronic data-capture tools), recruiting an initially anonymous base of Internet users to participate through a variety of online advertising strategies. Targeted advertising through paid ads such as Google ads, social media ads (i.e., Facebook or Twitter), or simple sharing and posting by individuals and organizations with large social media followings allows researchers to reach large convenience samples with study invitations. This approach allows for the targeted recruitment of individuals who would otherwise be underrepresented due to their niche interests or small numbers (22).

As with other research study designs, guidelines have been established to ensure best practices and to enhance the quality of survey methodology and reporting by researchers (23–25). For Web-based surveys specifically, the Checklist for Reporting Results of Internet E-Surveys (CHERRIES) protocol, first proposed in 2004, was intended to be analogous to the Consolidated Standards of Reporting Trials (CONSORT) statement for clinical trials or the Quality of Reporting of Meta-Analyses (QUORUM) statement for systematic reviews (26) and has been used in a variety of Web-based studies (18, 27, 28).

To date, the application of online recruitment and data collection in nutrition surveys is an emerging practice (29). Although Web-based surveys offer advantages of targeting specific groups, ease and convenience in administration, and lower cost, they typically pose challenges for achieving response rates as high as those for in-person surveys (30). We describe the pilot testing of Web-based recruitment and data collection in the Adhering to Dietary Approaches for Personal Taste (ADAPT) Feasibility Survey (FS), while adhering to the accepted reporting guidelines of CHERRIES for online surveys. The objectives of this survey were as follows: 1) to show the feasibility of recruiting individuals who self-identified as adhering to popular diets with the use of Web-based recruitment; 2) to evaluate Web-based data capture of demographic characteristics, lifestyle, and health history to maximize response rates; and 3) to assess study participants’ willingness to participate in more extensive online nutrition and health surveys in the future. This survey provided preliminary data showing the effectiveness of Web-based recruitment and data-capture methods for self-identified followers of popular diets.

Methods

The ADAPT FS was a Web-based, volunteer, nonprobability (or opt-in) survey designed to capture interest from individuals who identified as following a popular diet in ongoing Web-based diet and lifestyle research. The overall purpose of this phase of the study was to show feasibility in recruiting study participants via online recruitment strategies. The CHERRIES (Supplemental Table 1) was used as a guide to our survey design and administration (26). The survey was open for 8 wk, from 14 July 2015 through 14 September 2015, with an active recruitment period of 6 wk, from 14 July 2015 to 31 August 2015. Although the survey was hosted online, the research was conducted from the USDA Human Nutrition Research Center on Aging at Tufts University located in Boston, Massachusetts.

Self-identified diet assessment

We created a list of popular diets based on our knowledge of the field and on recent Google Trends data to track online search activity patterns with respect to popular diets. Our goal was to capture diets that represented a broad range of dietary intake; participants self-identified their current diet by responding to the following question:

“Imagine that you were chatting casually about your diet with someone you met in an elevator. Would you use any of the following terms to describe what you typically eat? It's okay if your diet doesn't 100% match what these diets are supposed to be. Please choose the one that best matches what you would say, or ‘Other diet’ if you follow a diet that is not listed here (you can give your own description in the space provided).”

Major self-identified diets of interest included vegan, vegetarian, whole-food, plant-based (WFPB), whole-food omnivorous, Paleo, low-carb, Mediterranean, gluten-free, and Weston A Price (WAP). We also allowed study participants to self-identify as following “No particular diet” or “No particular diet, but I have tried to eat healthy,” thereby allowing for potential comparison groups, as well as choosing “The diet I follow is not listed here” with a write-in option. A full copy of all survey questions can be viewed on the study website at http://hnrca.tufts.edu/adapt/files/2017/05/ADAPT_Feasibility_2015.pdf.

Recruitment

Participants were recruited from a convenience sample, offering no incentives, via social media (Facebook and Twitter) and e-newsletters shared by recruitment partners. Recruitment partners were considered leaders/experts (e.g., book authors) in their respective diet communities and were identified on the basis of public visibility through searches for relevant websites, or for relevant Facebook pages or Twitter accounts with large followings (≥5000). During the period of active recruitment, 10 recruitment partners had >50,000 Facebook followers (with 4 of these having >200,000 Facebook followers), and 2 recruitment partners had >125,000 Twitter followers. No other online announcements or advertisements were used, except for potential resharing of the original social media posts.

Recruitment partners were introduced to the study via an e-mail from the principal investigator (PI) or, when possible, through personal introductions of the PI via colleagues in the field. The recruitment partners were informed of the 8-wk duration for recruitment and were asked to schedule newsletter e-mail announcements, Facebook or Twitter posts, or other electronic announcements once or more during the recruitment period. Sample e-mails and social media posts were provided to the recruitment partners (Supplemental Table 2); however, it was possible that the final text used by recruitment partners may have been altered before sharing.

Survey development

The survey was developed by using an iterative, collaborative process supervised by the PI, with input from the PI's research team, with the use of pre-existing surveys as models for relevant questions. Qualtrics online survey software (www.qualtrics.com), which allows flexible options for question creation and a user-friendly interface, was the platform used to create and administer the survey. Adaptive questioning and skip logic were used to limit unnecessary questions and to reduce study participant burden. Multiple rounds of pilot testing were performed with a variety of respondents from the Tufts University research community (n ≈ 25), spanning various age, sex, and diet-preference groups. Questions were refined on the basis of pilot testing feedback. The time to complete the pilot test survey was recorded, and questions were modified to ensure that study participants could complete the questionnaire in a reasonable length of time (≤15 min). This survey consisted of 29 core questions (21 forced-response and 8 optional-response). Forced-response questions required participants to select a response to move forward in the survey. For all forced-response questions, participants were given the option to select “prefer not to answer.” This was done to allow participants the option to intentionally not answer while aiming to reduce the number of inadvertently skipped questions. The completion of question 29 was used to define survey completion: “Would you like to be added to the research center mailing list?” There were 41 additional questions branching off the core questions where applicable.

Survey domains

The final version of the survey included questions capturing the following: 1) diet preferences, including current self-identified diet, time on current diet, past diet(s) followed and time on these diets, reasons for following current diet, sources of nutrition and cooking information; 2) demographic, lifestyle, and medical questions including US residency, zip code, age, sex, weight, height, supplement use, self-reported history of medical conditions, physical activity [using the International Physical Activity Questionnaire Short Form (31)]; and 3) willingness to participate in future nutrition research. The country in which the survey was taken, browser information, survey metrics such as time-to-finish (in 10-min increments), and start dates were captured by embedded data fields in Qualtrics.

Administration

The survey was administered via an anonymous link. After a brief description of the survey, research purpose, and potential participant risks, informed consent was obtained via 2 screening questions: Are you at least 18? and Do you agree to participate? [See CHERRIES reporting (Supplemental Table 1) for additional details on the informed consent process.] To assess interest in future studies, participants were asked to voluntarily provide an e-mail address twice for verification. The survey did not screen out respondents who had previously participated and thus multiple responses from the same participants were unavoidable. However, separate, completed questionnaires with the same e-mail addresses were identified during data cleaning, and for these duplicates, only the most recent survey response was retained.

Due to a high number of participants abandoning the survey at the physical activity questions, the survey was modified 2 wk into the recruitment period by changing the physical activity questions from forced-response to optional. One additional modification made to the physical activity questions was to have more screen breaks, so fewer questions appeared on the screen at once, allowing participants to more easily see which questions were unanswered.

It was not possible to capture data on unique site visitors, unique survey visitors, view rate, and participation rate as defined by the CHERRIES protocol. For reference, “unique site visitors” are defined as the number of people who visit a webpage or social media posting and view the study invitation advertisement, “unique survey visitors” are defined as the number of people who visit the first page of a survey (whether or not they answered any questions), and “view rate” is defined as the ratio of unique survey visitors to unique site visitors. “Participation rate” is defined as the ratio of those who agreed to participate (completed the consent questions) to unique survey page visitors. Per the CHERRIES protocol, we were able to capture completion rate. The completion rate is defined as the ratio of those who completed the survey to those who agreed to participate. (See Supplemental Table 1 for the full reporting as per the CHERRIES protocol.) Response rates for individual questions were tracked by comparing the proportion who selected an answer with those who selected “prefer not to answer” or who skipped the question.

Participants were able to pause and return to their survey within 1 wk of beginning the survey. Qualtrics automatically saved any progress and returned a respondent at the same IP address to their last completed question. Surveys that were unfinished after 1 wk were automatically transferred from “in progress” to “finished,” despite the fact that these surveys were unfinished by participants.

This study was reviewed by the Tufts Medical Center/Tufts University Health Sciences Institutional Review Board.

Data cleaning

All of the data were stored on a secure, password-protected server and were de-identified before analysis by removing e-mail addresses (voluntarily provided for future contacting). Participants who voluntarily provided an e-mail address were asked to enter their e-mail twice to verify accuracy, and only the project manager viewed e-mail addresses for the purpose of adjudicating typos and removing duplicate entries. Cases in which the second e-mail provided did not match the first were resolved on a case-by-case basis, and apparent typos were corrected (i.e., @gmail was corrected to @gmail.com, etc.). Cases in which 2 mismatching e-mails were provided and could not be resolved were retained for analysis of survey completers; however, these participants will not be able to be reached for future research.

Diet groups were collapsed, on the basis of frequency, into the following categories: WFPB, vegan and raw vegan, Paleo, try to eat healthy (also including those who selected “low-fat”), vegetarian and pescatarian, whole food (also including those who selected Mediterranean and locavore), WAP, and low-carb. A miscellaneous category was created to house the remaining participants who identified as following a variety of different diets (e.g., medical/avoidance, weight loss) or “no particular diet.” Write-in responses for “other diet” and “doctor/practitioner-recommended diets” were hand-recoded into the previously identified diet categories or, if not possible, the miscellaneous category.

Analysis

The effectiveness of Web-based recruitment and survey administration was determined by survey metrics including completion rate, survey start dates, and time to complete. Descriptive statistics were generated to assess the reach of the Web-based recruitment strategies in attracting followers of a variety of targeted diets from a variety of geographic regions. Feasibility to conduct future research on these populations was also assessed by using the questions on willingness to participate in future research. Descriptive statistics were generated by using SAS 9.4 (SAS Institute).

Results

A total of 14,003 surveys were initiated during the recruitment period. After removing duplicate responses (n = 83 provided a duplicate e-mail address) and excluding those who did not consent (n = 133), a total of 13,787 participants with unique e-mail addresses agreed to participate in the ADAPT FS.

During the 2-mo active recruitment period, participation was closely aligned with the intended release of social media material by our larger recruitment partners. A smaller response was observed by participants during the periods between these larger, scheduled announcements, presumably through resharing of the survey invitation. Response rates for the survey are shown in Figure 1.

FIGURE 1.

Surveys started by date during the recruitment period.

A total of 13,787 participants consented to participate in the survey. The overall completion rate was 71%, with a total of 9726 out of 13,787 participants completing the survey. Question types, response rates including “prefer not to answer” responses, and skipped questions are shown in Table 1. Among participants who completed the study, the number who selected “prefer not to answer” for forced-response core questions was low, ranging from 0% to 2%. The exception was the forced-response question asking participants to write in sources of nutrition and cooking information, for which 10% of respondents selected “prefer not to answer.” Among optional core questions without the “prefer not to answer” option, the numbers of consented participants who skipped the questions were more varied, with 1% for height, 3% for weight, 11% for reason for current diet, and between 9% and 15% for the 3 physical activity questions that mentioned participants could skip the question. There was 1 optional, multiple-choice, core question that offered “prefer not to answer” without specifically mentioning that participants could skip the question. This question, which asked about hours per day spent sitting, had only 1% (n = 141) “prefer not to answer” responses and only n = 24 skips.

TABLE 1.

Question types, response rates, “prefer not to answer” responses, and skipped optional questions

Question	Question type	Forced response vs. optional	“Prefer not to answer” (out of n = 9726 completed), n (%)	Skipped in optional questions (out of n = 9726 total consented), n (%)
Your gender	Multiple choice	Forced	19 (0)	N/A
Your age bracket	Multiple choice	Forced	7 (0)	N/A
Your height in feet and inches. (For example, if you are 6΄2”, enter “6” in the field for feet and “2” in the field for inches. Please round to the nearest inch.) If you prefer not to answer, you can skip this question.	Write-in	Optional	N/A	69 (1)
Your weight in lbs (please round to the nearest lb). If you prefer not to answer, you can skip this question.	Write-in	Optional	N/A	312 (3)
Are you of Hispanic, Latino, or Spanish origin?	Multiple choice	Forced	57 (1)	N/A
What is your race?	Multiple choice	Forced	109 (1)	N/A
Do you live in a US state, US territory, or US military base? If you do, please enter your zip code.	Multiple choice	Forced	101 (1)	N/A
Imagine that you were chatting casually about your diet with someone you met in an elevator. Would you use any of the following terms to describe what you typically eat? It's okay if your diet doesn't 100% match what these diets are “supposed to be.” Please choose the one that best matches what you would say or “Other diet” if you follow a diet that is not listed here (you can give your own description in the space provided).	Multiple choice	Forced	4 (0%)	N/A
Please briefly describe your reasons for eating the way you eat currently. If you prefer not to answer, you can skip this question.	Write-in	Optional	N/A	1052 (11)
Do environmental concerns have any influence on your dietary choices?	Multiple choice	Forced	109 (1)	N/A
We would like to know about your history of chronic disease. If you prefer not to answer, you can skip this question. Please select all that apply.	Matrix	Optional	N/A	72 (1)
Did you follow any other diets in the past, prior to eating the way you currently eat? Please select all that apply.	Multiple choice	Forced	18 (0)	N/A
What are your main sources of information on nutrition and cooking for the diet you currently eat? Please select and list all that apply.	Multiple choice	Forced	956 (10)	N/A
Have you taken supplements in the last year, such as vitamin supplements, protein supplements, fiber-type supplements, or other supplements?	Multiple choice	Forced	10 (0)	N/A
During the last 7 days, on how many days did you do vigorous physical activities like heavy lifting, digging, aerobics, or fast bicycling? If you prefer not to answer, you can skip this question.	Multiple choice	Optional	N/A	907 (9)
During the last 7 days, on how many days did you do moderate physical activities like carrying light loads, bicycling at a regular pace, or doubles tennis? Do not include walking. If you prefer not to answer, you can skip this question.	Multiple choice	Optional	N/A	1460 (15)
Think about the time you spent walking in the last 7 days. This includes at work and at home, walking to travel from place to place, and any other walking that you have done solely for recreation, sport, exercise, or leisure. If you prefer not to answer, you can skip this question.	Multiple choice	Optional	N/A	1831 (19)
During the last 7 days, how much time did you spend sitting on one of those days?	Multiple choice	Optional	141 (1)	24 (0)
Complete online questionnaires about your diet, exercise, lifestyle, and health status?	Multiple choice	Forced	57 (1)	N/A
Complete an online questionnaire (called a “diet recall”) recalling the specific foods you have eaten over the past 24 hours?	Multiple choice	Forced	138 (1)	N/A
Complete a food diary, in which you write down all the foods and portion sizes you consume over the course of the day?	Multiple choice	Forced	227 (2)	N/A
Repeat the same questionnaires about your diet, exercise, lifestyle, and health status every 2–3 years?	Multiple choice	Forced	68 (1)	N/A
Provide a finger-stick blood sample using a kit that we mail to you and you mail back to us?	Multiple choice	Forced	68 (1)	N/A
Provide a blood sample from a full (venous) blood draw, similar to the type of blood draw you would have at your doctor's office?	Multiple choice	Forced	80 (1)	N/A
Provide a urine sample using a kit that we mail to you and you mail back to us?	Multiple choice	Forced	78 (1)	N/A
Provide a stool sample using a kit that we mail to you and you mail back to us?	Multiple choice	Forced	95 (1)	N/A
Come to the Boston area for laboratory measurements (e.g., blood pressure, weight, blood sample)?	Multiple choice	Forced	85 (1)	N/A
If you meet eligibility criteria, may we contact you when we are ready to begin recruitment for the larger online study to invite you to participate? If you answer “yes,” please enter your e-mail address below. We will keep your e-mail address on file for recruitment purpose only. We will not share your e-mail with anyone else.	Multiple choice	Forced	249 (3)	N/A
Would you like to be added to the mailing list for the Jean Mayer USDA Human Nutrition Research Center on Aging (HNRCA) at Tufts University to hear of nutrition news, research, and upcoming events at the center?	Multiple choice	Forced	194 (2)	N/A

The survey platform reported time-to-finish for all participants who began the survey (n = 14,003) in 10-min increments. The subsample of participants who completed the survey (n = 9726) could not be analyzed separately. Time-to-finish varied from <10 min to >30 min: <10 min (37%), 10–20 min (41%), 20–30 min (12%), and >30 min (10%). Only 237 respondents (2.4% of completers) took >4 h to complete the survey.

For the descriptive analysis, we included only those participants (n = 9536) who provided a response for current diet, sex, age, race, and ethnicity (i.e., did not select “prefer not to answer”). Of these, 83% were female, 93% were white, and 84% took the survey within the United States. The age distributions were 18–34 y (22%), 35–54 y (46%), and ≥55 y (33%). The distributions across race categories, age brackets, and regions were similar among men and women (within 3% for race, 5% for age, and 2% for region except for non-US country, which had 22% of men and 15% of women).

The diet distribution is shown in Table 2. The diet designations were collapsed into the following groups on the basis of frequency: WFPB (25%), vegan and raw vegan (19%), Paleo (14%), try to eat healthy (11%), vegetarian and pescatarian (9%), whole food (8%), WAP (5%), and low-carb (4%). Among US participants (n = 6714), the distributions of diet followers were similar to within 1% of the distribution among all participants. With respect to the physical activity questions, only 37% (n = 3483 of the 9536 who provided full demographic and diet data) responded to these questions in full.

TABLE 2.

Distribution of self-identified current diet

Diet group	Diet followers overall, n	Percentage of sample	Diet followers in United States, n	Percentage of sample in the United States	Percentage of US population per 10,000,000
Total	9536	—	6714	—	—
Whole-food, plant-based	2344	25	1690	25	52
Vegan and raw vegan	1763	18	1140	17	35
Paleo	1326	14	948	14	29
Try to eat healthy	1048	11	759	11	23
Vegetarian and pescatarian	883	9	628	9	19
Whole food	754	8	534	8	17
Miscellaneous	517	5	375	6	12
Weston A Price	493	5	400	6	12
Low-carb1	408	4	240	4	7

¹Low-carb, low-carbohydrate.

There were a total of 6714 participants who answered “yes” to the question “Do you live in a US state, territory or US military base?” and also provided a zip code. The total regional distribution of these 6714 out of the total 9536 was as follows: Northeast (14%), Midwest (14%), South (19%), West (22%), and Pacific region from Hawaii or Alaska (1%). Of the 9536, 14% of respondents declined to provide their zip code, although they did answer “yes” to living in a US state, territory, or military base. The 16% who completed the survey outside of the United States represented 86 countries (plus the US Virgin Islands), with the greatest numbers from Canada (5.5%), the United Kingdom (2.5%), and Australia (2.2%). The completion rates of the survey by current self-identified diet were as follows: WFPB (84%), vegan and raw vegan (84%), Paleo (81%), try to eat healthy (78%), vegetarian and pescatarian (80%), whole food (82%), WAP (86%), and low-carb (79%).

Diet group frequencies differed by state as shown in Table 3. Response rates, based on the percentage of state population per 1,000,000, found Massachusetts, Oregon, and Colorado to have the highest response rates of 59%, 56%, and 44%, respectively. Most frequently reported diets overall were WFPB, vegan and raw vegan, and Paleo. In Massachusetts, “try to eat healthy” was the most frequently reported diet.

TABLE 3.

Top 10 states with highest frequency of respondents and top reported diets within each state¹

	Total, n	Percentage of sample	Percentage of state population2 per 1,000,000	No. 1 reported diet (%)	No. 2 reported diet (%)	No. 3 reported diet (%)
US state respondents	6714	—	—	WFPB (25)	Vegan and RV (17)	Paleo (14)
California	948	14	24	WFPB (24)	Vegan and RV (21)	Paleo (15)
Massachusetts	402	6	59	Try to eat healthy (23)	WFPB (16)	Vegan and RV (13)
Texas	389	6	14	WFPB (24)	Paleo (20)	Vegan and RV (15)
New York	354	5	18	WFPB (27)	Vegan and RV (17)	Paleo (13)
Florida	338	5	17	WFPB (30)	Vegan and RV (17)	Vegetarian and Pescatarian (14)
Washington	277	4	39	WFPB (26)	Vegan and RV (18)	Paleo (16)
Illinois	254	4	20	WFPB (24)	Second Paleo (15)	Second Vegan and RV (15)
Colorado	242	4	44	WFPB (23)	Paleo (17)	Vegan and RV (14)
Ohio	242	4	21	WFPB (29)	Vegan and RV (23)	Paleo (12)
Oregon	226	3	56	WFPB (25)	Second Paleo (15)	Second Vegan and RV (15)
Remaining states	3042	45	—	WFPB (26)	Vegan and RV (16)	Paleo (14)

¹Restricted to respondents who answered “yes” to US residency and provided a zip code (n = 2822 missing due to not answering yes for living in the US or not providing a zip code from which the state could be calculated). Includes Puerto Rico, Guam, and the US Virgin Islands. RV, raw vegan; WFPB, whole-food, plant-based.

²Using 2015 data from the US Census Bureau, Population Division. Annual estimates of the resident population for the United States, Regions, States, and Puerto Rico: April 1, 2010 to July 1, 2016 (NST-EST2015-01); release date: December 2016.

The willingness to participate in future research was determined by a variety of questions relating to future online questionnaires or providing biological samples (Supplemental Tables 3–6). Overall willingness to participate in future research and complete online questionnaires was 86% and was similar across diet patterns, ranging from 81% (try to eat healthy) to 88% (WFPB). Of those who answered “yes” to completing online questionnaires, the overall willingness to repeat them every 2–3 y was 85% and ranged from 83% (try to eat healthy and WAP) to 88% (WFPB). The overall willingness to complete a diet recall was 93%, and of those, 72% were willing to complete 6 recalls in 1 y. The overall willingness to complete a food diary was 75%, and of those, 58% were willing to complete two 7-d diaries in 1 y, and 21% were willing to complete two 3-d diaries in 1 y. The preferred format for completing a food diary was typing into a website (53%) followed by using an app on a smartphone or tablet (32%). The overall willingness to provide a finger-stick blood sample was 60% and ranged from 55% (try to eat healthy and vegetarian) to 66% (WFPB). The overall willingness to provide a venipuncture blood sample was 44%, to provide a urine sample was 58%, and to provide a stool sample was 42%. Participants from within states neighboring Massachusetts had an overall willingness of 49% to come to Boston for testing. The overall consent to be contacted in the future was 87%.

Discussion

The ADAPT FS explored the challenges and possibilities of building a Web-based study designed to capture a variety of popular diet followers. Web-based recruitment and electronic data capture have been emerging as practical and cost-effective strategies in research (19). US-based cohort studies that historically collected data by using paper-based surveys mailed to participants (20, 21) have begun transitioning from paper-based to electronic data capture using computer-based versions of their surveys (32, 33). In terms of feasibility for future research, a high proportion (87%) of respondents indicated that they would be willing to participate in a more demanding research study that would require future data collection, including providing biological samples. This willingness ranged from 42% to provide a stool sample to 93% to complete a diet recall. More comprehensive data collection is necessary to characterize the nutrient intake of popular diet followers. Previous work has found discrepancies between the self-identified labels that many diet followers choose to use and the food choices they actually make, such as with meat (34) or gluten (35). Our question capturing respondents’ self-identified diet intentionally did not define any of the dietary patterns so as to avoid social priming that might introduce response bias, because it is our eventual goal to compare self-identified diet followed with actual dietary intake. On the basis of the response to this survey, we are now in the next phase of implementing a larger survey (ADAPT–Pilot) that administers a variety of questionnaires to capture demographic characteristics, medical history, supplement use, psychobiological factors such as social support and self-efficacy, food environment, and dietary intake.

The use of the CHERRIES protocol added methodological rigor to our Web-based survey, which is essential assuming the use of Internet-based nutrition research is likely to increase in the future. Adhering to this protocol can improve reporting of methods and results from online surveys. Even with the plethora of suggested guidelines for reporting survey methodologies, a 2010 systematic review of 165 leading journals reported that few high-impact journals provided author guidance or expectations for reporting survey methods and results (25). At the same time, peer-reviewed articles also often lack complete data, such as providing the survey instrument, defining the response rates, or addressing the representativeness of the survey sample (25).

The CHERRIES protocol was developed to provide guidance around the challenges in calculating response rates (26). Web-based research methods are subject to additional limitations, compared with other methods, such as accurately capturing response rates. For example, it is impossible to calculate true response rates to assess how many people viewed the survey webpage relative to the number of people who complete the survey. The survey platform we used, Qualtrics, did not allow us to capture the number of site visitors to a webpage hosting a survey. At this time, anonymous surveys published online and displayed on webpages via Google ads or on social media feeds such as Facebook, Twitter, or Instagram, typically cannot distinguish between the number of individuals who view an invitation but decline to click on it from the number who do not see the invitation (36). Although it is possible to capture a response rate in the context of a nonanonymous survey, such as that administered to a closed, predetermined e-mail list on a college campus (37) or predetermined list of stakeholders in clinical research trials (38), there is a continued need for reporting standards specific to Web-based surveys. Adhering to the CHERRIES protocol aids researchers in appropriate and complete reporting of Web-based surveys.

On the basis of our observed survey responses, we suggest that the best practice in question design to maximize response rates is to put all questions in a forced-response, multiple-choice framework (both true multiple-choice and write-in). This will ensure participants do not entirely skip a question and will force them to select “prefer not to answer,” thus eliminating the option to skip a question without actively indicating so. Our optional questions without “prefer not to answer” had the greatest number of skipped responses (from 900 to 1800 for the physical activity questions). In contrast, the one optional question (time spent sitting) that offered "prefer not to answer" as a multiple choice selection had few skips (n = 24, 0%), and participants chose “prefer not to answer” very few times. The skip rate for this question was lower than for the other 3 optional questions that did not offer “prefer not to answer.” In other words, adding "prefer not to answer" seemed to both deter participants from skipping the question as well as result in them providing a valid response.

There are numerous benefits to Web-based data collection for research. Online recruitment provides researchers with the ability to capitalize on the participatory culture of social media sharing to elicit responses. For example, researchers have greater access to a larger sample of potential study participants who are representative of their population of interest or can target underrepresented subgroups. Internet advertising offers the ability to expand recruitment by reaching potential study participants at a relatively low cost (39) and via recruitment partners or organizations with large followings. If pay-per-click, targeted marketing strategies, such as display ads, Google ads, Facebook ads, or others are used, then electronic tracking of advertising and recruitment enables more precise evaluation of the most successful and cost-effective recruitment strategies. For future online studies, incorporating targeted online marketing may yield a more representative study cohort in terms of sex and racial and ethnic minorities. Women are typically more willing to participate in research and thus the higher proportion observed in our study (83% women) is consistent with other studies (40). In addition, women are among the top-identified user groups of Facebook (41), which hosted much of our recruitment announcements. Our sample was also primarily white (93%).

Recruitment in the ADAPT FS was done through strategic partnerships with respective leaders from the various diet communities, and thus the sample is a self-selected sample. Because of this, the ADAPT survey results reflected the recruitment strategy—robust numbers were achieved for the most defined, active diet group communities (Paleo, vegan, whole food, plant-based). For the “no particular diet” group, recruitment was poor and translated to represent <5% of the sample. One self-selected diet group, “try to eat healthy,” was not a top-reported diet either; the exception to this was in Massachusetts, the state in which the research center was based. “Try to eat healthy” was reported by 23% of respondents in Massachusetts. The location of the research center in Massachusetts potentially enabled a broader range of connections, which influenced the recruitment differently in this locale. Furthermore, the response to our feasibility survey showed that most responses occurred after a recruitment announcement, thus underscoring the necessity of constant new announcements during a recruitment period.

Additional benefits of Web-based methods for scientists include potentially shortening the time required for data collection and administration as well as lessening the burden of manual data entry and cleaning (19). Potentially, this could lead to larger and more complete data sets through required question settings in online surveys, leading to less-intensive data cleaning than that from self-completed paper-based questionnaires. Although not used in this survey, certain software platforms for research data collection can auto-analyze results, eliminating the need for manual analysis, such as dietary data collected through the Automated Self-Administered 24-h Recall (42) (online 24-h recall) or the Diet History Questionnaire (DHQ-II) (43) (FFQ), both from the National Cancer Institute. We are using both of these instruments in the ADAPT Pilot Study, which is now ongoing.

Methodological limitations

There are several methodological limitations with the use of survey data, the primary being that the data are self-reported and may be subject to reporting bias as well as the fact that the individuals who responded to the survey were self-selected to participate in a Web-based nutrition survey. With regard to Web-based surveys, social media advertising can lead to recruiting an extremely narrowly focused sample population because respondents are limited to individuals who already subscribe to or follow the social media used. In the ADAPT FS, we could not calculate a response rate because it was a volunteer, nonprobability survey (or opt-in survey) advertised without tracking of view rates of ads. It is a common limitation to be unable to capture unique visitor rate, unique survey rate, view rate, or participation rate in Web-based contexts. Thus, it is impossible to determine the probability of selection from the total population from which the respondents were recruited (44) or whether these respondents are representative of the overall diet group populations. This, along with the prerequisites of having Internet access and a personal interest in diet and health, as well as the high proportion of white women in the sample, limits validity to the convenience sample from which the study population was drawn and is likely not generalizable to the general population. The lack of questions on income or education in this survey limits our ability to assess socioeconomic status. It is likely that the method of recruitment (social media) and the prerequisite of having Internet access may cause selection bias of higher socioeconomic status respondents.

Additional limitations of the survey resulted from the choice to make the physical activity questions optional 2 wk into the survey, as well as the inability to know the reason for survey incompletion or long time-to-finish. Other strategies could be used to encourage completion, such as incentives (36). For the time-to-finish, it is likely that >30 min may have occurred due to pausing the survey, keeping the Internet browser window open, and continuing at a later time; however, this is impossible to confirm.

Because the purpose of this phase of the study was show the feasibility of recruitment, we limited the total length of the survey to reduce respondent burden. However, an understanding of the relation between socioeconomic status and dietary choice is important in accurately characterizing these diet groups and, eventually, in understanding the factors that enable individuals to adhere to their specific diets over time. Our demographics questionnaire currently in use in the ADAPT Pilot Study contains questions on income, education, food assistance, and the affordability of the respondent's diet.

The popular diets considered in the ADAPT FS encompass a spectrum of food and nutrient intakes, resulting in wide variation in macronutrient composition from high-carbohydrate/low-fat to high-fat/high-protein/low-carbohydrate and consequently wide variation in dietary fiber and micronutrient intakes. The motivations of followers in following these diets vary, having been identified in other research as ranging from “seeking better health” to “making a positive impact on the environment” to consideration of animal welfare (45). A commonality among most of these diets is the intention to make dietary choices that improve upon the typical American diet, such as avoiding refined, highly processed foods and foods high in added sodium, sugar, and fat, (46), making such popular diets, and adherence to them, of great scientific and public health interest. Self-identification with current diet was the primary goal for this survey to test the feasibility of recruiting participants for future research to compare targeted intakes on the basis of self-identification with actual intakes based on dietary assessment. Incorporating followers of popular diets into dietary surveys will provide greater variation in dietary patterns and macronutrient composition as well as enable more in-depth study of factors that support permanent dietary behavior change in a variety of dietary patterns of higher quality than the typical American diet.

The results of this survey add new knowledge on the feasibility of research within populations. In addition, this survey adds to the growing body of literature on best methodological practices in designing and executing Web-based research studies. As research moves further in the direction of Web-based methods, maximizing response rates and increasing completion rates are 2 important aspects of sound data collection.

Conclusions

This survey suggests that it is feasible to recruit followers of popular diets with the use of Web-based methods. Recruitment strategies were most successful in attracting followers of the most well-defined and socially connected diet groups—namely, WFPB, vegan and raw vegan, and Paleo diets. The unique and highly specific nature of the study sample and recruitment methods limit its generalizability to similar diet followers. Forced-response, multiple-choice questions produced the highest response rates.

Recruitment strategies need to be refined to capture a broader demographic of study subjects; however, the unbalanced sample with respect to sex, race/ethnicity, and geographic location could be corrected through the use of targeted online marketing techniques such as Google ads or Facebook ads to better represent the overall US population. Future research on these populations should measure dietary intake to assess actual intake as compared with self-identified diet group.

Notes

Supported by USDA–Agricultural Research Service agreements 58-1950-4-003 and the Lisa Wendel Memorial Foundation.

Author disclosures: MCK, AHL, CDE, SCF, GR, PFJ, KAL, KMR, and NMM, no conflicts of interest.

Supplemental Tables 1–6 are available from the “Supplementary data” link in the online posting of the article and from the same link in the online table of contents at https://academic.oup.com/cdn/.

Abbreviations used:

ADAPT

Adhering to Dietary Approaches for Personal Taste

FS

Feasibility Survey

low-carb

low-carbohydrate

PI

principal investigator

WAP

Weston A Price

WFPB

whole-food, plant-based