Table 1.

Methods of Detecting and Preventing Internet Study Duplication and Fraud and Their Implications

Level of Intervention	Type of Intervention	Method of Detection	Method of Prevention	Pros	Cons	Additional Ethical Issues
Questionnaire/Instrument	Questions in Survey	Inconsistent Reponses	Check for proper/consistent answers	Indicates level of attention Can detect “bots”	Subjects may skip questions because of discomfort
			Include same/similar/strange questions throughout study	Indicates level of attention	Can impact experimental design
			Include questions of social desirability	Possibly help assess personality traits associated with providing inaccurate responses	Low, if any, predictability If “fraudster” not paying attention, then questions of social desirability are not helpful
	Software for Administering Survey		No back button	Subjects can’t easily resubmit survey	Doesn’t prevent, just makes fraud more difficult Eligible participants may want to change answers upon reflection
			Change order of questions with each administration	Indicates level of attention Can detect “bots”
			CAPTCHA*	Detects “bots”
			Collect paradata (i.e., subject’s behavior, e.g., time stamp, how mouse moved on the screen)	Examines how subject responding to survey	Programs that allow tracking of paradata are costly	Ethical questions of what we can see with paradata – whether to disclose to participants what we can see of their behavior
Tracking Non-Questionnaire Data	Personal Information	Similar/same email, username, password between “different” participants	Contact participant about “red flag,” and if no response, remove from study	Clears up misunderstandings	Could yield a response bias Could deter eligible subjects Doesn’t stop multiple dissimilar email, username, password	Needs to balance protecting integrity of data and subject privacy and confidentiality are particularly important
		Inaccurate/fake address & phone numbers	Researchers request to provide phone number/address to get through registration process	Participants need valid number in order to proceed	“Fraudsters” can create temporary phone numbers
			Check whether person, address, phone number is valid (through Facebook, whitepages.com, etc.)	Confirms for consistent information Deters “fraudsters” and multiple submissions	Can discourage eligible participants from taking part Subjects do not always have external validation data to ensure eligibility
			Ask participants for a website where they are listed (e.g., Facebook)	May deter “fraudsters” and multiple submissions	Can discourage eligible participants from taking part May encourage “fraudsters” to provide fake information (e.g., creating fake Facebook account)
Computer Information	IP Addresses	Same IP as another participant	Check whether IP address is the same or if it is encrypted	Can determine how many times participants took survey and whether participant fulfills location criteria (i.e., living in US)	IP addresses may be shared (roommates, university/coffee shop) IP addresses can be encrypted/scram bled/fake (e.g., using US IP address abroad) Programs to check re-routing IP addresses are costly	If eligible participant takes survey multiple times, should researchers trust the initial survey? Privacy Issue (is an IP address personal information/identification?) Should consent forms mention that researchers are tracking/not tracking IP addresses?
			Block IP address if participant is ineligible	Avoids “fraudsters” from participating	Could be dynamic IP address and not ineligible participant
	Internet Cookies	Cookies detecting completion of study and multiple attempts access study	Enable cookies	Can detect multiple submissions by tracking the progress/completion of study	“Fraudsters” can disable cookies “Fraudsters” can use different browsers Computers may be legitimately shared (e.g., computer labs or roommates) If “fraudsters” use multiple usernames, cookies would not be able to detect multiple submissions Can reveal personal information if someone checks cookies on the computer (e.g., parents seeing their child took part ina study about homosexuals)	Administering cookies without people’s knowledge. How should researchers inform participants of cookies without discouraging eligible participants from taking part?
	Tracking Survey URL	URL posted in unintended locations	Tracking/Googling URL on Internet	Can see if website where URL located is targeting proper audience	Doesn’t prevent “frausters” taking study multiple times
			Provide link in email to website and track referring URL	Can control who receives email Can avoid entries submitted that were not from the email	Researchers don’t always know the targeted population
Study Design	Informed Consent		Break up consent online and only provide compensation information at the end of all the forms	Subjects need to pay attention; and creates longer process to receive compensation, discouraging “fraudsters” Reduces “bots” from entering the system	May deter eligible pa1rticipants
	Compensation	Many gift certificates mailed to same address	Mention that subjects will not be compensated if suspect of fraudulent behavior	Avoids paying “fraudsters” yet keeps incentive
			Only inform participants of their eligibility for the study after survey	Avoids paying “fraudsters” yet keeps incentive Gives researchers time to review and determine “fraudsters” Delayed gratification may deter “fraudsters”
			Ask for mailing address (vs. email address) and verify addresses	Deters ineligible participants if researchers have means to verify addresses	May deter eligible participants (because of need to provide personal information)
			Check if multiple gift certificates are being sent to one location	Can avoid paying participants if suspected of fraudulent behavior yet keeps incentive	Linking identification to data can threaten confidentiality
			De-incentivize fraud by paying less and/or emphasizing research and the importance of social/community costs of fraud	Potential “fraudsters” may be persuaded not to skew results	Deters eligible participants if compensation not large enough “Fraudsters” may not care about importance of research or costs Gives people the idea to engage in fraudulent behavior
			Provide lottery for compensation (do not pay every person)	Gives researchers time to review and determine “fraudsters” before compensating	Lottery may not be enough of incentive for eligible participants to take part “Fraudsters” may take survey more often to increase chances of winning
	Including Interview	See whether subject already participated and/or is lying on responses	Audio Interview	May deter “fraudsters” from participating Another means to detect lying	Fraud can be hard to detect (“good liars”) Lose anonymity – could discourage eligible participants	Needs to balance protecting integrity of data and subject privacy and confidentiality are particularly important
			Skype/”face-to-face” Interview
IRBs	IRB Structure		Having an online/computer expert as a member of the IRB	Has ability to assess the study at hand and find appropriate balance to protect subjects and ensure data quality Can be up-to-date as technology and “fraudsters” advance to understand how to best prevent “fraudsters”	Does not deter “fraudsters” from taking survey multiple times
	Have PIs Report Information on “Fraudsters” to IRB		IRBs can follow and monitor to make appropriate decisions for current and future studies	May deter “fraudsters” from participating	May deter legitimate participants “Fraudsters” can create new names, emails, IP addresses for each study to avoid detection as a “fraudster”
Broader Regulatory and other Entities	Reporting Information on “Fraudsters”		PIs create “fraudster” list for other PIs and share information	PIs can have a list as a reference to easily remove “fraudster” entries May deter “fraudsters” from participating	“Fraudsters” can create new names, emails, IP addresses for each study to avoid detection as a “fraudster”	Possible harm of individuals are incorrectly classified as “fraudsters” and reported externally? Need to ensure that characterization as “fraudster” is accurate
			Reporting fraudulent behavior to Internet Crime Complaint Center (IC3.gov), OHRP or funders	May deter “fraudsters” from participating	May deter legitimate participants (some may wonder if researchers will extend reporting to include other illicit activities) “Fraudsters” can create new names, emails, IP addresses for each study to avoid detection as a “fraudster”

^*Completely Automated Public Turing test to tell Computers and Humans Apart