Abstract
Background
Enrollment and retention difficulties remain major barriers to conducting clinical trials. Financial incentives may promote clinical trial enrollment, however delivery methods to maximize enrollment, maximize retention, and minimize cost remains uncertain.
Methods
We conducted a single-blind, web-based randomized controlled trial of five financial incentive strategies on enrollment and retention rates in a longitudinal study of advance directives among community-dwelling older adults. Participants were eligible to receive a fixed total financial incentive, but the disbursement amounts at each study timepoint (baseline, 2-weeks, 4-weeks, and 6-weeks) differed between study arms. At each timepoint, participants completed a different advance directive. We conducted an intention-to-treat analysis for the primary and secondary outcomes of enrollment and retention.
Results
1803 adults were randomized to one of five incentive strategies: constant n = 361; increasing n = 357; U-shaped n = 361; surprise n = 360; self-select n = 364. Overall, 989 (54.9%) participants elected to enroll in the advance directive study. There were no differences in enrollment rates between the control (constant 53.5%) and any of the four intervention study arms (increasing 54.3%, p = 0.81; U-shaped 57.3%, p = 0.30; surprise 56.9%, p = 0.35; and self-select 52.2%, p = 0.73). There were no differences in retention rates between the control (constant 2.1%) and any of the four intervention study arms (increasing 5.2%, p = 0.09; U-shaped 3.9%, p = 0.23; surprise 2.4%, p = 0.54; self-select 2.1%, p = 0.63).
Conclusions
Financial incentive programs for trial enrollment informed by behavioral economic insights were no more effective than a constant-payment approach in this web-based pilot study.
Keywords: Enrollment, Retention, Recruitment, Compensation, Behavioral economics, Incentives
Abbreviations: RCT, Randomized controlled trial
1. Introduction
Participant recruitment and retention have been referred to as “the most difficult and challenging aspect of clinical trials.” [1] Indeed, over 40% of National Cancer Institute-sponsored clinical trials were never completed, typically due to difficulties with trial enrollment [[2], [3], [4], [5]]. Even when investigators successfully reach their target enrollment, they rarely do so on schedule [5,6]. Moreover, slow recruitment leads to costlier studies [7], and under-enrollment may result in studies that are underpowered or aborted early resulting in wasted financial resources [8] and raising ethical concerns for exposing patients to risks with reduced likelihood of benefit [9]. These issues extend beyond enrollment as low retention rates result in data missingness that risks both a decrement in power and introduces selection bias despite randomization [10].
Few interventions have been shown to address patient-reported barriers to trial participation, including uncertainty associated with randomization, potential side effects of experimental drugs, time demands, and costs [11]. Such barriers result in additional resources expended for recruitment and may compromise scientific rigor [[12], [13], [14]]. Evidence for financial incentives to promote participation in clinical trials is promising and most participants consider the practice acceptable [[15], [16], [17], [18]], but constrained research funds and ethical concerns about potential inducement limit their use.[19,20] Insights from behavioral economics may offer innovative approaches to deliver financial incentives for trial enrollment that improve their effectiveness without increasing costs [21].
For this pilot study we sought to evaluate the impact of five innovative financial incentive strategies on clinical trial enrollment and retention rates. We conducted a web-based randomized clinical trial (RCT) to test the hypotheses that a U-shaped payment strategy across the four data collection timepoints would maximize enrollment rate, while an increasing payment strategy would maximize retention, and both would be more effective than a constant payment strategy. We further hypothesized that allowing participants to control their own payment strategy or be attracted by the element of surprise would improve enrollment rates when compared to a constant incentive strategy. The goal of this study was to identify promising incentive strategies to be tested in a real-world clinical trial.
2. Methods
Setting, design and participants. We conducted a five-arm, single-blind RCT comparing five financial incentive strategies aimed at increasing clinical trial enrollment rates in a web-based, longitudinal study of healthcare advance directives. We used the web-based research survey platform Qualtrics, (Provo, Utah, USA) to recruit community-dwelling adults >50 years of age from across the United States. An older population was preferred since completion of advance directives is considered most relevant for older adults or those with a chronic life-limiting illness [22]. Potential study participants were identified by their active member status in a Qualtrics panel and invited via email to participate in a “health-related research study.” The invitation email did not include payment information. By clicking on the embedded study web link participants were automatically enrolled in the financial incentives trial and taken to the Qualtrics research site to learn more about the advance directives study. The electronic informed consent document included standard institutional internal review board-approved language about the study objective, benefits, and risks of participation to most closely represent real-world conditions. Participants were not prospectively informed about the financial incentive trial and were unblinded to their assigned payment strategy at the time of consent for participation in the advance directive study. This approach enabled an unbiased evaluation of participants’ true trial enrollment decisions based on financial incentives rather than relying on hypothetical decision-making, which may not reliably reflect real life decisions. Reminder emails were sent at each study timepoint to participants who enrolled in the advance directive study.
Intervention. Participants were randomized with 1:1 allocation to receive one of five payment strategies: constant, increasing, U-shaped, surprise, or self-select (Table 1). The study groups differed by disbursement amounts at each study timepoint, but the total potential $1.00 incentive was equal across study arms. This amount was chosen because it was consistent with the standard compensation for surveys of a similar length conducted among the targeted Qualtrics panel, and pilot testing showed a ceiling effect (98% enrollment rate) with higher incentives that more closely resembled amounts used in typical biomedical survey research. For each group, payment at each timepoint was contingent upon completion of the survey, and electronic payments were made immediately thereafter. Participants were considered “lost to follow-up” after failure to complete one study timepoint.
Table 1.
Financial incentive strategies.
| Financial Incentive | Baseline | Week 2 | Week 4 | Week 6 | Total Study Payments | Behavioral Rationale |
|---|---|---|---|---|---|---|
| Constant | $0.25 | $0.25 | $0.25 | $0.25 | $1.00 | Control condition |
| Increasing | $0.10 | $0.20 | $0.30 | $0.40 | $1.00 | Sunk cost bias and loss aversion [23,24] |
| U-shaped | $0.40 | $0.10 | $0.10 | $0.40 | $1.00 | Larger incentives more effective than smaller incentives [25] |
| Surprisea | ($0.20) | ($0.10) | ($0.30) | ($0.40) | $1.00 | Lottery-based incentive [26] |
| Self-select | participants chose preferred incentive arm | $1.00 | Giving control enables intrinsic motivation [27] | |||
Participants in the surprise arm were informed of the total potential incentive amount, but not the disbursement amounts at each timepoint.
The constant payment strategy served as a control group. We hypothesized that an increasing payment strategy would lead to similar enrollment rates but higher retention rates, while a U-shaped payment strategy would maximize enrollment but might risk lower retention due to the natural tendency for people to discount rewards too far in the future [28], such as the higher last payment. Payments in the surprise arm were blindly allocated in random order in an attempt to leverage the element of surprise in augmenting enrollment. Similarly, participants in the self-select arm were asked to choose their preferred payment strategy among the four options in an effort to offer a sense of control.
Outcomes and data collection. The primary outcome of enrollment rate was defined as the proportion of eligible individuals who provided informed consent to participate in the advance directive study after effectively being “approached” for recruitment upon clicking on the study web link in the email invitation, which did not contain any study specific information in order to minimize potential bias. The secondary outcome was retention rate, defined as the proportion of enrolled participants who completed the entire study. Other subject data collected included sociodemographics, self-reported past medical history, prior experience with the intensive care unit, and palliative care.
Statistical analysis. We assumed a 60% enrollment rate in the constant payment arm. A total sample size of 1800 participants would achieve >80% power to detect an absolute 10% difference in the proportion of patients who enroll between the control arm and each intervention arm at an of 0.05 based on a chi-square test [29]. Summary data are reported as N(%) unless otherwise indicated. We performed bivariate analyses of subject characteristics by enrollment status using chi-square or t-test for categorical and continuous variables, respectively. Due to data collection limitations on the Qualtrics platform among participants who declined to participate in the advance directive study, we were unable to assess differences in subject characteristics across study arms. We performed unadjusted intention-to-treat analyses of enrollment and retention rates between the control and each intervention arm using a chi-square test or fisher's exact test as appropriate. Given the exploratory nature of this study we did not adjust for multiple comparisons [[30], [31], [32], [33]]. All analyses were performed using Stata v14.2 (StataCorp, College Station, Tx) and R 3.5.0. The University of Pennsylvania Institutional Review Board deemed this minimal risk research eligible for review exemption.
3. Results
We randomized 1837 Qualtrics panel members to receive a financial incentive strategy (constant n = 367; increasing n = 368; U-shaped n = 367; surprise n = 368; self-select n = 367) for participation in the advance directive study (Fig. 1). Post-randomization subject exclusions included 34 for self-reported age <50 years and 3 for completely missing data, for a total of 1800 participants included in analyses.
Fig. 1.
Assessment for eligibility and randomization. a3 participants were dropped from the intention-to-treat analysis for completely missing data.
Participants were predominantly Caucasian (84.8%), female (63.5%), and had a mean age of 60.8 years (standard deviation 7.8). Most participants assigned to the self-select incentive strategy chose a constant payment (66.8%), with fewer opting for the U-shaped (15.3%), surprise (16.3%) or increasing (1.6%) payments. Overall, 989 (54.9%) participants enrolled in the advance directive study. Participants who enrolled tended to be younger and female gender (Table 2). Among enrolled participants, there was an imbalance of co-morbid lung disease(p = 0.03); other characteristics were similar across the five study arms (Table 3).
Table 2.
Participants characteristics (intention-to-treat populationa).
| Characteristic | Enrolled (n = 989) | Declined (n = 811) |
|---|---|---|
| Age, mean (±SD)b | 60.0 (±7.5) | 61.8 (±8.0) |
| Gender | n (%) | n (%) |
| Femaleb | 654 (66.1) | 487 (60.3) |
| Race/Ethnicity | ||
| White | 831 (84.1) | 694 (85.6) |
| Black | 58 (5.9) | 44 (5.4) |
| Hispanic | 48 (4.9) | 36 (4.4) |
| Other | 51 (5.2) | 37 (4.6) |
| Employment | ||
| Not employed | 510 (54.4) | 404 (53.4) |
| Gross household income | ||
| $39,999 | 342 (36.2) | 267 (35.7) |
| $40,000–74,999 | 309 (32.7) | 213 (28.5) |
| $75,000 | 294 (31.1) | 268 (35.8) |
| Marital status | ||
| Married | 519 (52.5) | 462 (57.0) |
| Education completed | ||
| College or beyond | 446 (45.7) | 368 (47.7) |
The intention-to-treat population includes all participants who underwent randomization to the five study groups and met age eligibility criteria and had any available data.
P < 0.01; all other characteristics showed nonsignificant differences with P > 0.05.
Table 3.
Participant characteristics by financial incentive arm among those who enrolled (N = 989).
| Characteristic | Constant (n = 193) | Increasing (n = 194) | U-shaped (n = 207) | Surprise (n = 205) | Self-Select (n = 190) |
|---|---|---|---|---|---|
| Age, mean (±SD) | 59.6 (±7.5) | 61.1 (±7.6) | 59.9 (±7.5) | 59.9 (±7.6) | 59.7 (±7.3) |
| Gender | n (%) | n (%) | n (%) | n (%) | n (%) |
| Female | 134 (69.4) | 128 (66.0) | 130 (62.8) | 135 (65.9) | 127 (66.8) |
| Race/Ethnicity | |||||
| White | 165 (85.5) | 159 (82.4) | 172 (83.1) | 170 (82.9) | 165 (86.8) |
| Black | 8 (4.1) | 17 (8.8) | 13 (6.3) | 12 (5.9) | 8 (4.2) |
| Hispanic | 11 (5.7) | 9 (4.7) | 10 (4.8) | 9 (4.4) | 9 (4.7) |
| Other | 9 (4.7) | 8 (4.1) | 12 (5.8) | 14 (6.8) | 8 (4.2) |
| Employment | |||||
| Employed | 85 (46.2) | 79 (43.4) | 89 (44.7) | 88 (45.1) | 87 (48.9) |
| Gross household income | |||||
| $39,999 | 65 (35.1) | 67 (35.8) | 80 (40.4) | 67 (33.7) | 63 (35.8) |
| $40,000–74,999 | 53 (28.6) | 67 (35.8) | 64 (32.3) | 64 (32.2) | 61 (34.7) |
| $75,000 | 67 (36.2) | 53 (28.3) | 54 (27.3) | 68 (34.2) | 52 (29.5) |
| Marital status | |||||
| Married | 104 (53.9) | 100 (51.5) | 109 (52.7) | 104 (50.7) | 102 (53.7) |
| Education completed | |||||
| College or beyond | 87 (45.5) | 81 (42.4) | 91 (44.4) | 108 (53.7) | 79 (42.0) |
| Comorbidities | |||||
| Cancer | 15 (7.8) | 13 (6.7) | 8 (3.9) | 15 (7.3) | 13 (6.8) |
| Cardiac disease | 14 (7.3) | 20 (10.3) | 15 (7.2) | 12 (5.9) | 13 (6.8) |
| Kidney disease | 6 (3.1) | 9 (4.6) | 6 (2.9) | 6 (2.9) | 5 (2.6) |
| Liver disease | 4 (2.1) | 3 (1.5) | 8 (3.9) | 3 (1.5) | 1 (0.5) |
| Lung disease | 5 (2.6) | 10 (5.2) | 19 (9.2) | 8 (3.9) | 8 (4.2) |
| Mental health disorder | 15 (7.8) | 13 (6.7) | 18 (8.7) | 13 (6.3) | 18 (9.5) |
| Neurologic disease | 6 (3.1) | 12 (6.2) | 9 (4.3) | 6 (2.9) | 12 (6.3) |
| Serious illness experience | |||||
| ICU patient | 7 (3.6) | 18 (9.3) | 17 (8.2) | 15 (7.3) | 14 (7.4) |
| ICU visitor | 110 (57.0) | 119 (61.3) | 117 (56.5) | 118 (57.8) | 113 (59.5) |
| Death of loved one | 139 (72.0) | 154 (79.4) | 144 (69.6) | 143 (69.8) | 142 (74.7) |
In intention-to-treat analyses, there were no differences in enrollment rates between the control (constant 53.5%) and any of the four intervention study arms (increasing 54.3%, p = 0.81; U-shaped 57.3%, p = 0.30; surprise 56.9%, p = 0.35; and self-select 52.2%, p = 0.73) (Fig. 2). There were also no differences in retention rates between the control (constant 2.1%) and any of the four intervention study arms (increasing 5.2%, p = 0.09; U-shaped 3.9%, p = 0.23; surprise 2.4%, p = 0.54; self-select 2.1%, p = 0.63) (Fig. 2). The median time to complete each advanced directive was 4 min and 49 s (IQR: 2 min 50 s–7 min 9 s).
Fig. 2.
Enrollment and retention rates by financial incentive strategy.
4. Discussion
Clinical trials represent the highest standard for evidence-based medicine, yet slow and under-enrollment continue to pose threats to the cost and scientific rigor of such critical research efforts. In this large randomized pilot study, we found that none of the four financial incentive strategies informed by principles of behavioral economics increased enrollment or retention rates in this clinical research study when compared to a constant incentive strategy. The enrollment rate in the constant (control) payment arm was moderate and consistent with our expectations. However, we were surprised to find no improvement in enrollment rates with the U-shaped, surprise or self-select strategies.
There are several potential explanations for the study's null findings. First, and perhaps most relevant for future research, financial compensation for time and effort is just one of several considerations for individuals deciding whether to enroll in a clinical trial. Thus, different approaches to offering actuarially equivalent incentives may be immaterial if barriers such as perceived study risks or distrust of biomedical research are highly operative [34]. Second, the enrollment rate being similar regardless of initial payment amount may highlight a ceiling or floor effect of enrollment in research studies implemented on this web-based platform. Although the direction of the effect cannot be determined from these data, we attempted to avoid both by scaling the payments to the historical compensation plan used among this Qualtrics member panel. However, beyond the median completion time of 4 min and 49 s, it is unknown how the cognitive effort for completing different versions of advance directives compared to other studies offered to this panel. We purposefully chose a high cognitive effort task to mimic real-world clinical research and ensure future applicability of effective interventions. The overall payments in our study may have been perceived by eligible participants as relatively too low, thereby negating the intended effects of different payment paradigms. Third, although the total payment was scaled appropriately for this panel, it is possible that the absolute difference between individual disbursement amounts at each timepoint was not sufficiently large to influence decision-making. Finally, just as people respond differently to health-related behavioral interventions [35], it is possible that certain types of individuals were more responsive to their assigned financial incentive strategy than others [36,37]. For example, finding that younger and female participants were more likely to enroll overall merits further investigation.
Another interesting finding was the preferred option of a constant incentive among individuals assigned to select their payment strategy. Traditional economic theory might have predicted that a higher incentive early on, as with the U-shaped approach, would promote clinical trial enrollment [25]. The failure to detect such an effect may again be attributable to an overall ceiling or floor effect, but could also be attributed to ordering effects since the constant incentive option was listed first [38].
The third finding was that contrary to our expectation retention rates in this web-based study were not superior for the increasing or any other interventional payment strategy compared with control. However, the overall very low retention rate precludes reasonable inferences from such data. We suspect this result was primarily related to a loss of interest or unanticipated competing tasks. Given the increasing focus on long-term outcomes after serious illness or hospitalization, poor participant retention in longitudinal research may be as problematic as slow or under-enrollment. While it is likely that some decision-making processes overlap between enrollment and follow-up, there are additional barriers to consider when designing behavioral interventions to improve study retention, including competing tasks, time constraints, loss of interest, or decline in health status over time. Using non-financial behavioral economic interventions at follow-up timepoints, such as appealing to one's altruistic motivations or moral duty, using social norms, or providing peer-based comparisons [[39], [40], [41], [42], [43]], may be more effective at improving retention than financial incentives alone.
There are several important limitations of this study. First, the use of a web-based platform may limit generalizability to real world clinical trials. Although the optimal scenario would have been to test different financial incentive strategies for enrollment in an actual clinical trial, few real-world trials provide the sample size necessary to test several promising approaches at one time. Second, the responses of community-dwelling adults may not predict how patients would respond. We limited the eligible age criteria to >50 years in an effort to identify people more likely to have the intrinsic motivation to complete an advance directive, yet the rates of chronic disease in this cohort were low. Third, the financial incentives used in this study were lower than is typical for most biomedical research studies and may have been too low to motivate behavior change. As mentioned above, the payments were chosen intentionally to avoid a ceiling effect on enrollment by relatively over-incentivizing this Qualtrics panel and all arms were eligible to receive the same total amount. Finally, we were unable to evaluate heterogeneity of treatment effects or participant characteristics independently associated with enrollment due to our inability to access stratified sociodemographic data among those who declined to enroll.
5. Conclusions
Financial incentives may be an important tool in improving participation in clinical trials, but the most cost-effective way to offer incentives remains unknown. In this pilot web-based RCT, we found that applying insights from behavioral economics to modify the schedule of financial incentives for participation in a clinical trial did not increase enrollment or retention over time. Future research should focus on understanding which types of patients are less likely to enroll or follow-up, and why, in order to develop novel behavioral interventions that can be targeted to address such barriers.
Funding
The project described was supported in part by Grant Number UL1TR001878 from the National Center for Advancing Translational Science through the Institute for Translational Medicine and Therapeutics (ITMAT) at the University of Pennsylvania (KRC). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Advancing Translational Science or the National Institutes of Health.
Author's contributions
KRC, KNY, EC, SB, and SDH contributed to the study design and protocol. DCK, KRC and SDH drafted the manuscript. KNY, SB, and EC provided critical feedback and revisions. All authors have provided final approval of the study protocol.
Footnotes
Supplementary data to this article can be found online at https://doi.org/10.1016/j.conctc.2019.100390.
Contributor Information
Dustin C. Krutsinger, Email: Dustin.Krutsinger@uphs.upenn.edu.
Kuldeep N. Yadav, Email: kyadav@mail.med.upenn.edu.
Elizabeth Cooney, Email: elcooney@upenn.edu.
Steven Brooks, Email: stevegbrooks@gmail.com.
Scott D. Halpern, Email: shalpern@upenn.edu.
Katherine R. Courtright, Email: katherine.courtright@pennmedicine.upenn.edu.
Appendix A. Supplementary data
The following is the Supplementary data to this article:
References
- 1.Nathan R.A. How important is patient recruitment in performing clinical trials? J. Asthma. 1999;36:213–216. doi: 10.3109/02770909909075405. [DOI] [PubMed] [Google Scholar]
- 2.Stensland K.D., McBride R.B., Latif A. Adult cancer clinical trials that fail to complete: an epidemic? J. Natl. Cancer Inst. 2014:106. doi: 10.1093/jnci/dju229. [DOI] [PubMed] [Google Scholar]
- 3.Baldi I., Lanera C., Berchialla P., Gregori D. Early termination of cardiovascular trials as a consequence of poor accrual: analysis of ClinicalTrials.gov 2006-2015. BMJ open. 2017;7 doi: 10.1136/bmjopen-2016-013482. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Carlisle B., Kimmelman J., Ramsay T., MacKinnon N. Unsuccessful trial accrual and human subjects protections: an empirical analysis of recently closed trials. Clin. Trials. 2015;12:77–83. doi: 10.1177/1740774514558307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Schroen A.T., Petroni G.R., Wang H. Preliminary evaluation of factors associated with premature trial closure and feasibility of accrual benchmarks in phase III oncology trials. Clin. Trials. 2010;7:312–321. doi: 10.1177/1740774510374973. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Tang C., Sherman S.I., Price M. Clinical trial characteristics and barriers to participant accrual: the MD anderson cancer center experience over 30 years, a historical foundation for trial improvement. Clin. Cancer Res.: Off. J. Am. Assoc. Cancer Res. 2017;23:1414–1421. doi: 10.1158/1078-0432.CCR-16-2439. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Emanuel E.J., Schnipper L.E., Kamin D.Y., Levinson J., Lichter A.S. The costs of conducting clinical research. J. Clin. Oncol.: Off. J. Am. Soc. Clin. Oncol. 2003;21:4145–4150. doi: 10.1200/JCO.2003.08.156. [DOI] [PubMed] [Google Scholar]
- 8.Institute of Medicine Committee on Cancer Clinical . Trials the NCI cooperative group. In: Nass S.J., Moses H.L., Mendelsohn J., editors. A National Cancer Clinical Trials System for the 21st Century: Reinvigorating the NCI Cooperative Group Program. National Academies Press (US) Copyright 2010 by the National Academy of Sciences; Washington (DC): 2010. All rights reserved. [PubMed] [Google Scholar]
- 9.Halpern S.D., Karlawish J.H., Berlin J.A. The continuing unethical conduct of underpowered clinical trials. Jama. 2002;288:358–362. doi: 10.1001/jama.288.3.358. [DOI] [PubMed] [Google Scholar]
- 10.Kearney A., Daykin A., Shaw A.R.G. Identifying research priorities for effective retention strategies in clinical trials. Trials. 2017;18:406. doi: 10.1186/s13063-017-2132-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Carroll R., Antigua J., Taichman D. Motivations of patients with pulmonary arterial hypertension to participate in randomized clinical trials. Clin. Trials. 2012;9:348–357. doi: 10.1177/1740774512438981. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Free C., Hoile E., Robertson S., Knight R. Three controlled trials of interventions to increase recruitment to a randomized controlled trial of mobile phone based smoking cessation support. Clin. Trials. 2010;7:265–273. doi: 10.1177/1740774510367687. [DOI] [PubMed] [Google Scholar]
- 13.Treweek S., Pitkethly M., Cook J. Strategies to improve recruitment to randomised trials. Cochrane Database Syst. Rev. 2018;2 doi: 10.1002/14651858.MR000013.pub6. Mr000013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Jennings C.G., MacDonald T.M., Wei L. Does offering an incentive payment improve recruitment to clinical trials and increase the proportion of socially deprived and elderly participants? Trials. 2015;16:80. doi: 10.1186/s13063-015-0582-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Breitkopf C.R., Loza M., Vincent K. Perceptions of reimbursement for clinical trial participation. J. Empir. Res. Hum. Res. Ethics: JERHRE. 2011;6:31–38. doi: 10.1525/jer.2011.6.3.31. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Slomka J., McCurdy S., Ratliff E.A., Timpson S., Williams M.L. Perceptions of financial payment for research participation among African-American drug users in HIV studies. J. Gen. Intern. Med. 2007;22:1403–1409. doi: 10.1007/s11606-007-0319-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Stunkel L., Grady C. More than the money: a review of the literature examining healthy volunteer motivations. Contemp. Clin. Trials. 2011;32:342–352. doi: 10.1016/j.cct.2010.12.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Shah A., Efstathiou J.A., Paly J.J. Prospective preference assessment of patients' willingness to participate in a randomized controlled trial of intensity-modulated radiotherapy versus proton therapy for localized prostate cancer. Int. J. Radiat. Oncol. Biol. Phys. 2012;83:e13–e19. doi: 10.1016/j.ijrobp.2011.11.072. [DOI] [PubMed] [Google Scholar]
- 19.Martin R. Undue inducement in clinical research. Lancet (London, England) 2005;366:275–276. doi: 10.1016/S0140-6736(05)66964-4. [DOI] [PubMed] [Google Scholar]
- 20.Elliott C., Abadie R. Exploiting a research underclass in phase 1 clinical trials. N. Engl. J. Med. 2008;358:2316–2317. doi: 10.1056/NEJMp0801872. [DOI] [PubMed] [Google Scholar]
- 21.Volpp K.G., Pauly M.V., Loewenstein G., Bangsberg D. P4P4P: an agenda for research on pay-for-performance for patients. Health Aff. 2009;28:206–214. doi: 10.1377/hlthaff.28.1.206. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Halpern S.D. Shaping end-of-life care: behavioral economics and advance directives. Semin. Respir. Crit. Care Med. 2012;33:393–400. doi: 10.1055/s-0032-1322403. [DOI] [PubMed] [Google Scholar]
- 23.Magalhaes P., Geoffrey White K. The sunk cost effect across species: a review of persistence in a course of action due to prior investment. J. Exp. Anal. Behav. 2016;105:339–361. doi: 10.1002/jeab.202. [DOI] [PubMed] [Google Scholar]
- 24.Emanuel E.J., Ubel P.A., Kessler J.B. Using behavioral economics to design physician incentives that deliver high-value care. Ann. Intern. Med. 2016;164:114–119. doi: 10.7326/M15-1330. [DOI] [PubMed] [Google Scholar]
- 25.Halpern S.D., Karlawish J.H., Casarett D., Berlin J.A., Asch D.A. Empirical assessment of whether moderate payments are undue or unjust inducements for participation in clinical trials. Arch. Intern. Med. 2004;164:801–803. doi: 10.1001/archinte.164.7.801. [DOI] [PubMed] [Google Scholar]
- 26.Halpern S.D., Kohn R., Dornbrand-Lo A. Lottery-based versus fixed incentives to increase clinicians' response to surveys. Health Serv. Res. 2011;46:1663–1674. doi: 10.1111/j.1475-6773.2011.01264.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Turnbull A.E., O'Connor C.L., Lau B., Halpern S.D., Needham D.M. Allowing physicians to choose the value of compensation for participation in a web-based survey: randomized controlled trial. J. Med. Internet Res. 2015;17:e189. doi: 10.2196/jmir.3898. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Stein J.S., Sze Y.Y., Athamneh L., Koffarnus M.N., Epstein L.H., Bickel W.K. Think fast: rapid assessment of the effects of episodic future thinking on delay discounting in overweight/obese participants. J. Behav. Med. 2017;40:832–838. doi: 10.1007/s10865-017-9857-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.D'Agostino R.B., Chase W., Belanger A. The appropriateness of some common procedures for testing the equality of two independent binomial populations. Am. Statistician. 1988;42:198–202. [Google Scholar]
- 30.Althouse A.D. Adjust for multiple comparisons? It's not that simple. Ann. Thorac. Surg. 2016;101:1644–1645. doi: 10.1016/j.athoracsur.2015.11.024. [DOI] [PubMed] [Google Scholar]
- 31.Feise R.J. Do multiple outcome measures require p-value adjustment? BMC Med. Res. Methodol. 2002;2:8. doi: 10.1186/1471-2288-2-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Perneger T.V. What's wrong with Bonferroni adjustments. Br. Med. J. 1998;316:1236–1238. doi: 10.1136/bmj.316.7139.1236. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Rothman K.J. No adjustments are needed for multiple comparisons. Epidemiology. 1990;1:43–46. [PubMed] [Google Scholar]
- 34.Shavers V.L., Lynch C.F., Burmeister L.F. Racial differences in factors that influence the willingness to participate in medical research studies. Ann. Epidemiol. 2002;12:248–256. doi: 10.1016/s1047-2797(01)00265-4. [DOI] [PubMed] [Google Scholar]
- 35.Edlind M., Mitra N., Grande D. Why effective interventions do not work for all patients: exploring variation in response to a chronic disease management intervention. Med. Care. 2018;56:719–726. doi: 10.1097/MLR.0000000000000939. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Fehr E., Kosfeld M. The lure of authority: motivation and incentive effects of power. Am. Econ. Rev. 2013;4:1325–1359. [Google Scholar]
- 37.Zuckerman M., Porac J., Lathin D., Smith R., Deci E.L. On the importance of self-determination for intrinsically motivated behavior. Pers. Soc. Psychol. Bull. 1978;4:443–446. [Google Scholar]
- 38.Krosnick J.A., Alwin D.F. An evaluation of a cognitive theory of response-order effects in survey measurement. Publ. Opin. Q. 1987;51:201–219. [Google Scholar]
- 39.Harris J. Scientific research is a moral duty. J. Med. Ethics. 2005;31:242–248. doi: 10.1136/jme.2005.011973. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Ridout B., Campbell A. Using Facebook to deliver a social norm intervention to reduce problem drinking at university. Drug Alcohol Rev. 2014;33:667–673. doi: 10.1111/dar.12141. [DOI] [PubMed] [Google Scholar]
- 41.Schaefer G.O., Emanuel E.J., Wertheimer A. The obligation to participate in biomedical research. JAMA. 2009;302:67–72. doi: 10.1001/jama.2009.931. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Persell S.D., Friedberg M.W., Meeker D. Use of behavioral economics and social psychology to improve treatment of acute respiratory infections (BEARI): rationale and design of a cluster randomized controlled trial [1RC4AG039115-01]--study protocol and baseline practice and provider characteristics. BMC Infect. Dis. 2013;13:290. doi: 10.1186/1471-2334-13-290. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Jeffrey J., Whelan J., Pirouz D.M., Snowdon A.W. Boosting safety behaviour: descriptive norms encourage child booster seat usage amongst low involvement parents. Accid. Anal. Prev. 2016;92:184–188. doi: 10.1016/j.aap.2016.03.006. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.