Development and Preliminary Results of the Financial Incentive Coercion Assessment (FICA) Questionnaire

Abstract

Financial incentives are often used in research, yet no measure exists to determine whether they lead to perceptions of coercion in subjects. We present a preliminary evaluation of a recently developed Financial Incentive Coercion Assessment (FICA) questionnaire. FICA measures perceived coercion specifically related to payment for participation in a research study. Two hundred and sixty-six subjects were recruited from a large randomized controlled trial; 152 returned for a six-month follow-up and completed the FICA. Approximately 30% of participants reported the major reason for participating was “for the money,” but less than 5% felt that the financial incentives were coercive. FICA results are consistent with levels of perceived coercion using an alternative measure. Initial assessment of responses on the FICA suggests that it may provide a novel approach to measuring perceived coercion from financial incentives in research. Future work will refine the FICA and analyze its psychometric properties.

1. Introduction

Efficient, valid, and generalizable research requires investigators to recruit and retain adequate and representative samples (Robiner 2005) and to maximize adherence to protocols. Swift recruitment, high retention, and good adherence, however, are difficult to achieve and often present persistent challenges for many clinical and behavioral studies. Prolonged recruitment increases the costs of conducting research, reduces the number of studies that can be funded, and slows the rate at which research and accompanying medical advances can proceed (Neuer 2003; LoRusso et al. 2010). Poor adherence delays the translation of new treatments to the clinical setting, increases the cost of research, and increases the likelihood that clinical trial data may be misinterpreted or that the validity of the results may be compromised (Urquhart 1999; Johnson & Remien 2003; Robiner 2005; Touloumi et al. 2002; Robiner 2005).

A common solution to these challenges has been to offer financial incentives. Studies have shown that offering potential subjects financial incentives has led to more rapid recruitment of study participants (Halpern et al. 2004; Watson & Torgerson 2006; Caldwell et al. 2010). For example, offering financial incentives greatly increased recruitment, participation, and retention in interventions for HIV/STD treatment and prevention (Kamb et al. 1998; Walker et al. 2011), substance abuse treatment (Roozen et al. 2004; García-Fernández et al. 2011), and dietary behavior (Wall et al. 2008; Thompson et al. 2008). Financial incentives have also been shown to increase compliance and retention in behavioral interventions (e.g., Volpp et al. 2009; Cahill & Perera 2011), to increase abstinence from smoking (Bains et al. 1998; Volpp et al. 2006; Volpp et al. 2009), and to increase response rates for mail surveys (Doody et al. 2003).

While it is clear that financial incentives improve recruitment, retention, and compliance in research studies (Lind 1996), the use of such incentives has long been controversial (Ackerman 1989; Macklin 1989). The Office for Human Research Protection of the National Institutes of Health (formerly the Office for Protection from Research Risks) provides extensive discussion of the use of financial incentives in research in its Institutional Review Board Guidebook and emphasizes that the informed consent process should be free of coercion and undue influence for participation (OHRP 1993, Chapter 3, Section G). The guidebook, however, does not define in any detail what constitutes an appropriate payment or incentive. While U.S. regulations stress the need for the consent process to be voluntary and free of coercion and undue influence, they do not provide specific direction as to which incentives are appropriate (Palmer 1985; Brody 1998).

The question as to whether financial incentives in research studies are coercive remains unanswered. At a fundamental level, some individuals question whether offering payment for research participation falls within the philosophical definition of coercion (Wertheimer 1987; Wilkinson 2005; Singer & Bossarte 2006; Wertheimer & Miller 2008). This debate has redirected the discussion to focus on participants’ perception of whether payments are coercive or unduly influential in their decisions about participating (Festinger et al. 2005; Croft et al. 2007; Festinger et al. 2008).

One of the main barriers to determining whether financial incentives in research are perceived as coercive is a lack of established measures of perceived coercion developed specifically for this situation. Researchers are often forced to modify measures of coercion that, while valid, were originally designed to measure coercion to treatment rather than research or that have only a limited applicability to financial coercion (Festinger et al. 2005; Festinger et al. 2008). This underscores the need for a particular type of coercion assessment instrument. The Financial Incentive Coercion Assessment (FICA) questionnaire that we have developed, and which will form the foundation of a perceived coercion instrument, addresses both research-related perceived coercion and the levels of perceived coercion in a series of non-research related scenarios. This allows for measurement of an individual’s overall tendency for perceiving any payments as coercive. Furthermore, the newly developed questionnaire measures coercion along a continuum rather than on a more restrictive binary true/false response scale. Although we ultimately dichotomized the continuum in the results presented here, use of a continuum allows researchers to choose the number of levels of perceived coercion that they feel is appropriate for their situation.

This paper presents findings from the pilot evaluation of the FICA within a larger host study that examined the ethics and efficacy of using different modes and magnitudes of follow-up payments (Festinger et al. 2008). The goal of the research presented here is to examine initial results from the FICA, and determine whether research and non-research related incentives are viewed as coercive and whether they are positively associated with each other.

2. Materials and Methods

For this study, Institutional Review Board (IRB) review and approval was obtained from the Treatment Research Institute’s IRB and the City of Philadelphia.

2.1 Research design and participants

The host study (see Festinger et al. 2008 for a detailed description) utilized a 2×4 factorial design to examine the ethics and efficacy of providing different modes and magnitudes of follow-up payment. Specifically, participants recruited from a large outpatient substance abuse treatment program were randomly assigned to receive $70, $100, $130, or $160 in either cash or a gift certificate for attending a scheduled six-month follow-up assessment. The primary aims of the host study were to determine whether mode or magnitude of payment influenced new drug use, improved follow-up rates, and/or increased perceived coercion to participate in the study. The primary aim of the research reported here was to undertake an initial assessment of a new questionnaire, the FICA. The purpose of the FICA is to assess perceived coercion, both in a research setting where financial incentives for participation are present and in non-research scenarios.

The FICA questionnaire was added to the host study after the study had been initiated. Therefore, although there were 419 participants in the host study, only 266 individuals were enrolled in the study after the FICA questionnaire was added. Those individuals who enrolled before the FICA was included did not complete the FICA at follow-up. As in the host study, individuals who were enrolled after the FICA was added were enrolled in the study within two weeks of initial admission to the substance abuse treatment program. All participants provided informed consent, and all procedures were in accord with the Helsinki Declaration of 1975. At baseline, immediately following provision of informed consent, participants completed a questionnaire that collected basic demographic and contact information. Participants were then scheduled for a six-month follow-up assessment and told of the mode and magnitude of remuneration that they would receive for attending the follow-up. Participants were aware of the different levels and modes of payment when they consented, as it was not feasible to keep them blinded to this information. At the six-month follow-up, participants completed a brief set of instruments that included a modified Perceived Coercion Scale of the MacArthur Admissions Experience Survey (MAES) (Gardner et al. 1993) and the FICA. We chose to use only the five-item Perceived Coercion Scale, as it is most relevant for the current study. In our modified version, the items were changed to reflect perceptions of coercion to participate in research rather than coercion related to inpatient psychiatric admission. A sample item from the original version is “I felt free to do what I wanted about coming to this hospital.” In the modified measure, this item was changed to “I felt free to do what I wanted about coming in for follow-up.” All instruments and questionnaires were administered by a Research Associate to ameliorate any potential literacy concerns.

2.2 Development of the FICA

There were three main components of the FICA. The first and most straightforward component addressed the perceived coerciveness of the research payment. Individuals were asked to pick the response choice that best fit their reaction to being offered the financial incentive (i.e., “When I was offered the compensation for participating in the study…”) (Table 1).

Table 1.

Non-research and research offers and statements describing attitudes towards these scenarios and the financial incentive for research participation

Non-Research Scenarios: Individuals picked the response choice that most closely matched their reaction to the scenario.

A person from your apartment building offered to give you $1000 to feed her cat for 2 days

You receive a post card that offered you a $5 coupon to visit a store

A person on your block offered to give you $100 to pick up his mail for a week

A local park offered you $5 to pick up trash in the park for a day

A local store offered you $1000 to hold up an advertisement banner for 8 hours on a busy  highway

The parks and recreation department offered you $100 if you repaint 100 blocks of yellow  curb

Research Related Question: Individuals picked the response choice that most closely matched their reaction to being offered the compensation.

When I was offered the compensation for participating in the study….

Choice of Responses/Attitude towards Offers

I felt like I had no choice but to agree so that I could get the moneyc

I felt a great deal of pressure to agree so that I could get the moneyc

I felt like I had no choice but to agreec

I felt it was too good of an offer to turn down

I felt it was a tempting offer

I felt very influenced by the money offered

I didn’t feel pressured to accept this offer

^cIndicates those phrases that express the perception of coercion

Second, six non-research scenarios were developed to gauge overall tendency for perceived coerciveness (Table 1). These scenarios represented a broad range of task difficulty and payment levels, including easy tasks for a small amount of money (e.g., “You receive a post card that offered you a $5 coupon to visit a store.”), difficult tasks for a large amount of money (e.g., “A local store offered you $1000 to hold up an advertisement banner for 8 hours on a busy highway.”), easy tasks for a large amount of money (e.g., “A person from your apartment building offered to give you $1,000 to feed her cat for two days”), and difficult tasks for a small amount of money (e.g., “A local park offered you $5 to pick up trash in the park for a day”). Again, individuals were asked to pick the response choice that best fit their reaction to each scenario.

The third component was the response choices. In developing these, we aimed at creating a continuum of responses that indicated a high level of perceived coercion (e.g., “I felt like I had no choice but to agree so that I could get the money”) to low levels of perceived coercion (e.g., “I didn’t feel pressured to accept this offer”). For the results presented here, we dichotomized the seven possible response choices to indicate whether the respondent perceived the offer as coercive. All response choices are shown in Table 1, with those that are categorized as indicating perceived coercion labeled with a superscript “c.”

In developing both the six non-research scenarios and the seven response choices, we reviewed the perceived coercion literature and consulted with both an expert panel and a wide range of individuals, including office workers at Treatment Research Institute and clients at the treatment center. The latter group comprises the population from which participants for the study were recruited. We originally identified a larger range of scenarios and choice responses. Then, through an iterative process of consultation and item/response revision, we developed and settled on a more condensed set of items for this preliminary version of the FICA. The final step was to assess the relative placement of the choice responses on a perceived coerciveness continuum, as well as which choice responses indicated perceived coercion. To do this, we circulated the choice responses to individuals with expertise in ethics, substance abuse treatment, and health economics, as well as to selected TRI employees who have a long history of working with substance abuse treatment clients. We asked them to rank the choice responses by how much “coercion” they conveyed, and also to rate each phrase as indicating perceived coercion or not. Following that process, the study team and content experts met to reach consensus on the rankings of the phrases and the ratings of coerciveness. In this way, gathering a variety of opinions and obtaining expert panel consensus, phrases were rated as either expressing feelings of coercion or not.

In addition to the items noted above, the FICA asked the participants to indicate the reason(s) that they chose to join the host study among a list of eight options. The FICA also asks subjects whether they would have participated in the host study in the absence of compensation. If they answered “no,” a follow-up question inquired about the minimum compensation that would be necessary to convince them to participate. Finally, the FICA included two questions, developed by the investigators, concerning the perceived fairness of the compensation. On a six-point scale ranging from “strongly agree” to “strongly disagree,” participants were asked a) whether they participated in the study despite feeling that the compensation was unfairly low and b) whether they participated in the study because they needed the money, however little it may have been.

2.3 Analyses

Given the nature of the FICA data associated with the research questions for this pilot study, the analyses comprised basic calculations and comparisons of variable counts, means, and distributions. First, we calculated the number and percentage of individuals who chose each of the possible reasons for participating in the host study as their main reason as well as the number of individuals who endorsed other reasons as a secondary factor. Next, we examined the number of times each of the designated perceived coercion phrases were endorsed by participants to indicate how they viewed each of the non-research scenarios. These choices were then aggregated to arrive at the number of individuals we labeled as perceiving coercion from the offer of money in each situation. We present the same information in relation to the financial incentive that was offered for participation in the research study and determine whether the percentage of participants endorsing each phrase varied by the magnitude of the payment received.

We next explored the relationships between the primary reason for participation, the response to compensation for participation, and the response to each of the non-research scenarios. We also calculated an overall perceived coercion score for each individual by summing the number of times that the phrase endorsed for the non-research scenarios was labeled as coercive. The maximum value for this collective score is 6, as there are 6 non-research scenarios. We then calculated the correlation of this overall score with the binary variable for whether the financial incentive was viewed as coercive. We calculated descriptive statistics for whether participants said that they would have participated without compensation and what minimum dollar amount was needed for those who said that they would not participate. Finally, we analyzed questions pertaining to the perceived fairness of the compensation.

3. Results

Two hundred and sixty-six individuals provided consent and were enrolled in the pilot study after the FICA was introduced into the host study. Only 152 individuals returned for the follow-up interview, which included completion of the FICA; thus, the results presented here only include 152 participants. There was a significant association between type of incentive offered to a participant and the likelihood that the follow-up interview was completed (χ² = 16.94, p=0.018). For any given magnitude of incentive, returning for follow-up was more likely for those individuals receiving a cash incentive (as opposed to a gift certificate incentive), and for both modes of incentive, follow-up increased with the level of payment. At the extremes, 75.0% of those offered a $160 cash incentive returned for follow-up, compared with 35.5% of participants offered a $70 gift certificate. There was no difference in demographics (i.e., age, gender, marital status, race, ethnicity, education, and income) between those participants who did and did not return for follow-up.

The majority of this pilot study sample is male (58%), African-American (63%), non-Hispanic (93%), never married (72%), and relatively young, with an average age of 38.1 years (SD = 9.7) (see Table 2). Educational attainment is generally low, with a little over half of the population not having a high school degree. Most (71%) are not working or are working odd jobs. With the majority of the sample unemployed or out of the labor force, average personal annual income is correspondingly low.

Table 2.

Demographics of the study sample (N = 152)

Demographic characteristics	# (%)
Female	64 (42.1)
Age	38.1 (9.7)*
Ever married	42 (27.6)
Race
White	47 (30.9)
African-American	96 (63.2)
Other	9 (5.9)
Hispanic	11 (7.2)
Education
Less than high school	78 (51.3)
High school degree	58 (38.2)
More than high school	16 (10.5)
Employment
Full- or part-time job	44 (29.0)
Unemployed or odd jobs only	108 (71.0)
Personal Annual Income	4832.9 (7712.7)*
Treatment history
New to treatment	72 (47.4)
Previous treatment	80 (52.6)

^*Means and (standard deviations)

The most common primary reason for participating in the host research study was for the money (30%), followed by curiosity about the project (21%) (Table 3). This is not a surprising result for individuals with limited financial resources. It is interesting to note, however, that the amount of the incentive payment that the participants received was not significantly associated with the main reason reported for joining the study.

Table 3.

Main and secondary reasons for participating in the study

	Main reason for participating					Other reasons endorsed** # (%)
Reasons	All participants # (%)	By payment amount*
		$70	$100	$130	$160
For the money	46 (30.3)	5	10	10	21	47 (30.9)
I wanted to help science	15 (9.9)	4	5	3	3	42 (27.6)
I was bored	5 (3.3)	1	1	1	2	9 (5.9)
I felt obligated to the person asking	8 (5.3)	1	4	2	1	12 (7.9)
I was curious about the project	32 (21.1)	4	10	9	9	59 (38.8)
Peer pressure	0 (0)	0	0	0	0	8 (5.3)
I wanted to help the investigative team	22 (14.5)	6	3	8	5	30 (19.7)
I have a personal interest in research	21 (13.8)	5	4	6	6	36 (23.7)
Other	3 (2.0)	1	0	1	1	= =
Total	152	27	37	40	48

^*No significant differences in main reason selected by participant’s payment amount; chi²= 16.90, p=0.717.

^**Other reasons endorsed does not sum to 100% as participants could select more than one other reason.

Table 4 shows the phrases that were endorsed for each of the non-research scenarios. For the two scenarios with the highest amount of money offered, and for another scenario with high payment and a routine task, the phrase most often selected was the feeling that it was too good of an offer to turn down. The three phrases that were judged by the investigators to be indicative of coercion were rarely selected. Specifically, the number (%) of times a coercive phrase was chosen ranged from 7 (4.6%) to 19 (12.5%).

Table 4.

“Perceived Coercion” phrases endorsed for each of the non-research scenarios

	Feed cat	Coupon	Pick up	Pick up	Ad banner	Paint curb
	$1000	$5	mail $100	trash $5	$1000	$100
I felt like I had no choice but to agree so	4 (2.6)	5 (3.3)	8 (5.3)	3 (2.0)	5 (3.3)	8 (5.3)
that I could get the moneyc I felt a great deal of pressure to agree	0 (0)	1 (0.7)	2 (1.3)	7 (4.6)	6 (3.9)	8 (5.3)
so that I could get the moneyc I felt like I had no choice but to agreec	5 (3.3)	1 (0.7)	5 (3.3)	9 (5.9)	2 (11.3)	3 (2.0)
I felt it was too good of an offer to turn	73 (48.0)	21 (13.8)	61 (40.1)	12 (7.9)	63 (41.5)	12 (7.9)
down I felt it was a tempting offer	19 (12.5)	22 (14.5)	27 (17.8)	10 (6.6)	26 (17.1)	14 (9.2)
I felt very influenced by the money	20 (13.2)	5 (3.3)	5 (3.3)	3 (2.0)	15 (9.9)	3 (2.0)
offered I did not feel pressured to accept this	31 (20.4)	97 (63.8)	44 (29.0)	108 (71.1)	35 (23.0)	104 (68.4)
offer Coercive phrase	9 (8.6)	7 (4.6)	15 (9.9)	19 (12.5)	13 (8.6)	18 (13.2)
Not coercive phrase	143 (91.4)	145 (95.4)	137 (90.1)	133 (87.5)	139 (91.4)	132 (86.8)

^cIndicates those phrases that express the perception of coercion

Calculating the total number of times each participant selected a phrase indicative of coercion, we found that the overall coercion score was low, with 98 individuals having an overall coercion score of 0, 34 having a score of 1, 12 having a score of 2, and 8 having a score of 3.

When directly asked about the payment they received for participating in the host study (Table 5), the largest percentage of individuals (46%) indicated that it was too good of an offer to turn down, followed by feeling no pressure to accept the offer (28%). More importantly, less than 5% of the sample chose phrases indicative of coercion. There were no significant differences either in the percentage of individuals at each payment level endorsing each phrase (χ² = 16.67; p = 0.546) or in the aggregated coercion/non-coercion endorsement (χ² = 5.70; p = 0.127).

Table 5.

“Perceived Coercion” phrases endorsed for the financial incentive for participation in the study

	All participants	By payment amount*
		$70	$100	$130	$160
I felt like I had no choice but to agree so	2 (1.3)	0	1	0	1
that I could get the money I felt a great deal of pressure to agree so	3 (2.0)	0	0	0	3
that I could get the moneyc I felt like I had no choice but to agreec	2 (1.3)	0	0	1	1
I felt it was too good of an offer to turn	64 (42 .1)	10	16	16	22
down I felt it was a tempting offer	24 (15.8)	5	8	7	4
I felt very influenced by the money offered I did not feel pressured to accept this offer	15 (9.9) 42 (27.6)	1 11	3 9	5 11	5 11
Coercive phrase	7 (4.6)	0	1	1	5
Not coercive phrase	145 (95.4)	27	36	39	43

^cIndicates those phrases that express the perception of coercion.

^*There were no differences in the percentage of individuals at each different payment amount endorsing each phrase (chi²=16.67, p=0.546) or in the aggregated coercion/no coercion phrase endorsement (chi²=5.70, p=0.127).

We calculated the correlation between the individuals’ perception of the study payment amount and the individuals’ perception of the non-research scenarios (Table 6). The correlation coefficients were significantly different from zero for the scenarios of feeding the neighbor’s cat (ρ = 0.21; p = 0.009) and picking up trash (ρ = 0.20; p = 0.013). The correlation of the endorsed phrase for study payment and the measure of tendency to perceive coercion in the offer of payments, as measured by the overall non-research score, was nearly significant (ρ = 0.16; p = 0.055).

Table 6.

Correlation of selection of non-research perceived coercion phrase with research perceived coercion phrase

Non-research scenario	Correlation coefficient, ρ (p value)
Feed cat, $1000	0.210 (p=0.009)
Coupon, $5	−0.048 (p=0.555)
Pick up mail, $100	0.033 (p=0.691)
Pick up trash, $5	0.202 (p=0.013)
Ad banner, $1000	0.045 (p=0.582)
Paint curb, $100	0.012 (p=0.885)
Overall non-research score	0.156 (p=0.055)

Only 29 subjects stated that they would not have participated without the incentive payment. Of those, 26 reported a minimum amount that would be necessary for participation. This amount ranged from $20 to $300, with a mean of $62.3 (SD = 60.0). This range, however, is a bit deceptive and inconsistent with revealed behavior as the minimum payment reported by one subject ($300) is actually greater than the amount received. Approximately 16% of the subjects (25 individuals) agreed with the statement that the financial payment they received was too low to be fair, whereas 34% of the subjects (51 individuals) agreed with the statement that they participated in the study because they needed the money even though the amount was small. Somewhat surprisingly, in neither case was agreement with these statements correlated with the size of the payment received (χ² = 1.39 (p = 0.707) and χ² = 1.08 (p = 0.782), respectively).

4. Discussion

The primary aim of this pilot study was to obtain preliminary data from the FICA, a brief questionnaire measuring the extent of perceived coercion when using financial incentives to recruit research participants. In the parent study, results from a modified version of the MAES showed a very low rate of perceived coercion among subjects. The finding was replicated with data from the FICA, thereby showing consistency of the FICA with a previously established measure.

Even though results were similar for the MAES and the FICA, we believe that the approach taken by the FICA (i.e., using a continuum of coercive phrases rather than a binary response) more closely reflects how individuals view offers or situations that might be perceived as potentially coercive. In addition, we found with the FICA that individuals’ average perceptions of coercion in non-research situations were related to their level of perceived coercion for actual financial payments for participation in research (although not quite reaching statistical significance). Our initial assessment of the FICA questionnaire suggests that it is a useful research tool to measure general perceived coercion and it has value in predicting overall reactions to research payments. If future research confirms this finding and provides evidence of the validity of the questionnaire, new research projects may benefit from screening participants for general perceived coercion prior to recruitment or payment offers and potentially counseling those who express a higher tendency to perceive coercion from financial incentives.

Logistically, there are several ways in which this questionnaire might be used in practice. First, if future research establishes an association between individuals’ general level of perceived coercion (as measured on the non-research items) and incentive-related perceived coercion, then a researcher could assess prior to enrollment the likelihood that a participant would feel coerced. Special attention could then be given to ensure that the individual understood that participation was voluntary. Alternatively, the FICA could be administered after enrollment and, if a high level of perceived coercion is found, researchers could initiate further discussion with the participant concerning voluntariness.

There are several limitations to the current study. First, participants were not blinded to the alternative magnitudes and mode of incentive. Because of the treatment center setting, participants would have quickly learned whether they were being treated differently from other participants. Rather than have them learn about the incentive arrangement independently and possibly feel resentful or cheated, we elected to inform them of the incentive conditions upfront. However, no participant in previous studies that have been conducted by the investigators in the current study, where participants were also not blinded to incentive levels, has expressed dissatisfaction with their comparative level or type of payment.

A second potential limitation is the generalizability of which phrases are viewed as indicating coercion. Although we used both an expert panel and a diverse group of individuals to make this determination during development of the FICA, a different cohort could potentially reach a different conclusion. A further limitation is the fact that the results are based on a single parent study and thus a very specific demographic group. Thus, the generalizability of these results cannot be established. Future research, however, will assess the use of a revised FICA in a broader array of studies and diverse populations. Although the findings presented here are preliminary, they suggest that the FICA can be useful in a variety of research settings.

In summary, these preliminary results from the FICA are encouraging yet somewhat limited. We plan to use the information collected in this study, along with informal administrator/subject feedback, to refine and revise the FICA. In particular, the non-research scenarios will be modified slightly, and some of the response categories will be further clarified. We will then retest the instrument in new settings with larger sample sizes. To rigorously address and resolve the debate about whether financial payments to research subjects are coercive or unduly influential, we need to have a valid and reliable instrument for measuring perceived coercion. We believe that the FICA possesses promise in this regard.