The backfiring effects of monetary and gift incentives on Covid-19 vaccination intentions

Abstract

We provide evidence that material inducements for Covid-19 vaccination may backfire. Results from a hypothetical survey experiment in China (N = 1365) show incentives of 8–125 USD reduce vaccine uptake intentions compared to simply offering vaccination for free. Ours is the first Covid-19 vaccine study to separately consider and directly compare the effects of monetary and goods-based incentives, both of which have been widely employed by countries seeking to increase uptake; we demonstrate that both types backfire equally. Results are compared against the burgeoning literature on Covid-19 vaccine incentives.

1. Introduction

The Covid-19 pandemic has caused extraordinary damage and disruption to societies and economies around the world, and infection rates remain a global problem as the disease transitions into an endemic phase. Governments have used vaccines as a primary tool to combat Covid, aiming to persuade high proportions of their populations to receive jabs. With the prospect of increasingly protective vaccines being developed in future, and a regularly mutating virus, high vaccination coverage is likely to remain an important policy goal. However, while vaccines have been provided free of charge in most countries, vaccination programs have often been met by strong resistance among some societal segments, resulting in difficulties in achieving target rates.

In order to accelerate vaccination, countries have offered various incentives with the hope of motivating people's vaccination willingness. For example, the American government has encouraged businesses to offer many creative incentives, including free doughnuts, basketball tickets, and gift cards to people who get vaccinated (The White House, 2021). Many American companies have directly reimbursed vaccinated employees with money, ranging from $25 to $750 (Krouse, 2021). In an Indian city, women who got vaccinated were given gold nose pins, and men were given hand blenders (Anon, 2021). In China, people have received various rewards such as eggs, rice, and flour for vaccine uptake (Meng, Shan, & Zhang, 2021). Cash-formed incentives also exist - for example, in parts of Shanghai, people were offered a direct 200 RMB transfer (Zhang, 2021). In general, gifts and cash are both widely used to encourage vaccination uptake. The question then arises as to whether or not these strategies are effective in raising vaccine uptake willingness.

In this paper, we report a hypothetical survey-based online experiment examining the impact of providing incentives on the willingness toward Covid-19 vaccination in China. This experiment is designed to address whether material inducements, at various different levels of value, can succeed in making citizens more prepared to accept vaccination – or, paradoxically, whether they may instead backfire and lower vaccine willingness, in the same vein that prior literature has sometimes found material incentives to negatively motivate desired behavior. A key contribution of our paper is that we are, to the best of our knowledge, the first to test the effects for Covid-19 vaccination of both monetary incentives and those based on goods, and directly compare the impact of each.

In our study, we presented subjects with one of seven hypothetical scenarios, in which a new and effective Covid-19 vaccination is being distributed among the Chinese population. Across treatments, we varied the availability, type and value of incentives for those agreeing to take the vaccine. In the control treatment, the vaccine would simply be offered for free. In the other treatments, a payment of either money or goods would be provided for vaccine recipients, which we varied in value across three levels ranging from 50 to 800 RMB (approximately 7.85 to 125.90 USD). Our outcome variable is subjects' stated willingness to receive the vaccine.

We find a striking tendency for the incentives to backfire. Even at the highest incentive level, vaccination willingness is lower than under the control treatment, suggesting that far from supporting policymakers' aims of expanding vaccination coverage, the use of low-to-medium sized incentives in China is in fact likely to be counterproductive. A possible reason for this, which we will discuss, is that offering payment for taking a vaccine may be regarded by the population as a negative signal about its safety or quality. We find only a weak tendency for the effects of incentives to become less negative as a result of increasing the value of the incentives. On the basis of previous literature showing stronger motivational effects of gifts than of monetary payments of the equivalent value (Heyman & Ariely, 2004; Kube, Maréchal, & Puppe, 2012; Lacetera & Macis, 2010), we hypothesized that our gift treatments would outperform those employing money. However, here too we provide disappointing news for policymakers: the effects of non-monetary incentives did not significantly improve upon those of purely financial rewards.

Our study contributes to the rapidly growing literature on policy interventions aimed at increasing Covid-19 vaccination rates. In Section 2, we provide a review of the existing literature investigating the effects of monetary incentives for Covid-19 vaccination. This will provide a background against which our results can be compared. Weighed against the bulk of this literature, our results appear surprising. Many of the studies we review have found positive effects even of relatively small incentives (e.g. Campos-Mercade et al., 2021; Klüver, Hartmann, Humphreys, Geissler, & Giesecke, 2021). However, others have found null effects of cash incentives or lotteries (e.g. Kreps, Dasgupta, Brownstein, Hswen, & Kriner, 2021; Lang, Esbenshade, & Willer, 2021), while others still have found negative effects (e.g. Serra-Garcia & Szech, 2022). The substantial heterogeneity in the findings in this literature – which casts doubt on whether incentives necessarily represent an optimal policy option – provides motivation for our study focused on the Chinese population, particularly as a plausible reason for some of the heterogeneity is that results may be influenced by the cultural or institutional contexts of the countries in which the research is conducted. We note that most of these studies have been run in western countries, while the one with the most similar result to ours was done in Central Asia (Seitz, Yamada, & Shimizutani, 2023). Ours is the only study in this literature to have been run in Mainland China.

Skeptical readers might query our apparently surprising results on the basis that they are derived from hypothetical responses, i.e. stated preferences for vaccination uptake, rather than revealed preferences uncovered by real vaccination behavior. In this regard, as demonstrated in Section 2, our study is typical of the vast majority of existing research on Covid-19 vaccination incentives and other types of intervention (see the review by Batteux, Mills, Jones, Symons, & Weston, 2022). Field experiments offering genuine material rewards for taking vaccines are much more logistically challenging and are likely to take longer to emerge; only a handful have appeared so far in the context of Covid-19 (Campos-Mercade et al., 2021; Jacobson, Chang, Shah, Pramanik, & Shah, 2022; Mehmood, Naseer, & Chen, 2022; Duch et al., 2023). These studies have, unlike ours, found either positive or null effects of incentives – although these contrasting results could of course be due to other differences between our studies (e.g. research location) beside the use of real or hypothetical incentives. While there is some evidence that the effects of incentives on Covid-19 vaccination behavior may scale up from hypothetical to real decisions (Campos-Mercade et al., 2021), we acknowledge the limitations of hypothetical studies in general, and the understandable priority economists typically give to revealed preference data. However, when this is scarce or non-existent we argue that there is great value for policymakers provided by stated preference research. Covid-19 remains a new, and acutely pressing, problem facing humanity; at this stage where useful information remains both limited and greatly coveted by policymakers, our research provides the only publicly available experimental evidence on the effects of material incentives for Covid-19 vaccination that was elicited among the Chinese population. We leave it up to Chinese policymakers to judge whether evidence of negative effects from a hypothetical experiment in China provides better guidance of the likely impact of incentives on real vaccination rates in this country than evidence of null or positive effects drawn from studies in other countries observing actual behavior.

The structure of the rest of this paper is organized as follows: Section 2 reviews related literature; Section 3 describes the experimental setup and methods. Hypotheses are set out in Section 4. Results and analysis are presented in Section 5, followed by a related discussion and conclusion in Section 6.

2. Literature review

Monetary incentives are common in health contexts, and it has been proven they can improve individuals' health-related behaviors. For example, prior studies have demonstrated that financial incentives can encourage people to improve diet (Gardiner & Bryan, 2017), increase health monitoring (Sen et al., 2014) and physical activity frequency (Patel et al., 2018), and quit smoking (Volpp et al., 2009). Prior to Covid-19, monetary inducements have been used to encourage people to vaccinate themselves against other diseases, and these interventions have sometimes been successful (e.g. Banerjee, Duflo, Glennerster, & Kothari, 2010; Mantzari, Vogt, & Marteau, 2015; Sato & Fintan, 2020; Tressler & Bhandari, 2019).

Nevertheless, there is an ongoing debate on when monetary incentives do or do not succeed in increasing desired behavior (Gneezy, Meier, & Rey-Biel, 2011; Kamenica, 2012). Researchers have found, across a wide range of contexts, that incentives can backfire and even decrease individuals' work efforts and prosocial behaviors. In one famous example, for instance, Gneezy and Rustichini (2000) found individuals were less motivated to raise charitable funds when paid a small amount than when not paid at all. Such backfiring effects may result from material payoffs crowding out intrinsic motivation. Of perhaps even greater relevance to our context, the offer of compensation may be taken as a signal that an act is unpleasant or dangerous (Benabou & Tirole, 2003; Frey & Oberholzer-Gee, 1997).

The stage was thus set for a contentious debate to emerge, as soon as vaccinations had presented themselves as a potential escape route from the pandemic, over the role that material incentives could and should play in advancing vaccine uptake. This debate played out in media, policy and academic circles from the first year of the pandemic, with scholarly contributions both arguing in favor of and warning against the use of incentives (e.g. Kim, 2021; Mankiw, 2020; Volpp, Loewenstein, & Buttenheim, 2021). One concern is that, unlike certain other health behaviors, becoming vaccinated may not only protect oneself but also others1 ; this prosocial dimension to the act may render it more reliant on intrinsic motivation and therefore more susceptible to the backfiring effects of any intervention liable to crowd this out. An even greater worry, perhaps, is that the offer of payment might be interpreted, especially by citizens who are already vaccine-hesitant, as a signal that the vaccine being distributed is unsafe or causes unpleasant side-effects. Similar effects have been found in other medical contexts: Cryder, London, Volpp, and Loewenstein (2010) found subjects believed involvement in health research to be riskier the more money they were offered for their participation.2

With a view to informing this debate, there has already emerged a rapidly growing body of empirical research exploring the effects of monetary incentives on Covid-19 vaccination. Overall, this literature suggests financial inducements can successfully encourage vaccination, but in many contexts appear to be ineffective or even counterproductive.3 A large portion of the research has used experiments to estimate the effects of hypothetical (or occasionally real) monetary incentives on stated willingness to be vaccinated (or actual vaccination rate). The findings of these studies are summarized in Table 1. 4 The table reports for each experiment whether there is evidence for a backfiring effect (i.e. a significant negative effect on vaccination willingness/uptake resulting from the presence of incentives), and an estimated range on the level of incentives required to induce any significant positive effect. The latter is constructed as the range between the lowest level of incentives found in the study to produce a significant positive effect and the highest level found to produce a null effect, if there was one. We only include studies which specified the fixed value of the incentives in their treatments and reported sufficient data to identify which incentive levels produced significant results.

Table 1.

Summary of experimental literature on effects of financial incentives on Covid-19 vaccination.

Study	Date of research	Population	Stakes	Backfiring effect found?	Minimum incentives required for positive effect
Kreps et al. (2021)	October 2020	United States	Hypothetical	No (lowest incentive tested = $10)	Positive effect not found (highest incentive tested = $100)
Robertson et al. (2021)	December 2020	United States	Hypothetical	No (lowest incentive tested = $1000)	≤$1000
Serra-Garcia & Szech (2022)	December 2020 – February 2021	United States	Hypothetical	Yes, for incentives ≤ $20	$50–100
Klüver et al. (2021)	March 2021	Germany	Hypothetical	No (lowest incentive tested = $29.4)	≤$29.4
Sprengholz, Henkel, and Betsch (2022)	April 2021	Germany	Hypothetical	No (lowest incentive tested = $294)	$3531-3825
Mehmood et al. (2022)	April 2021	Pakistan	Real	No (lowest incentive tested = $7.5)	Positive effect not found (highest incentive tested = $15)
Seitz et al. (2023)	May–June 2021	Tajikistan, Uzbekistan, Kazakhstan	Hypothetical	Yes, for incentives ≤ $46.62	Positive effect not found (highest incentive tested = $46.62)
Jacobson et al. (2022)	May–July 2021	United States	Real	No (lowest incentive tested = $10)	Positive effect not found (highest incentive tested = $50)
Campos-Mercade et al. (2021)	May–July 2021	Sweden	Real	No (lowest incentive tested = $24)	≤$24
Morbée et al. (2022)	June 2021	Belgium	Hypothetical	No (lowest incentive tested = $61)	Positive effect not found (highest incentive tested = $61)
Fishman, Salmon, Scheitrum, Schaefer, & Robertson (2022)	September 2021	United States	Hypothetical	No (lowest incentive tested = $100)	≤$100
Stamm et al. (2022)	October 2021	Austria	Hypothetical	No (lowest incentive tested = $115.7)	≤$115.7
George et al. (2022)	November–December 2021	South Africa	Hypothetical	No (lowest incentive tested = $3.3)	$3.3–23.3
Fishman, Mandell, Salmon, and Candon (2023)	2021 (month not reported)	United States	Hypothetical	No (lowest incentive tested = $200)	≤$200
Duch et al. (2023)	February 2022	Ghana	Real	No (lowest incentive tested = $3)	≤$3
Stamm et al. (2022)	July–August 2022	Austria, Italy	Hypothetical	No (lowest incentive tested = $505)	≤$505

Notes for Table 1: The dependent variable is stated willingness to be vaccinated for studies with hypothetical stakes; vaccination rate for studies with real stakes. Backfiring is defined as a significant decrease in the dependent variable resulting from monetary incentives; a positive effect is defined as a significant increase in the dependent variable resulting from them. The currency is USD; for studies reporting monetary amounts only in local currencies, we converted these into USD using the exchange rate at the time the research was conducted. The table only includes results based on experimental methods, from studies reporting sufficient data for it to be possible to calculate ranges on the maximum/minimum required incentives for backfiring/positive effects to occur.

As the table shows, results are mixed. Only two studies find significant backfiring effects, at relatively low incentive levels (Seitz et al., 2023; Serra-Garcia & Szech, 2022). Some of the studies only test the effects of much larger incentives and therefore do not shed light on whether backfiring would occur at lower levels. Other experiments, which do test incentives at lower levels, find these have either insignificant effects (sometimes in a negative direction) or even significantly positive ones. Experiments testing much larger incentives tend to find positive effects, though in some cases there are null effects up until rather high levels. Overall, there is certainly not consistent evidence that we should necessarily expect positive effects of small to medium incentives, with several studies producing only null results (Jacobson et al. (2022); Kreps et al., 2021; Mehmood et al., 2022; Morbée et al., 2022). The minimum amount needed to induce a significantly positive effect ranges from less than or equal to 3 USD (Duch et al., 2023) to >3500 USD (Sprengholz et al., 2022). Robertson, Scheitrum, et al. (2021) only tested incentives in the range above 1000 USD, finding these generated a strong increase in vaccine willingness of around 8 percentage points, suggesting positive effects could also be created by much smaller incentives. Differences between studies in the required thresholds for significant positive or negative effects may well be influenced by differences in the populations investigated, the time the research was conducted, and methodological features including sample sizes.5

In addition to the experimental literature, a number of studies have attempted to assess the effects of incentives using real-world vaccination data. This method has the advantage of observing real behavioral outcomes, while it also faces the drawback that confounding variables inevitably make it more difficult to cleanly identify and/or quantity causal effects. Among these studies, many have focused on the effects of lottery incentives offered in some US states for vaccination (Acharya & Dhakal, 2021; Barber & West, 2022; Brehm, Brehm, & Saavedra, 2022; Cohn, Chimowitz, Long, Varma, & Chokshi, 2022; Dave, Friedson, Hansen, & Sabia, 2021; Grossi, 2022; Guo, Gao, & Sims, 2022; Lang et al., 2021; Law, Peterson, Walkey, & Bosch, 2022; Mallow, Enis, Wackler, & Hooker, 2022; Robertson, Schaefer, & Scheitrum, 2021; Sehgal, 2021; Sload, Bechtolsheim, & Gifford, 2022; Thirumurthy, Milkman, Volpp, Buttenheim, & Pope, 2022; Walkey, Law, & Bosch, 2021; Wang, Hernandez and Stoecker, 2022); the results of these are largely split between null and significantly positive effects, with some suggestion that patterns may differ across states. Others have addressed the effects of guaranteed payments for vaccinations, with Wong et al. (2022) finding a 25 USD incentive increased vaccination rates in North Carolina, and Chetty-Makkan et al. (2022) identifying a positive effect of offering older adults in South Africa a shop coupon worth 7 USD. Erdem, Erdem, and Monson (2022) found a positive effect of offering $50 gift cards for children to be vaccinated.

The literature has generally focused on the short-term consequences of monetary incentives. A potential concern, however, is that negative effects could emerge in the longer term. For instance, compensation for receiving a first Covid vaccination jab might reduce a person's subsequent willingness to obtain a second or third jab. One recent study to directly address this was Schneider et al. (2023), who found incentives neither exerted negative consequences on uptake rates for later doses, nor did they reduce long-term trust in vaccines or negatively influence Covid-unrelated health behaviors.

As noted in our introductory section, the real-world incentives implemented for Covid-19 vaccinations have not been limited to money but have often involved goods. It is therefore surprising that, at the time of writing (January 2023) there is scant research within this rapidly developing literature studying the effects of non-monetary gift incentives, or how they compare against those of monetary payments. A few of the aforementioned studies have investigated the effects of vouchers (Chetty-Makkan et al., 2022; Erdem et al., 2022; Morbée et al., 2022; Stamm, Partheymueller, et al., 2022), with Stamm, Partheymueller, et al. (2022) including a comparison of cash and coupon incentives, which found no significant difference in their effects.6 However, vouchers may lie in a psychological category rather closer to money than the type of physical goods we feature as gifts in our experiment. To the best of our knowledge, there are no previous studies in the Covid vaccination literature studying the use of physical goods as incentives.

Previous literature provides justification for comparing the effectiveness of monetary and non-monetary incentives, since evidence shows that compensations of the same nominal value but in different forms can elicit different reactions (Gilchrist, Luca, & Malhotra, 2016; Heyman & Ariely, 2004; Kube et al., 2012). The main argument of these studies is that in-kind compensations aimed at improving behaviors lead to better positive responses compared to directly incentivizing behavior with cash (especially for small values of cash). This is despite the fact that, from a traditional economics perspective, money should always be weakly preferred to equivalently priced gifts, since the money can be exchanged for whatever one wants (Waldfogel, 1993). Notably, Kube et al. (2012) found that non-monetary gifts have a much stronger impact in triggering workers' reciprocity, in the form of work effort, than monetary gifts of equivalent value.7 It therefore seems plausible that through similar effects on citizens' motivation to behave reciprocally toward policymakers, gift incentives for Covid-19 vaccine uptake may work more effectively than money. Our study aims to fill this gap in the literature.

Another feature of the existing literature on incentives and Covid-19 vaccination is that the large majority of research has been conducted in western countries. A few studies have been conducted in other parts of the world, but to the best of our knowledge our paper is the first to examine this topic in Mainland China.8

3. Experimental design and procedures

We conducted an online, survey-based experiment from March 8–22, 2022, with respondents recruited via the panel service provided by the Chinese survey platform Wenjuanxing.9 During the survey, we asked respondents to imagine a new, hypothetical Covid-19 vaccination had been made available and was more effective than those distributed in China to date. We took this approach, rather than basing our study on existing Covid-19 vaccines, because by the time of the experiment many people in China were already vaccinated and asking them about their willingness to receive existing vaccines might create confusion.

Respondents' willingness to take the hypothetical vaccine was elicited under three different incentive type conditions: Money, Gift, and Control. In the Control treatment, no vaccination incentive was mentioned. Under Money it was mentioned that those opting for vaccination would receive money, while under Gift it was stated that they would receive a gift. This gift took the form of a daily necessity (rice), which reflects the type of item that has actually been offered in China for Covid-19 vaccination (Meng et al., 2021). In both conditions, the benefit that the recipient would obtain was described as being “as thanks” for accepting vaccination.

Both Money and Gift contained 3 sub-conditions, varying in the value of the incentive: 50, 200, or 800 RMB. We label these treatments M50, M200 and M800 for Money and G50, G200 and G800 for Gift. The middle incentive level of 200 RMB was chosen to be in line with what had been implemented in Shanghai (Zhang, 2021). At the time of the experiment, the three incentive levels equated to about 7.85, 31.50 and 125.90 USD respectively.

In order to control participants' perceptions about the value of the gift, we provided a picture of the rice and set a price tag next to it which explicitly marked its market price (see Fig. 1 ).10 Thus, the perception of the monetary value of the incentive should be held constant between, for instance, subjects in the M50 and G50 treatments (although, as alluded to in the previous section, under standard economic theory individuals may tend to prefer the money).11

Fig. 1.

Example of picture of gift shown to subjects.

Each respondent was randomly assigned to one of the seven treatments with essentially equal probability.12 In each treatment, they were asked to indicate their willingness to receive the vaccine on a scale from 0 to 100, where 100 meant “completely willing to accept” and 0 “completely unwilling to accept”. The full wording of the questions is presented in the Online Appendix.

In order to negate any effects of the transportation cost of carrying the gifts, the wording specified that they would be delivered free of charge to vaccine recipients' homes within two working days. For consistency, the waiting time associated with the cash payment was the same; recipients were told the money would be sent to their WeChat account within two working days.

At the end of the questionnaire, we collected information about gender, age, monthly income, and hometown. Several questions were also introduced to elicit participants' opinions on Covid-19, including whether they trusted existing vaccines, how anxious they felt about the disease, and whether they knew anyone closely who had been infected with it. Respondents were assured that responses to all parts of the survey were anonymous.

3.1. Sample characteristics

A total of 1365 respondents participated in the experiment. Subjects were drawn from across China. 50.3% of the sample was female. Participants were roughly evenly distributed between the following five age categories: 18–25, 26–30, 31–40, 41–45, and 46 and above.13 Low-income (<5000 RMB per month), medium-income (5000-8000RMB per month), and relatively high-income individuals (8000-15000RMB per month) each represented about 30% of the sample, with the remaining 10% made up of respondents whose monthly income exceeded 15,000 RMB. 7% of the participants said they knew someone closely who had been infected with the coronavirus,14 and 65.1% of respondents reported having a high or moderate degree of concern about contracting it.

In Table B1 (Online Appendix B) we report checks confirming that the sub-samples assigned to the different treatment groups in the experiment were similar in terms of observable characteristics. This is done by regressing these characteristics against treatment dummy variables. Of the 36 coefficients estimated on the treatment variables, only two reach statistical significance at the 10% level or below. From this absence of significant differences in the types of subjects assigned to each treatment, we conclude that the random assignment to treatment was successful.

4. Hypotheses

Theory and previous literature offer conflicting insights on the effects the incentives may have. From a traditional economic perspective, receiving either monetary or gift compensation would raise the utility levels associated with taking the vaccine, and should therefore increase demand. Moreover, the greater the value of the incentives provided, the greater this increase in demand should be. The latter prediction is consistent with the evidence of several of the studies reported in Table 1, which found the positive effects of incentives for Covid-19 vaccinations were stronger when the amounts offered were higher. The same has been found for other types of prosocial behavior, such as blood donation (Lacetera, Macis, & Slonim, 2014). On this basis, we offer the following hypotheses:

Hypothesis 1a

Incentives increase willingness to receive Covid-19 vaccination.

Hypothesis 1b

Willingness to receive Covid-19 vaccination increases in the size of the incentives offered.

On the other hand, as mentioned earlier, previous literature – including in the context of Covid-19 vaccination (e.g. Serra-Garcia & Szech, 2022) – raises the possibility that incentives may backfire, especially at low levels. Offers of either money or gifts may crowd out individuals' intrinsic motivation to become vaccinated, or be interpreted as a signal of the vaccine's undesirability, leading to a negative impact on vaccination willingness relative to simply offering it for free. We therefore consider, in opposition to Hypothesis 1a, the following additional hypothesis:

Hypothesis 2

Incentives reduce willingness to receive Covid-19 vaccination.

As stated above, gifts have been shown to operate as better incentives than money in some contexts (Heyman & Ariely, 2004; Kube et al., 2012). Gift-based compensation might frame a transaction as a social interaction and produce positive psychological effects, whereas a direct cash transfer may instead be perceived as a pure market interaction. It is possible that gifts could therefore trigger a stronger willingness to adhere to a policymaker's wishes by becoming vaccinated. We propose the following hypothesis:

Hypothesis 3

Gifts are more effective than equivalent sized monetary incentives in raising vaccination willingness.

5. Results

Fig. 2 presents the willingness to receive the vaccine for each treatment. As shown on the vertical axis, without any incentives, the average willingness toward vaccine uptake in the Control treatment is 85.42. It is clear that on average, there was no increase in vaccine uptake willingness when incentives were provided. On the contrary, it reduced in all treatments relative to the baseline. Two-tailed t-tests found the differences between each money or gift treatment and the control treatment to differ at the 5% significance level or lower, except for the M200 treatment in which vaccination willingness was not significantly different from in the Control. Hypothesis 1a is, therefore, never supported, while Hypothesis 2 is supported for all gift treatments and all bar one monetary incentive treatment. Overall, the results suggest that, far from helping to increase vaccination willingness, offering incentives at best has no effect and in most cases significantly backfires.

Result 1

Incentives are either ineffective or reduce willingness to take the vaccine.

Fig. 2.

Vaccination willingness by treatment.

Note for Fig. 2: The figure displays mean vaccination willingness, with 95% confidence intervals.

From Fig. 2, we can see a mild increase in average vaccine uptake willingness as the magnitude of the incentives rises, for both money and gifts. In order to assess Hypothesis 1b, we run pairwise comparisons testing for significant differences across incentive levels, holding the incentive type constant: M50 vs M200; M50 vs M800; M200 vs M800; G50 vs G200; G50 vs G800; G200 vs G800. Two-tailed t-tests do not generate significant results on any of these tests, except for M50 vs M200. There is, therefore, only limited support for our hypothesis that vaccination willingness increases in the level of incentives offered. We note that the decrease in willingness when offered 800 RMB relative to 200 RMB is counterintuitive, but as this is not significant it may be a mere statistical error. Overall, the results provide some weak evidence that increasing the size of incentives mitigates their backfiring consequences, but even the highest levels of incentives we tested failed to eliminate them, as evidenced by the significantly negative effects reported above for both the M800 and G800 treatments.

Result 2

Increasing the size of the incentives leads to a mild increase in vaccination willingness for monetary incentives, and does not significantly increase vaccination willingness for gift incentives.

We have shown that gift and money compensations lead to the reduction of vaccine uptake willingness, but do they erode willingness to different extents? To address Hypothesis 3, we ran two-tailed t-tests comparing vaccination willingness under M50 vs G50, M200 vs G200 and M800 vs G800. The results of these were all insignificant. Therefore, unlike in certain other contexts (e.g. Kube et al., 2012), we have not found that gifts work better than monetary inducements toward producing the desired behavior. Hypothesis 3, that gifts would better incentivize vaccination willingness than the equivalent money, is not supported.

Result 3

The effects of monetary incentives do not significantly differ from those of gift incentives of equivalent value.

5.1. Regression analysis

Table 2 presents OLS regression models. The dependent variable is vaccination willingness. Treatment dummy variables are included; demographic control variables are also introduced, while models (2) and (3) control further for subjects' stated trust levels in existing Covid-19 vaccines, degree of worry about being infected, and experience or not of a socially close person having been infected.15 The significance levels of the treatment dummy coefficients corroborate the evidence presented above about the effects of each treatment. In model (1) the difference between the Control and G800 treatments falls in significance to the 10% level, while in model (2) the difference between the Control and M800 treatments does so. Otherwise, levels of significance are the same as identified by the t-tests. The regressions are also useful in quantifying the treatment effect sizes. The largest are for the M50 and G50 treatments, which reduce vaccination willingness by about 9.3 and 8.6 percentage points respectively, according to model (2).

Table 2.

OLS regressions.

	Dependent variable = Vaccination willingness
	(1)	(2)	(3)
Female	−5.055⁎⁎⁎	−5.605⁎⁎⁎	−5.589⁎⁎⁎
	(−3.53)	(−4.01)	(−3.99)
Age	0.688⁎	0.714⁎	0.774⁎
	(1.69)	(1.80)	(1.94)
Age2	−0.006	−0.007	−0.008
	(−1.14)	(−1.43)	(−1.56)
Income	2.76e−6	5.81e−6	−1.69e-4⁎
	(0.06)	(0.14)	(−1.88)
M50	−9.613⁎⁎⁎	−9.285⁎⁎⁎	−6.102
	(−3.62)	(−3.60)	(−1.60)
M50*Income			−4.65e−4
			(−1.45)
M200	−3.006	−2.533	−5.979⁎⁎
	(−1.13)	(−0.98)	(−1.98)
M200*Income			2.78e-4⁎⁎
			(2.18)
M800	−5.478⁎⁎	−4.366⁎	−7.045⁎⁎
	(−2.06)	(−1.69)	(−2.34)
M800*Income			2.18e-4⁎
			(1.66)
G50	−8.634⁎⁎⁎	−8.584⁎⁎⁎	−11.488⁎⁎⁎
	(−3.26)	(−3.33)	(−3.62)
G50*Income			2.48e−4
			(1.42)
G200	−7.163⁎⁎⁎	−6.775⁎⁎⁎	−7.240⁎⁎
	(−2.70)	(−2.63)	(−2.20)
G200*Income			1.86e−5
			(0.10)
G800	−4.883⁎	−5.593⁎⁎	−9.185⁎⁎⁎
	(−1.85)	(−2.17)	(−3.05)
G800*Income			2.18e-4⁎⁎
			(2.26)
Trust Current Vaccine		8.415⁎⁎⁎	8.410⁎⁎⁎
		(8.03)	(8.02)
Worry about Covid		−0.201	−0.206
		(−0.24)	(−0.25)
Know Infected Person		−9.710⁎⁎⁎	−9.497⁎⁎⁎
		(−3.56)	(−3.48)
			(1.66)
Constant	72.119⁎⁎⁎	48.497⁎⁎⁎	49.541⁎⁎⁎
	(9.77)	(5.91)	(5.91)
N	1365	1365	1365
R2	0.030	0.087	0.096

t statistics in parentheses.

^⁎p < 0.1.

^⁎⁎p < 0.05.

^⁎⁎⁎p < 0.01.

Vaccination willingness was lower for females than males, consistent with previous research on Covid-19 (Zintel et al., 2022). Age was, with weak significance, positively associated with vaccination willingness,16 while income was unrelated with it. In model (3), however, we introduce interaction terms between income levels and the treatment dummies, and find some of these to be significant. Specifically, there is some tendency for both the money and gift incentives, at levels of 200 and 800 RMB, to have greater effectiveness on higher income individuals. In other models, which we do not report in the table, we tested for interactions between treatment effects and either gender or age, and found these to be insignificant in all cases.

6. Discussion and conclusion

Our results demonstrate that providing compensations for Covid-19 vaccination can backfire, reducing people's willingness to be vaccinated relative to merely offering the vaccine for free. Our results in this direction are striking, and perhaps surprising, in light of the experimental papers reviewed in Section 2. Only two of them reported significantly negative effects of Covid-19 vaccine incentives – one of which found that this effect, in the United States, was negated once incentives were raised to 50 USD (Serra-Garcia & Szech, 2022). Other studies have found significantly positive effects of incentives even smaller than this (e.g. Campos-Mercade et al., 2021; Klüver et al., 2021). In contrast, our experiment continues to identify a significant backfiring effect even for incentives worth roughly the equivalent of 125 USD. This is higher also than the largest incentives found by Seitz et al. (2023), in Central Asia, to trigger a backfiring effect, although they did not test higher incentive levels and therefore cannot rule out this effect continuing to hold at all levels we studied.

We identify a mild tendency for the incentives to work better (i.e. less badly) when they increase in value. We did not test incentives worth >800 RMB, and therefore cannot rule out that their effects may become positive at much higher levels. Nevertheless, our study offers a possible lesson to policymakers: the use of incentives, at small-to-medium levels, to encourage Covid-19 vaccination among the Chinese population may not be an effective use of resources and may well be counterproductive toward achieving the policy goal of high vaccination rates. We speculate that the negative consequences we identify may stem from signaling effects similar to those considered in earlier literature focusing on possible reasons why incentives may backfire (e.g. Benabou & Tirole, 2003; Cryder et al., 2010); the offer of material compensation for receiving a vaccine may be interpreted as a negative sign about its quality or safety. This effect may be particularly strong in our experiment, where decision-makers lack any information about the new vaccine and are particularly sensitive to whatever they can learn from their available choice set. A supplementary explanation could be that the incentives crowd out intrinsic motivation (Bénabou & Tirole, 2006). Without material inducements individuals may wish to become vaccinated because this is what society promotes, and doing so gives them a positive feeling from performing their civic duty; the introduction of extrinsic incentives to vaccinate oneself may reduce the self-esteem or social-esteem benefits derived from the act.

A further striking result is that the backfiring effects occur not only for monetary incentives but also for non-monetary gifts. The equivalent effectiveness of money and gifts in encouraging Covid-19 vaccination represents a novel research finding. Previous literature has shown gifts can be more successful than money in motivating certain behaviors, but disappointingly we find no evidence that this is the case for Covid-19 vaccination. It is possible that gifts provided by the government may fail to have the same positive motivating effects that gifts from employers have, because the exchange is more impersonal. However, we do encourage further research on the effects of gift incentives for Covid-19 vaccination. Our study only considers one type of gift (rice) and it is possible that others could be more successful. It is also possible that there is a cultural element to our results. In China, there is a relatively strong tradition of giving money as a gift (e.g. at weddings) and using it to cement interpersonal relationships; the distinction between money and non-monetary gifts may therefore be less pronounced than in other cultures.

The main limitation of this study is that we elicited hypothetical vaccination willingness rather than actual behavior. For practical reasons, the vast majority of existing research on Covid-19 vaccination interventions has investigated their effects using hypothetical methods like ours. Would our findings remain unchanged if we had studied real vaccination decisions? Some reassuring evidence comes from the three studies we have encountered to date which measure the effects of monetary incentives on both stated vaccination intentions and subsequent actual behavior, finding similar results for each (Campos-Mercade et al., 2021; Duch et al., 2023; Schneider et al., 2023). This suggests that in this specific literature eliciting stated intentions rather than actual behavior may not substantially damage a study's external validity. However, in those three studies, although the outcome variable was sometimes a stated intention, the offered incentive was always real, while in our study both the outcome variable and the incentive were hypothetical, placing our measurements at a further distance from reality. It is certainly the case in other literatures that treatment effects for hypothetical behaviors do not always match up with those on real choices (e.g. Holt & Laury, 2002). We therefore cannot guarantee that the effects of incentives our study observes on vaccination intentions would hold true if we had studied actual vaccinations. We would certainly welcome any future research which tests whether our findings can be replicated under real scenarios.

We also cannot state with confidence that our findings would be externally valid outside of China. It is possible that the cultural and institutional context of a country affects how citizens respond to vaccine incentives, including any signals they interpret from them on vaccine quality or safety. This could be one reason for the heterogeneity of findings across studies – and, as noted earlier, one of the reasons why running a study in China, rather than merely generalizing from results elsewhere, represented a worthwhile exercise.17 It is notable that the study with results most similar to ours was conducted in a region neighboring China (Seitz et al., 2023). It is possible, for instance, that the negative signaling effects of material compensation are stronger in countries where the baseline levels of education and understanding about vaccines are lower.18 However, we do not argue that the results from China can necessarily be expected to generalize to all other parts of the developing world; there is evidence, for instance, of successful use of Covid vaccine incentives in Africa (Chetty-Makkan et al., 2022; Duch et al., 2023).

Our results therefore pose a dilemma for Chinese policymakers. We argue that, in predicting the likely effect of material incentives on vaccination behavior in China, research conducted on the Chinese population should be regarded as more reliable than research from elsewhere. On the other hand, this may be balanced against the consideration that experiments on hypothetical behavior are in general less likely to be externally valid than those measuring actual choices. Our study has the advantage of providing evidence from China, but it produces a result which is out of line with studies of real behavior in other countries (though not out of line with hypothetical evidence from neighboring Central Asia, in Seitz et al., 2023). We leave it to policymakers to conclude whether the hypothetical evidence presented in this paper offers a better or worse guide to the effects of vaccine incentives in China than real choice data from other countries. Finally, we note that our results are more likely to apply in the context, like that of our experiment, where the vaccine being offered is new and the population has little pre-existing knowledge about it. Our study has focused on the short-term consequences of incentives and we do not draw conclusions on their longer-term effects.

Appendix A. Supplementary data

Supplementary material: Online Appendix A and B

Supplementary material 1

Supplementary material 2

Data availability

The data will be made freely available to download as a supplementary material via the journal's website