Behavioral economic methods predict future COVID-19 vaccination

Abstract

Increasing vaccine utilization is critical for numerous diseases, including COVID-19, necessitating novel methods to forecast uptake. Behavioral economic methods have been developed as rapid, scalable means of identifying mechanisms of health behavior engagement. However, most research using these procedures is cross-sectional and evaluates prediction of behaviors with already well-established repertories. Evaluation of the validity of hypothetical tasks that measure behaviors not yet experienced is important for the use of these procedures in behavioral health. We use vaccination during the COVID-19 pandemic to test whether responses regarding a novel, hypothetical behavior (COVID-19 vaccination) are predictive of later real-world response. Participants (N = 333) completed a behavioral economic hypothetical purchase task to evaluate willingness to receive a hypothetical COVID-19 vaccine based on efficacy. This was completed in August 2020, before clinical trial data on COVID-19 vaccines. Participants completed follow-up assessments approximately 1 year later when the COVID-19 vaccines were widely available in June 2021 and November 2021 with vaccination status measured. Prediction of vaccination was made based on data collected in August 2020. Vaccine demand was a significant predictor of vaccination after controlling for other significant predictors including political orientation, delay discounting, history of flu vaccination, and a single-item intent to vaccinate. These findings show predictive validity of a behavioral economic procedure explicitly designed to measure a behavior for which a participant has limited-to-no direct prior experience or exposure. Positive correspondence supports the validity of these hypothetical arrangements for predicting vaccination utilization and advances behavioral economic methods.

Behavioral economic methods can predict COVID-19 vaccination 1 year later above and beyond significant predictors including political orientation, history of flu vaccination, and a single-item intention to vaccinate.

Implications.

Practice: Behavioral economic methods can predict future vaccination behavior, thus identifying those who may benefit from corrective interventions.

Policy: Simulated behavioral economic procedures can provide insight into factors influencing health behavior engagement (e.g., vaccination uptake) offering targets for policy changes.

Research: Future research can evaluate predictive capacity in real-world environments in randomized clinical contexts.

INTRODUCTION

A measurable goal of behavioral science is the development of rapid and scalable methods to forecast future behavior for informing preventive health efforts and identifying mechanisms of health behavior engagement. The recent decade has seen a rapid growth in the application of behavioral economic theory to address issues of public policy, most notably in addiction science [1, 2]. Behavioral economics integrates behavioral psychology and microeconomics to explain decision-making and choice processes. Approaches developed within behavioral psychology evaluate choice for and consumption of commodities under conditions in which commodity cost or delay to/likelihood of receiving a commodity may vary (i.e., conditions of constraint) [3]. Advances in methods have led to increases in behavioral economic research using hypothetical, quick-to-administer tasks that can measure demand (i.e., effort exerted to defend baseline consumption of a commodity with increases in cost) and discounting (i.e., decreases in the subjective value of a commodity due to delay or probability) [4].

A foundational methodological question with direct clinical implications is the extent to which behavior measured in these hypothetical tasks predicts real-world behavior. Task responses show good convergent validity by being associated with corresponding person-level measures of risk and severity measured cross-sectionally (e.g., relationships between drug demand and drug use frequency/severity) [5]. Limited, but compelling, research has also shown how hypothetical tasks correspond well to incentivized versions when measured simultaneously [6, 7].

Less studied is the extent to which responses on hypothetical tasks may predict future behavior, particularly for novel, or yet-to-be experienced contexts. Studies in addiction science have shown that drug demand can predict future drug consumption [8, 9] and response to behavioral and pharmacological interventions [10, 11], although these studies often rely on measures of demand collected in participants with already well-established drug use repertoires. We recently showed that behavioral economic methods can be adapted to measure preventive behaviors that are yet experienced, such as receiving a COVID-19 vaccine before such vaccines were publicly available [12, 13]. These studies showed that demand for a hypothetical COVID-19 vaccine was sensitive to factors like perceived efficacy, side-effect framing, and vaccine development timeline. It remains unknown whether responses on these hypothetical tasks predicted future vaccination once COVID-19 vaccines were developed, authorized for use, and widely available. Demonstration of this correspondence would critically support the say-do relationships central to purported clinical prediction for a range of preventive health behaviors measured in this vaccine demand procedure.

The purpose of the present analysis was to evaluate the predictive validity of a vaccine behavioral economic task in forecasting future vaccination. Data were used from a COVID-19 vaccine demand measure completed in August 2020 before availability of the COVID-19 vaccines to predict future COVID-19 vaccination in June 2021 and November 2021 during which the COVID-19 vaccines were readily available to adults across the USA. To emphasize, this vaccine demand task was completed at a time before publication of clinical trials about the efficacy of COVID-19 vaccines thus providing an opportunity to test the extent to which responses regarding this novel and at the time strictly hypothetical behavior were predictive of real-world behavior approximately 1 year later. We hypothesized that vaccine demand would predict future vaccination and that this would be uniquely predictive above and beyond other expected demographic and health variables.

METHODS

General procedures

Participants were recruited from the crowdsourcing resource Amazon Mechanical Turk (mTurk) as a part of a longitudinal cohort study. Participants who completed a Behavioral Economic supplement survey (N = 497) from August 2, 2020 to August 12, 2020 (for the purpose of this analysis referred to as Baseline Assessment) were considered for this analysis. Participants were required to have a 97% or higher mTurk approval rate, more than 100 previously approved tasks, and current U.S. residence to enroll in the parent longitudinal study. Data systematicity checks were also included (e.g., task reversals) as described previously [13].

Assessments occurred in two follow-up surveys: Follow-Up 1 (June 14, 2021 to June 23, 2021; ~10 months later) and Follow-Up 2 (November 16, 2021 to November 29, 2021; ~15 months later). Participants who completed at least one follow-up survey in which vaccination status was assessed (N = 420 of 497; 84.5%) did not differ from those that did not complete a follow-up with the exception that participants with follow-up data were older (mean = 35.0 vs. 41.0; p < .001). The primary analytic sample included 333 participants who completed both follow-up assessments. All procedures were approved by the host university Institutional Review Board.

Vaccine demand task

Participants completed a hypothetical purchase task in August 2020 (Baseline Assessment) to measure COVID-19 vaccination intent [13]. Briefly, vignettes described a scenario in which the FDA had approved a COVID-19 vaccine that was immediately and freely available. Scenarios were presented to simulate going to a healthcare provider for the flu vaccine and having an option to bundle the COVID-19 vaccination at that visit. Participants responded if they would be vaccinated (Yes/No binary) across a series of efficacies defined as percentage reduction in COVID-19 hospitalization risk (100% to 0% effective in 10% increments). A full vignette is included in the Supplemental Materials.

Individual values for minimum required efficacy for each vaccine task were calculated as the individual median value between last accepted and first rejected vaccine efficacy. For example, a participant accepting values up to 50% and rejecting values lower than 40% was assigned a minimum required efficacy of 45%. Participants who rejected the vaccine at all values were assigned a value of 100 and those accepting at all values were assigned a value of 0.

Vaccine, demographic, and health measures

Vaccination status was assessed by asking participants the type of vaccination received and the number of doses received. Full vaccination was defined consistent with FDA guidance (i.e., completion of both doses of Moderna/Pfizer or the single dose of Johnson & Johnson). Full vaccination status was coded as early vaccinator (by Follow-Up 1 in June 2021), late vaccinator (by Follow-Up 2 in November 2021), and unvaccinated (no vaccination by November 2021).

Demographic measures included age, gender, race, education, and political affiliation (Republican, Democrat, and Independent). Domain-general (i.e., money) delay discounting was assessed with the Monetary Choice Questionnaire and probability discounting with the Probability Discounting Questionnaire [14, 15]. Health variables included flu vaccination in the past 3 years (Yes/No) and intention to receive a COVID-19 vaccination (Yes/No binary intent to get the COVID-19 vaccine). Health insurance was not collected, but was not expected to impact results given that COVID-19 vaccines were widely available at no cost and no cost for vaccination was specified in the task vignette. These measures were collected in the Baseline Assessment in August 2020.

Data analysis

Descriptive statistics were first computed. Primary analyses focused on comparing participants with data from follow-ups (N = 333) given the ability to determine vaccination status across the full time period. Sensitivity analyses using single follow-up showed similar results and are shown in the Supplemental Materials. Predictors collected in the Baseline Assessment in August 2020 of vaccination status during the follow-ups were analyzed using multinomial logistic regression. Multinomial models used unvaccinated as the reference group. Unadjusted models and multivariable models that included all predictors are presented. All analyses were conducted in R using two-tailed tests and type I error rate of 0.05.

RESULTS

Table 1 contains sample characteristics. Two-thirds of participants (66.7%) were vaccinated by June 2021 while another 9.3% were vaccinated by November 2021. No participant reported inconsistent vaccination status from June to November 2021 (i.e., changing responses from vaccinated to unvaccinated).

Table 1.

Multinomial logistic regression for vaccination status

	Sample characteristics	Early vaccination (n = 222)		Late vaccination (n = 31)
	mean(SE)/%	OR (95% CI)	AOR (95% CI)	OR (95% CI)	AOR (95% CI)
Age (years)	42.0 (0.6)	1.02 (0.98, 1.04)	1.04 (1.00, 1.07) *	1.00 (0.98, 1.04)	1.02 (0.97, 1.07)
Female	56.2%	0.90 (0.54, 1.52)	1.08 (0.52, 2.2)	0.66 (0.29, 1.51)	0.47 (0.17, 1.28)
White	81.7%	1.25 (0.65, 2.40)	1.33 (0.56, 3.14)	0.72 (0.27, 1.90)	1.18 (0.38, 3.68)
College	61.6%	2.28 (1.35, 3.85) **	1.54 (0.77, 3.1)	0.87 (0.38, 1.99)	0.49 (0.19, 1.30)
Democrat	42.6%	2.82 (1.46, 5.45) **	1.17 (0.48, 2.83)	9.18 (2.39, 35.22) **	9.98 (1.91, 52.00) **
Independent	31.8%	1.16 (0.63, 2.17)	0.82 (0.35, 1.90)	2.81 (0.69, 11.40)	3.64 (0.69, 19.22)
Would get vaccine	60.4%	14.91 (7.83, 28.37) ***	3.80 (1.54, 9.36) **	3.57 (1.45, 8.80) **	0.99 (0.28, 3.49)
Recent flu vaccine	52.6%	7.68 (4.16, 14.20) ***	3.65 (1.71, 7.80) **	2.89 (1.18, 7.10) *	1.97 (0.70, 5.54)
Delay discounting	−2.3 (0.1)	0.54 (0.39, 0.75) ***	0.61 (0.40, 0.94) *	1.13 (0.66, 1.94)	1.22 (0.68, 2.19)
Probability discounting	0.46 (0.02)	0.78 (0.38, 1.57)	0.84 (0.32, 2.20)	1.31 (0.41, 4.14)	1.54 (0.41, 5.81)
Vaccine demand	36.2 (1.9)	0.96 (0.95, 0.97) ***	0.98 (0.97,0.99) ***	0.98 (0.96, 0.99) ***	0.98 (0.96, 1.00) *

Note. Unvaccinated is the reference group (n = 80) for multinomial models. Bold values are statistically significant. Reference group for political orientation is Republican. Descriptive statistics for the full sample are presented in column two. AOR, adjusted odds ratio; OR, unadjusted odds ratio.

p < .05.

p < .01.

p < .001.

Figure 1 (top panel) contains aggregate mean group curves by vaccination status. All groups showed a systematic decrease in vaccination intention as a function of decreasing vaccine efficacy. Individual-level summary (Fig. 1 bottom panel) showed a systematic increase in minimum required efficacy by delay to vaccination (e.g., higher minimum required efficacy in the unvaccinated group than early vaccinator group).

Fig. 1.

COVID-19 vaccine demand by future vaccination status. Plotted are vaccine demand task by future vaccination status (early vaccinator, n = 222, gray circle; late vaccinator, n = 31, blue triangle; unvaccinated, n = 80, red square). The bottom panel presents minimum required efficacy from individual subject responses (mean with 95% confidence intervals). Significance (** p < .01; *** p < .001) and Cohen’s d effect sizes are also presented.

Results of multinomial regression models are presented in Table 1 (the unvaccinated group served as the reference group for comparisons). Vaccine demand was a significant predictor of early (OR = 0.96) and late vaccination (OR = 0.98) in both unadjusted and adjusted models with larger effect sizes observed for early vaccination status. Self-reported intention to vaccinate (OR = 14.91 early vaccination), democratic party affiliation (OR = 2.82 early vaccination), and history of recent flu vaccine (OR = 7.68 early vaccination) were generally associated with a greater odds of vaccination while greater delay discounting (OR = 0.54 early vaccination) was associated with a lower odds of vaccination.

DISCUSSION

The current analysis evaluated whether a behavioral economic method evaluating COVID-19 vaccination before dissemination of the vaccine could predict vaccination status approximately 1 year later when vaccines were widely available. Our findings demonstrated a robust relationship wherein vaccine demand collected before vaccine approval was associated with vaccination above and beyond other likely predictors including demographics and preventive health factors. Importantly, we found that vaccination was also predicted by this task above and beyond a single-item measure of COVID-19 vaccination intention (i.e., I am likely to get the COVID-19 vaccine when available). To our knowledge these findings are one of the first findings of predictive validity for a behavioral economic procedure explicitly designed to measure a behavior for which a participant has limited-to-no direct prior experience or exposure.

Relevant to consider is the distinction between associational or propensity prediction and direct prediction. Specifically, we found that responding on the demand procedure was strongly associated with future vaccination propensity. We did not, however, explicitly test or demonstrate whether the response under a certain efficacy condition was directly comparable to the future behavior in that specific condition. Here this would mean that a participant’s response at a specific efficacy in the hypothetical arrangement that closely mirrored the true vaccine efficacy would match their future vaccination behavior. This distinction is likely relevant to the broader behavioral economic literature; for example, studies have routinely shown that “trait-like” alcohol purchase task responses tend to overestimate alcohol consumption in the real-world, while at the same time still showing significant prediction of future alcohol risk and associations with real-world drinking behavior [16]. We argue that a key strength of simulated procedures is as predictive measure of general propensity and, especially when paired with experimental manipulations, to forecast the likely impact of simulated environmental or policy conditions on behavior.

Notable secondary findings of this analysis involve the demographic and health factors evaluated for their prediction of future vaccination. Specifically, we found that greater domain-general delay discounting for money, but not probability discounting of money, was associated with a lower odds of vaccination. This finding is consistent with recent work showing this relationship with vaccination [17] and with the broader delay discounting literature indicating a positive relationship between health behavior engagement and lower delay discounting [18]. Future work evaluating delay discounting as a predictive tool or intervention target in these contexts may be warranted.

Limitations include the focus on self-reported vaccination status collected via this crowdsourcing method. Important to note is that we have limited reason to question the validity of the self-reported vaccination data given the uniform correspondence across follow-ups and that systematic data checks used elsewhere in the crowdsourcing literature were employed [19]. Moreover, recent evidence suggests self-reported COVID-19 vaccination status is valid against antibody analyses, with good sensitivity and specificity [20]. The study vignette also referenced attending the clinic visit to receive a flu vaccine. This may have implied a general vaccine acceptance increasing vaccination willingness or, alternatively, increased vaccine hesitancy given concerns about co-administered or combination vaccines. Offsetting this limitation is the ecological validity of the arrangement in that bundling of COVID-19 vaccinations with other annual vaccinations is likely as the pandemic progresses. Other limitations include the reliance on a single test that measured the effect of hypothetical efficacy and relied on language that may not have directly modeled the early clinical landscape (e.g., use of FDA approval vs. “emergency use” authorization). It is likely that alternative tasks that measure different arrangements and factors like side effect severity or probability could show different, and possibly greater, validity [21].

Using the COVID-19 pandemic as an example context, we describe a direct demonstration of how a behavioral economic task designed to measure a preventive health behavior yet experienced can predict that future behavior. Decisions made during the COVID-19 infectious disease pandemic have proved relatively novel and require respondents of questionnaires and measures to consider generalized decision-making repertoires, such as deciding to take precautions in avoiding individuals with the common cold or influenza virus. The findings here provide support for the validity of these hypothetical arrangements and more broadly advance the use of these procedures in preventive medicine.

Funding

Dr. Strickland’s work was supported by the National Institute on Drug Abuse (NIDA) of the National Institutes of Health (T32 DA07209; R03 DA054098). Dr. Reed’s work was supported by a General Research Fund award from the University of Kansas. Dr. Johnson’s work was supported by NIDA (R01 DA049814; R01 DA042527).

Conflict of Interest: We have no known conflicts of interest to disclose.

Human Rights: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent: Informed consent was obtained from participants included in the study.

Transparency: This study was not formally registered. The analysis plan was not formally preregistered. De-identified data from this study are not available in a public archive. De-identified data from this study will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author. Analytic code used to conduct the analyses presented in this study are not available in a public archive. They may be available by emailing the corresponding author. Some of the materials used to conduct the study are presented in a public archive are presented in the Supplemental Materials.