Broad cross-national public support for accelerated COVID-19 vaccine trial designs

Abstract

A vaccine for COVID-19 is urgently needed. Several vaccine trial designs may significantly accelerate vaccine testing and approval, but also increase risks to human subjects. Concerns about whether the public would see such designs as ethical represent an important roadblock to their implementation; accordingly, both the World Health Organization and numerous scholars have called for consulting the public regarding them. We answered these calls by conducting a cross-national survey (n = 5920) in Australia, Canada, Hong Kong, New Zealand, South Africa, Singapore, the United Kingdom, and the United States. The survey explained key differences between traditional vaccine trials and two accelerated designs: a challenge trial or a trial integrating a Phase II safety and immunogenicity trial into a larger Phase III efficacy trial. Respondents’ answers to comprehension questions indicate that they largely understood the key differences and ethical trade-offs between the designs from our descriptions. We asked respondents whether they would prefer scientists to conduct traditional trials or one of these two accelerated designs. We found broad majorities prefer for scientists to conduct challenge trials (75%) and integrated trials (63%) over standard trials. Even as respondents acknowledged the risks, they perceived both accelerated trials as similarly ethical to standard trial designs. This high support is consistent across every geography and demographic subgroup we examined, including vulnerable populations. These findings may help assuage some of the concerns surrounding accelerated designs.

1. Introduction

Each week without a COVID-19 vaccine exacts an unimaginable toll on global public health, economic livelihoods, and political stability [1]. At present, well over 100 COVID-19 vaccine candidates have been identified, at least 23 of which are in clinical trials worldwide [2]. Traditional efficacy trial designs take many months and rely on the persistence of high transmission rates around the trial site. However, several trial design options may expedite the process. Two of the leading options are, first, the use of “human challenge studies,” wherein study participants volunteer to be exposed to the virus instead of having scientists wait for participants to be exposed to it in their daily lives (potentially accompanied by additional safety testing in a diverse population) [3]; and, second, the integration of smaller Phase II safety and immunogenicity trials into larger Phase III efficacy trials, wherein more study participants receive a vaccine candidate before data about its safety and immunogenicity are available from a traditional Phase II trial. These approaches may significantly accelerate COVID-19 vaccine development [3], [4], [5], [6], [7]. However, they also create risks to participants: in the case of challenge trials, deliberately exposing participants to the virus; and, in the case of integrated trials, exposing additional participants to a vaccine before Phase II safety and immunogenicity studies have been completed.

While technical questions also surround the use of these and related approaches in COVID-19 vaccine testing [8], [9], [10], [11], several of the roadblocks to their implementation depend in part on the answer to an empirical question: would members of the general public see these designs as ethical [12], [13], [14], [15], [16]?

To understand how members of the public informed of the ethical trade-offs involved would view these questions, we measured public opinion on the ethics of accelerated vaccine trial designs.

There are at least three reasons why public opinion towards the ethics of these designs is relevant to vaccine trialists, ethicists, and policymakers.

First, there has been considerable debate about the ethics of accelerated designs for COVID-19 vaccine trials, particularly human challenge trials [3], [8], [13], [14], [15], [16], [17], [18]. Ethicists have held that part of what determines whether proposed research is ethical are societal attitudes toward that research, especially among those individuals and communities who will be most affected by it [19]. This is one reason that community members have long been included on institutional research ethics committees [20]. Decisions about whether to use standard or accelerated COVID-19 vaccine trial designs affect people worldwide. Accordingly, the WHO recently identified “consultation and engagement with the public” as a core criterion for the ethical acceptability of challenge trials [12]. “Community engagement” is said to be an important ethical safeguard “to $\dots$ ensure that the research is consistent with the community’s values, show respect for members of the community, and enhance transparency” [19]. Notable scholars have also explicitly called for “public opinion surveys [to] identify concerns” on the ethics of COVID-19 challenge trials [13] and “public engagement” to “assess the local acceptability of human challenge studies” [16]. Our research answers these calls.

Second, vaccines are only effective if the public gets vaccinated, and public trust in vaccines may depend in part on public views about the ethics of the process by which those vaccines were developed and tested [21]. As a result, some scholars have expressed concern that the use of human challenge trials could “feed distrust among the public,” and therefore “exacerbate challenges in vaccine roll-out and delay uptake of an effective vaccine” [22]. Our research can help empirically assess such concerns.

Third, as many decisions about vaccine trials and vaccine manufacturing will be made by government officials who are accountable to public opinion, it is important to know what the public thinks about the social and ethical trade-offs these options involve [23]. In practice, policymakers and regulators may be hesitant to support an accelerated design viewed as unethical by the public.

However, despite a plethora of research on public opinion regarding medical research [23] and on how individuals make ethical judgments [24], relatively little is known about how individuals perceive the ethics of accelerated vaccine trials on consenting volunteers, especially in the setting of the current global public health emergency. One exception is Gbesemete et al., who find strong public support for COVID-19 challenge trials in focus groups conducted with 57 young people in the United Kingdom [25]. Our work, described below, builds on their work with a larger sample size, cross-national data, a more diverse subject pool, and a different, complementary research design.

2. Materials and methods

To measure public opinion about COVID-19 vaccine testing, we surveyed people worldwide in May 2020. We recruited participants to the survey using the online sample provider Lucid. Prior research shows that US Lucid respondents demographically track well with US national benchmarks and that many political, psychological, and experimental results replicate on Lucid samples [26]. In addition to surveying $n=2180$ individuals in the United States, we also surveyed approximately 500 individuals through Lucid in each of the following predominantly English-speaking geographies worldwide: Australia ($n=500$), Canada ($n=687$), Hong Kong ($n=422$), New Zealand ($n=498$), South Africa ($n=548$), Singapore ($n=520$), the United Kingdom ($n=565$). Survey respondents in Australia, the United Kingdom, and the United States surveys were selected to match Census population benchmarks on age, gender, and race and ethnicity, but this was not possible in the other geographies. Although the results are not meaningfully different across geographies, respondents in Canada, Hong Kong, New Zealand, South Africa, and Singapore were not selected to match Census benchmarks and the results for these geographies should be viewed with this limitation in mind. Furthermore, prior work on the representativeness of Lucid has been limited to the US. All surveys were conducted in English. The Supplementary Materials provide further detail on the demographics of survey respondents. Later, we also report results from sensitivity analyses and placebo tests to further support the robustness of our findings.

Survey respondents were randomly assigned to participate in one of two studies. Study 1 asked respondents about a standard Phase III trial in which scientists wait for participants to be exposed to the virus in the real world and a challenge trial in which participants are intentionally exposed to the virus. Study 2 asked respondents about a standard Phase II followed by Phase III trial and an integrated Phase II and III trial in which the Phase II safety and immunogenicity trial is integrated into a larger Phase III efficacy trial, thereby reducing the vaccine testing timeline.1

In both studies, we described essential details of these research designs for respondents—including their procedures, risks to participants, and benefits to society. Fig. 1 provides an example description of the designs that we showed participants in Study 1 (see Fig. S4 for the stimulus used for the integrated trial). We followed these descriptions with a bulleted summary of the key differences (given in the Supplementary Materials).

Fig. 1.

Example Stimulus, Study 1 (Study A is the Challenge Trial in this Example). Note: In order to communicate the details and differences between standard and challenge trials, participants in Study 1 were shown a table like the one above. The highlighted elements were randomized across possible values detailed in the Supplementary Materials. The highlights did not appear for respondents. We randomized these parameters given uncertainty about how particular vaccine trials might be conducted, to ensure our findings were not sensitive to any of these parameters. We did not allow participants to move on from the page describing the trial design until at least 60 seconds had gone by. See the Supplementary Materials for an example from Study 2. As we show in the Supplementary Materials (Tables S2 and S6), the vast majority of survey respondents were able to correctly comprehend the key differences between the designs.

In both studies, we randomly assigned each respondent to several parameters of the vaccine trials we described, including their sample sizes, COVID-19 infection and death rates for their participants, the vaccine approval timeline under standard trials, and how much alternative designs would accelerate this timeline. These randomizations were intended to evaluate the robustness of our findings to a range of assumptions about the likely design and consequences of vaccine trials given the uncertainty surrounding all of these parameters. The Supplementary Materials provide the full wording of the scenarios.

There is ongoing scientific uncertainty and differences of opinion regarding the likely design, benefits, and risks of both conventional and accelerated COVID-19 vaccine trial designs. This ongoing uncertainty merits several comments about our summaries of the designs for respondents. First, our summaries should not be interpreted as making any new scientific claim about the likely trial designs; rather, they represented our best assessment of the available evidence as of when the survey was conducted (May 2020). Second, reflecting the scientific uncertainty about likely trial designs, as described above we also randomized a number of aspects of the scenarios (as shown in Fig. 1 and detailed in the Supplementary Materials). As we show below, our results are robust across these potential trial design details. Third, we did not include any scenarios in which accelerated designs took as long or longer than conventional designs, as the ethical case for accelerated designs is mainly relevant if they in fact do accelerate vaccine approval timelines (which is a separate question). Finally, our challenge trial scenarios always described it as “very unlikely any study participants would die of coronavirus” based on the available research about the infection fatality rate for young, healthy people, who we state would form the subject pool for a challenge trial [12], [16], [27], [28].2

After presenting the study designs, we asked respondents which of the two trials they would prefer to see scientists conduct. We next asked respondents to rate how ethical and how scientifically valid they considered each trial design and how likely they would be to take the vaccine if it had been tested using each design, if the vaccine were to be approved. We also asked several questions to measure respondents’ successful comprehension of the study designs; this allows us to determine whether our survey instrument successfully communicated the intended differences between the vaccine trial designs. We ended the survey with several demographic questions, which allow us to identify and separately analyze data for numerous sub-populations, including vulnerable sub-populations. We also use these demographic questions to construct weights for the US sample, allowing us to test whether our conclusions change when weighting our sample to the demographics of the US population. The Supplementary Materials provide the full question wording.

We pre-registered a pre-analysis plan, provided in the Supplementary Materials, that details the pre-specified analyses we planned to conduct, including which subgroups we would examine. The Supplementary Materials also detail two minor deviations from our pre-analysis plan.

The survey was approved by the Committee for the Protection of Human Subjects at the University of California, Berkeley (#2020-04-13250) and Yale University (#20000281000). In the survey, we first asked for informed consent to participate. 471 participants did not consent and were removed from the survey.

All of the individual participant data collected during the study, after de-identification, study protocol, pre-analysis plan, informed consent form, and analytic code will be available immediately following publication with no end date to anyone who wishes to access the data for any purpose. Data will be made available indefinitely at https://osf.io/bgxe4/.

All authors declare no competing interests.

3. Results

First, we find that a majority of respondents successfully understood the studies we described, as most correctly answered each of several scenario comprehension questions. For example, in Study 1, 84% (95% CI: 82–85%) of respondents correctly stated that the challenge trial involves intentionally infecting study participants with the virus. Similarly, 75% (95% CI: 73–76%) of respondents in Study 2 correctly stated that the standard trial involves additional safety testing not present in the integrated design. Results for additional scenario comprehension questions are presented in the Tables S2 and S6.

We pre-specified that our primary outcome of interest was participants’ answer to the question “If you had to choose, which study would you rather have scientists conduct?” Respondents had the choice of selecting “Study A” or “Study B.” Whether the accelerated design was labeled as Study A or B was randomly assigned for each respondent.

Fig. 2 shows our main results for this primary outcome. Overall, we find broad cross-national support for both the challenge trial and the integrated trial over standard vaccine trials. There are also similar, high levels of support in every subgroup we examined, including among vulnerable populations (e.g., those over 65, essential workers, racial minorities), politically relevant subgroups in the United States, those without a college degree, and among those who correctly answered all scenario comprehension questions asked after the scenario was presented (see also Tables S4 and S8).

Fig. 2.

Broad Support for Challenge Trials and Integrated Trials That Accelerate COVID-19 Vaccine Development. Notes: The Figure shows the percent of respondents who preferred that scientists conduct each accelerated trial instead of a standard trial. Study 1 considered the use of a challenge trial in which participants are intentionally exposed to the virus instead of, in a standard trial, waiting for participants to be exposed to it in their daily lives. Study 2 considered the integration of smaller Phase II safety and immunogenicity trials into larger Phase III efficacy trials instead of, in a standard trial, waiting for the completion of a Phase II safety and immunogenicity trial before commencing a Phase III efficacy trial. The Figure shows the mean proportion of respondents who say they would prefer scientists to conduct each accelerated design for each study overall, by geography, and across various demographic subgroups. 95% confidence intervals surround the point estimates. Sample sizes shown are totals across both studies; respondents are approximately evenly split across the two studies. See Tables S3, S4, S7, and S8 for numerical values. * This Figure presents non-white respondents from Hong Kong only. There were an unanticipatedly large number of participants in Hong Kong who indicated they were white. The 2016 Hong Kong Census estimates that only 0.8% of the Hong Kong population identifies as white; we discuss this issue in further detail in the Supplementary Materials.

These results are robust across the various trial design assumptions we randomized to each respondent (e.g., the sample size of each trial we described). Further evidence is presented in the Supplementary Materials. Of particular interest is that the results are not particularly sensitive to the amount of time by which we tell respondents the design could accelerate vaccine development: we find similar results across the range of 2–6 months (see Tables S5 and S9). The results are also robust when the sample from the United States is weighted to match Census targets for gender, race, age, and education (see Tables S3 and S7). Bolstering the breadth of public support for accelerating trial designs, multivariate regressions reveal few substantively or statistically significant demographic or attitudinal predictors of support (see Figs. S2 and S4).

We present further results in the Supplementary Materials for our three secondary outcomes. In Study 1, participants stated that they saw the challenge trial as slightly more likely to be ethical ($p<0.01$), scientifically valid ($p<0.001$), and that they would be more likely to take an approved vaccine if it had been tested using this design3 ($p<0.001$). For example, Fig. 3 shows the proportions of respondents who described each accelerated design as ethical. 72% (95% CI: 70–73%) of participants described the challenge trial as either “probably” or “definitely” ethical, as compared to 68% (95% CI: 67–70%) of participants who described the standard trial as such. These results are broadly consistent across geographies (see Table S3). In Study 2, for the integrated trial, although as described above participants on average are more likely to prefer that scientists conduct the integrated trial instead of the standard trial, the integrated trial has slightly lower averages than the standard trial for our three secondary outcomes (see Table S7). This is because those who prefer the integrated trial rate both trials approximately equally on average, whereas the minority who do prefer the standard trial rate the integrated trial slightly lower on average; however, most in this minority still describe the integrated trial as ethical (58%, 95% CI: 56–61%).

Fig. 3.

Vast Majority of Survey Respondents Describe Both Accelerated Designs as Ethical.

To help assess the potential concern that our conclusions spuriously arise from issues with the representativeness of our sample, we also conducted a Rosenbaum sensitivity analysis [29]. We find that if supporters and opponents of challenge trials were equally prevalent in the population (the null hypothesis), our results finding broader support for challenge trials could only be accounted for by sampling bias if pro-challenge trial individuals were $\gamma =2.78$ times more likely to be selected for our survey than anti-challenge trial individuals ($\gamma =1.68$ times more likely for the integrated study). As we discuss in the Supplementary Materials, this is an unusually high $\gamma$ value for social science studies. Furthermore, the lack of heterogeneity in support for accelerated designs within our sample on observed demographics (e.g., country, age, race, education) provides additional evidence that our results are unlikely to be driven by sampling bias: since none of the many subpopulations we examined in our sample oppose accelerated designs on average, this suggests it is unlikely that there exists a subpopulation very strongly opposed to accelerated designs that we undersample by a factor of 2.78 or 1.68 (for challenge and integrated, respectively).

To further assess this potential concern, we also conducted a placebo test by comparing our US sample with nationally-representative samples from the Pew Research Center and Gallup on items related to vaccine safety and scientific knowledge. First, Gallup data finds that 86% of Americans say that vaccines are not “more dangerous than the diseases they are designed to prevent” [30]; in our US sample, this number is 83%. Second, probing scientific knowledge more generally, Pew data finds that 76% of Americans know that ocean tides are caused by the pull of the moon and that 72% of Americans know that cell phones use radio waves [31]; in our US sample, these figures are 77% and 73%, respectively. That our sample closely reflects other, nationally representative samples on these benchmarks is encouraging for the external validity of our findings.

The results of the sensitivity analysis, which bounds unobserved bias, the lack of observed heterogeneity of responses across observable characteristics, and the close resemblance of our sample to other samples on related dimensions therefore all indicate that sampling bias is very unlikely to account for our results.

4. Discussion

We conducted large-N surveys in eight countries to gather systematic data about public support for accelerated COVID-19 vaccine trial designs. Our results suggest that there is broad public support across nations and demographics for accelerating COVID-19 vaccine trials through integrated Phase II and III trials, and especially through challenge trials. A majority of our respondents preferred the use of these accelerated designs and saw them as ethical. Indeed, on average our respondents found challenge trials to be more likely to be ethical than standard trials. These results are consistent among every subgroup we examined, including many vulnerable populations of interest, across the countries we surveyed, and across many plausible vaccine trial design details. While the vast majority of participants in our survey understood the key differences between the designs we described, it is of particular note that, among those who did understand them, fully 86% and 68% preferred for scientists to conduct challenge and integrated trials, respectively.

While our results do not at all settle the complex ethical debate about these accelerated trials, nor do they address the scientific or technical questions surrounding their use, they do bolster the case for these trials.

First, many scholars and the WHO have called for consulting the public regarding COVID-19 vaccine trial designs [12], [13], [16] given the longstanding view that part of what determines whether proposed research is ethical is whether the affected community supports the research [19], [20]. Our results suggest the public in multiple countries is broadly supportive of accelerated trial designs in this instance and sees them as ethical.

Second, our results help diminish the worry that, by appearing unethical, these designs would undermine public trust in vaccines, or in a resulting COVID-19 vaccine. Indeed, respondents said they would be more likely to take a vaccine that had been tested using a challenge instead of a conventional trial. Although this survey-based self-report should not be interpreted as likely diagnostic of future behavior, it can help assuage concerns that a challenge trial would undermine public willingness to take a COVID-19 vaccine. While our results do not directly speak to how the public would react to a severe adverse event in a challenge trial, it should be noted that such events are also possible in conventional trails, and potentially even likelier to occur given their larger sample sizes.

Finally, our results can provide some reassurance to policymakers and regulators concerned about potential negative public sentiment towards the use of these designs.

Our results have several limitations worth noting. First, our sample came from an online survey panel. Unfortunately, we determined that gathering a random probability sample would have required significant delay, past the point at which findings about public opinion could inform ongoing decision-making. That said, previous work has found that the sample we used is demographically representative and that a number of political, psychological, and experimental results replicate on it [26]. Moreover, the robustness of our results across geographies and demographic subgroups suggest it is unlikely that a different survey sampling approach in these geographies would produce qualitatively different results [32]. Finally, our sensitivity analyses and placebo tests empirically suggested that sampling bias was very unlikely to account for our results. Still, our data is likely to exclude those without internet access who therefore could not take our survey.

Second, although our survey found relatively consistent results in English-speaking countries worldwide, it is possible that results could differ in other locales [24]. The results could also change if they were re-run at a different time. The surveys were fielded in May 2020, prior to the commencement of Phase 3 trials, which may change how the public weighs the risks and benefits of the accelerated designs.

Third, when individuals consider moral dilemmas, they typically weigh ethical costs and benefits more, and their initial emotional reactions less, when they engage in more careful considerations of the alternatives at hand [33]. Our study participants spent a median of 176 seconds reading about the trial design and answering our primary and secondary outcome questions. This probably reflects less time and engagement than would be present were our respondents to deliberate with others [34], but potentially more than, e.g., when reading a news article. Consistent with the possibility that individuals would weigh social benefits even more (and thus prefer accelerated designs to an even greater extent) were they to engage in more deliberate reflection, Gbesemete et al. find even stronger public support for challenge trials in focus groups conducted with 57 young people in the UK [25]. This suggests that methods that facilitate additional participant education and deliberation, such as focus groups, may find even greater public support for accelerated designs than we found. Future work should nevertheless continue to assess the extent to which conclusions might change if individuals were to encounter information about these trial designs in other settings.

Fourth, explaining these trial designs to laypeople necessarily involved making judgments about how to describe the trials amidst scientific uncertainty surrounding the likely designs and their risks. However, we found that our respondents did largely understand the key trade-offs involved and that their support was consistent across the alternative assumptions and scenarios we presented. Nevertheless, scientific and technical knowledge about COVID-19 and associated vaccines is also evolving rapidly, and, although our results were consistent across many design details, it will be critical for future work to explore the robustness of public support for challenge studies and other trial designs across different conditions, especially if assessments of overall risks to participants or benefits to society change dramatically.

Finally, although we did demonstrate the robustness of our results across many possible trial design details, there are remaining questions we did not ask respondents and that future research should consider.

First, might respondents have preferred the accelerated designs to the standard designs even if they were not faster than the standard designs? For example, in challenge trials, fewer participants receive the experimental vaccine; respondents may prefer the challenge trial for this reason. In our studies, we always presented respondents with scenarios that stated that the accelerated designs were expected to produce a vaccine faster than a standard design, so our data do not speak to the question of whether respondents would still prefer the accelerated designs were they not to accelerate the vaccine approval timeline. However, we did find that respondents were not particularly sensitive to how many months the accelerated design was able to accelerate the vaccine approval timeline, especially in the case of challenge trials (see Tables S5 and S9). This is consistent with the plausibility of the hypothesis that some respondents preferred the accelerated designs for reasons unrelated to their accelerated timeline. Our research offers a blueprint for future research that can and should consider whether the public finds accelerated designs preferable even if they do not accelerate vaccine testing timelines. There are also alternative trial designs that we did not include in our surveys, such as enrolling very large samples in Phase 3 designs to speed up the testing timeline and adaptive designs, that future research could consider as well.

Additionally, COVID-19 challenge trials in healthy, young volunteers would probably be supplemented by additional safety testing in more diverse populations; to keep the scenarios relatively concise we did not describe this detail to respondents. Our work also leaves open questions about how the public would react to unforeseen adverse events during accelerated vaccine trials and whether such events would have different effects on public trust than adverse events in conventional trials, public perception of combinations of these accelerated designs, preferences between them, and between them and other accelerated designs, such as adaptive integrated trials [6]. However, our findings do show that it is possible to successfully explain relevant ethical trade-offs in trial design to a global public, and that people in many geographies broadly prefer approaches that accelerate COVID-19 vaccine trial timelines.

Contributors

All authors attest they meet the ICMJE criteria for authorship. All authors contributed to conceptualization and study design. DB, JK, and AG wrote the first draft of the manuscript. DB and JK contributed to the data collection and analysis. All authors contributed to critical revision of the manuscript for important intellectual content and gave final approval. DB and JS contributed funding.

CRediT authorship contribution statement

David Broockman: Conceptualization, Methodology, Formal analysis, Writing - original draft, Writing - review & editing, Visualization, Funding acquisition. Joshua Kalla: Conceptualization, Methodology, Formal analysis, Writing - original draft, Writing - review & editing, Visualization, Funding acquisition. Alexander Guerrero: Conceptualization, Methodology, Writing - original draft, Writing - review & editing. Mark Budolfson: Conceptualization, Methodology, Writing - review & editing. Nir Eyal: Conceptualization, Methodology, Writing - review & editing. Nicholas P. Jewell: Conceptualization, Methodology, Writing - review & editing. Monica Magalhaes: Conceptualization, Methodology, Writing - review & editing. Jasjeet S. Sekhon: Conceptualization, Methodology, Writing - review & editing, Funding acquisition.

Declaration of Competing Interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Supplementary material

The following are the Supplementary data to this article: