Advantages of Using Lotteries to Select Participants for High‐Demand Covid‐19 Treatment Trials

ABSTRACT

As hospitals have experienced a surge of Covid‐19 patients, investigators of Covid‐19 treatment trials face a difficult problem: when an institution has more eligible and interested patients than trial slots, who should be enrolled? Defining a clear strategy for selecting participants for “high‐demand” Covid‐19 treatment trials is important to avoid ad hoc and potentially biased decision‐making by local investigators, which could inadvertently compromise a trial's social value, participants’ interests, or fairness. In this article, we propose a set of ethical criteria for evaluating participant‐selection strategies for such trials. We argue that the pandemic context—in particular, great urgency to develop safe and effective treatments, uncertainty surrounding Covid‐19, and strain on the health care system that limits the time and effort available for trial enrollment—favors participant‐selection strategies that optimize the ease of enrollment and, ideally, social value. A lottery and, where possible, a weighted lottery have important advantages in these respects.

As hospitals have experienced a surge of Covid‐19 patients, investigators in Covid‐19 treatment trials face a difficult problem: when an institution has more eligible and interested patients than trial slots, who should be enrolled? We propose that a lottery—and, where possible, a weighted lottery—has several ethical advantages in addressing this challenge.

HIGH‐DEMAND TREATMENT TRIALS DURING THE COVID‐19 PANDEMIC

In response to the Covid‐19 pandemic, experimental therapies are being rapidly advanced into clinical testing. ¹ Given the limitations of supportive care, and absent proven targeted therapies for Covid‐19, patients and their families may view promising experimental treatments as valuable therapeutic options despite these treatments’ uncertain safety and efficacy. Such experimental treatments may seem to be, in the words of a family member of a patient in a recent remdesivir trial, the “only hope.” ²

However, Covid‐19 treatment trial slots can be limited, as trials enroll only the number of participants required to address the given scientific questions. ³ As of April 26, 2020, most of the 93 Covid‐19 treatment trials in the United States expect to enroll no more than 100 participants per institution, and 29% may enroll no more than 30 participants per institution (see figure 1). Institutions may also be unable to open their total number of trial slots at once, especially when accommodating a surge of patients.

Figure 1.

Average Expected Enrollment per Institution for U.S. Covid‐19 Treatment Trials

These estimates were obtained from clinicaltrials.gov on April 26, 2020. The average expected enrollment per institution was derived by dividing the total expected enrollment for each listed Covid‐19 treatment trial by the number of participating institutions.

It is difficult to estimate whether there will be enough slots at a given institution at a given time to accommodate all eligible patients, as this depends both on the locally available resources (such as investigational products, beds, and personnel) and on how many local patients are eligible and interested in enrolling. While we are unaware of specific data on these issues, some institutions’ experiences with engaging in Covid‐19 treatment trials are suggestive of high‐demand situations. ⁴ Furthermore, some nontreatment trials for Covid‐19 are experiencing a surplus of volunteers (for example, there are 200,000 volunteers for 10,000 slots in a National Institutes of Health serological survey). ⁵ Altogether, it seems likely that some Covid‐19 treatment trials will be “high‐demand” trials; that is, the number of eligible and interested prospective participants will exceed the number of available slots. A trial can become high demand only after its eligibility criteria have been designed and implemented; investigators then must determine who among eligible patients is enrolled first.

How to select participants for high‐demand trials is underexplored in research ethics. ⁶ This may be unsurprising, as meeting enrollment goals is the more common problem for studies. ⁷ In certain contexts, however, high‐demand trials do exist. Indeed, trial demand may exceed availability when the limited slots of a trial offer important health‐related, psychological, financial, or other benefits, motivating many people to seek enrollment. For example, high‐demand situations have been reported in gene‐transfer trials for degenerative diseases for which limited treatment options exist. ⁸ The Covid‐19 pandemic, in its early stages, has exhibited similar features: limited trial slots provide the only access to certain experimental treatments, which, considering the limited therapeutic alternatives, patients may view as appealing despite their uncertainty. Likewise, during the early HIV epidemic, trials for the promising drug azidothymidine (AZT) were high‐demand trials. ⁹

Defining a clear strategy for selecting participants for high‐demand trials is important to avoid ad hoc and potentially biased decision‐making by any local investigator, which could inadvertently compromise a trial's social value or fairness or participants’ interests. Conceivable participant‐selection strategies include prioritizing patients who would have the best individual risk‐benefit profiles if enrolled, taking the sickest first, prioritizing patients whose inclusion would enhance the trial's scientific and social value, enrolling patients on a first‐come‐first‐served basis, and using a lottery. Each strategy has advantages and disadvantages that may be prioritized depending on a trial's specific features and context. In what follows, we analyze how the context of a pandemic may affect the preferred selection strategies.

CRITERIA FOR EVALUATING SELECTION STRATEGIES

Any participant‐selection strategy in a high‐demand Covid‐19 treatment trial—that is, selection beyond the trial's eligibility criteria—affects at least four ethically relevant dimensions of the trial. First, it affects the trial's social value: the likelihood, magnitude, and distribution of health benefits produced by information generated by the trial. Covid‐19 treatment trials are socially valuable when they use scientifically sound methods to produce information that can be used, for instance, to improve patient care. For example, sufficiently enrolling patients from certain groups (such as patients with a clinically relevant comorbidity) could allow investigators to pursue exploratory analyses that produce additional relevant information, thereby enhancing a trial's social value (although investigators should be careful not to inadvertently introduce bias through participant selection). Second, a participant‐selection strategy affects a high‐demand trial's fairness, since it determines who among interested patients will bear the trial's risks and enjoy its potential benefits. Third, a participant‐selection strategy affects a trial's risk‐benefit profile for participants, since eligible patients may differentially stand to benefit from, or be harmed by, research participation. Fourth, a participant‐selection strategy affects the time and effort required for enrollment, which, in turn, can affect the trial's social value (for instance, by influencing the rate of enrollment and research completion) and the interests of prospective participants (such as by delaying access to experimental treatments). It can also affect activities beyond the trial (for example, by consuming health care workers’ time for other research or clinical care).

These four ethically relevant trial dimensions are grounded in widely recognized ethical criteria for clinical research: social value; fair participant selection; a favorable risk‐benefit profile; and, given the potential impact of clinical trials on local health care resources, collaborative partnership. ¹⁰ While these trial dimensions reflect the ethical context of clinical research, they also overlap considerably with ethical considerations in allocating scarce but proven interventions for clinical care. ¹¹ However, there are important differences between research and clinical care, ¹² meaning that allocation strategies designed for the latter should not simply be adopted to select participants in high‐demand trials. For example, while maximizing benefits to patients may be a primary allocation criterion in Covid‐19 clinical care, ¹³ this criterion may be less suitable in the research context, given both the uncertainty surrounding the potential benefits of experimental therapies and the possible negative impact of such a strategy on the social value of the research.

Participant‐selection strategies in high‐demand clinical trials typically involve trade‐offs between these four trial dimensions. Consider a trial of an experimental Covid‐19 treatment that affects glucose metabolism, though not sufficiently enough to justify making diabetes an exclusion criterion. A selection strategy that aims to reduce risks to participants might deprioritize patients with diabetes in a high‐demand situation. Yet this could reduce the trial's social value by insufficiently representing a clinically relevant subgroup.

A weighted lottery could enhance social value while preserving the ease of implementation and some of the fairness advantages that make a simple lottery attractive for high‐demand Covid‐19 treatment trials.

Evaluating such trade‐offs depends on a trial's context—in this case, a global public health emergency precipitated by an emerging infectious disease, with many deaths, many people at risk of infection, and major societal effects (such as severe economic impacts). Given the limitations of existing treatments, Covid‐19 treatment trials are therefore being conducted against a backdrop of urgency—both for affected patients (who may have a heightened interest in receiving experimental therapies) and for society (which would benefit from safe and effective treatments for widespread use). Current trials additionally involve a high degree of uncertainty because Covid‐19 remains poorly understood and experimental treatments are being fast‐tracked to clinical testing. Finally, the strain on health care systems caused by Covid‐19 imposes significant feasibility constraints on the time and effort that local investigators can invest in selecting trial participants.

Certain trial‐specific features also affect evaluation of these trade‐offs. For example, if an experimental treatment is anticipated to have a favorable risk‐benefit profile compared to standard care, ensuring fair access to its potential benefits may become more important.

In what follows, we propose a general strategy for selecting participants for high‐demand Covid‐19 treatment trials that focuses on the current pandemic context but does not reflect trial‐specific considerations. The core elements of the current context—urgency, uncertainty, and feasibility constraints—may change in importance or degree as the pandemic evolves. Thus, when developing participant‐selection strategies, principal investigators (in other words, trial leadership) should consider any changes in this context, as well as the relevant features of their given trials.

A LOTTERY: A PROMISING STRATEGY

Given the context of the pandemic, one intuitively plausible selection strategy for high‐demand Covid‐19 treatment trials is a lottery—that is, selecting among eligible and interested patients at random. The appropriate implementation timeline for a lottery (for example, every hour, day, or week) would depend on a given trial's demand and the importance of enrolling patients soon after they are determined to be eligible.

A lottery is particularly favorable in relation to two of the four ethically relevant trial dimensions described above: enrollment effort and fairness. A lottery is straightforward and fast for local investigators to implement. While this consideration would generally not be primary, reducing the effort of enrollment is key in a pandemic, given that health care systems are strained and local investigators likely have other roles in the pandemic response (such as providing patient care). A lottery also gives each eligible and interested patient an equal chance of entering the trial—although a lottery involving frequent draws (every hour, for example) may still favor well‐connected patients or those with better hospital access.

However, a lottery does not necessarily optimize a trial's risk‐benefit profile for participants, since participants are not selected intentionally to reduce risks or enhance potential benefits. For example, a lottery might randomly select patients who face higher risks in the trial relative to other eligible patients.

Moreover, a lottery does not necessarily optimize a trial's social value. Random selection among eligible patients frequently does not maximize the study sample's representativeness of the target population—particularly for small samples, as random selection is likely to over‐ or undersample relevant subgroups by chance. But even for larger samples, random selection cannot correct for potential biases reflected in which eligible patients seek participation, which can be difficult to predict in advance. Similarly, by randomly selecting among eligible patients, a lottery does not necessarily optimize the extent to which relevant subgroups are enrolled (such as patients with a clinically relevant comorbidity). Yet sufficiently enrolling patients in these subgroups can help answer secondary research questions or enable exploratory analyses based on preexisting but deprioritized hypotheses, or on hypotheses emerging in light of the rapidly evolving knowledge about Covid‐19.

While a lottery does not necessarily reduce a trial's social value or worsen its risk‐benefit profile for participants, it precludes the opportunity to optimize these ethically relevant trial dimensions. In our view, this can be problematic, as the urgency of developing safe and effective treatments for Covid‐19 provides strong reasons to enhance a trial's social value insofar as this is possible. A simple lottery is thus most appropriate when enhancing a trial's social value would be impossible or unduly burdensome, or when attempts to increase social value could actually compromise it. In other trials, however, participant‐selection strategies that increase social value may be preferable. Indeed, assuming that patients who would face unacceptably high risks as trial participants would be excluded, it is generally acceptable to select participants based on their differential potential to increase a given trial's social value—and thereby advance the fundamental goal of clinical research. ¹⁴

WEIGHTED LOTTERIES TO ENHANCE SOCIAL VALUE

We recommend that principal investigators in high‐demand Covid‐19 treatment trials explore whether it may be possible to enhance social value when selecting among eligible patients. For example, the representativeness of a trial's sample could be improved by enrolling patients with clinically relevant comorbidities or certain demographic characteristics at a proportional rate to the target treatment population.

Several participant‐selection strategies might enhance a trial's social value while maintaining the expediency and some fairness advantages of a lottery. One strategy is to establish enrollment quotas in relevant subgroups—for example, have five slots for patients with diabetes and conduct a lottery within each subgroup if necessary. Quotas, however, could become logistically complex, especially when investigators are attempting to increase representation of patients with multiple characteristics. Furthermore, rigid quotas for addressing secondary research questions may be inappropriate when they delay overall enrollment and thereby delay a trial's ability to answer its primary research questions.

More promising may be a “weighted lottery” that algorithmically increases the odds of enrolling certain patients in high‐demand Covid‐19 treatment trials so as to enhance the social value of the collected data. In practice, a trial's principal investigator would first identify evidence‐based ways to enhance social value—for instance, by enrolling a statistically meaningful number of patients with diabetes (say, 10% of the total participant sample). This information would then be translated into an algorithm that first compares the patient pool at each enrolling institution to the desired criteria and then adds appropriate weights to the relevant patient groups. For example, if patients with diabetes account for 5% of a local patient pool, those patients might receive a weight of 2 compared to a weight of 1 for other patients in the lottery, doubling their enrollment odds. The principal investigator would need to carefully consider whether the weighting could inadvertently reduce the trial's social value—for instance, by decreasing the overall representativeness of the sample through oversampling a subgroup and thereby jeopardizing the primary research question(s). If this is a risk, the principal investigator should consider possible mitigation measures, such as monitoring trial enrollment and adjusting the weighting algorithm as needed to avoid oversampling. If the risk is significant and cannot be mitigated, then it would be preferable not to weight the problematic patient characteristic(s) in a lottery.

Because the weighted lottery software would be disseminated to each enrolling institution, implementation would not require significant effort from local investigators. Moreover, although a weighted lottery would no longer offer all eligible and interested patients an equal chance of enrollment, it would offer all these patients some chance that is proportionate to their promotion of an important social good. Thus, a weighted lottery could enhance social value while also preserving the ease of implementation and some of the fairness advantages that make a simple lottery attractive for high‐demand Covid‐19 treatment trials.

CONSIDERING ALTERNATIVE STRATEGIES IN INDIVIDUAL TRIALS

Despite the important advantages of a lottery and especially, where possible, a weighted lottery for Covid‐19 treatment trials in general, principal investigators should carefully consider the specific trade‐offs between ethically relevant trial dimensions for any given trial. We previously highlighted the relevance of trial‐specific features, including the risk‐benefit profile of the experimental treatment. For example, a repurposed treatment with minor side effects in similar patient groups and strong efficacy signals for Covid‐19 in preclinical or early‐human studies could have a considerably more favorable risk‐benefit profile than an entirely novel experimental agent. In such cases, a weighted lottery could potentially be used to enhance a trial's social value while also increasing the odds of enrolling participants who stand to benefit more. Or, in a Covid‐19 vaccine trial, a weighted lottery might be designed to increase the odds of enrolling participants who are less likely to be harmed, given the importance of reducing risks in a generally healthy study population. In both these cases, a weighted lottery could potentially enhance other ethically relevant trial dimensions while remaining fair and straightforward to implement for local investigators and enhancing social value.

We also encourage principal investigators of high‐demand Covid‐19 treatment trials to compare a lottery against other selection strategies, using the four ethically relevant trial dimensions we have identified. For instance, a first‐come‐first‐served selection strategy might seem an attractive alternative to a lottery or weighted lottery in high‐demand Covid‐19 treatment trials. Specifically, a first‐come‐first‐served approach would likely be faster and more straightforward to implement, offering advantages with respect to enrollment time and effort. However, a lottery or weighted lottery increases fairness by pooling eligible patients and giving each of them a chance to enroll, thereby reducing the impact of inequities in hospital access that a first‐come‐first‐served strategy cannot mitigate. A weighted lottery also allows principal investigators to enhance a trial's social value and potentially other ethically relevant trial dimensions. These advantages of a lottery or weighted lottery might, in many cases, outweigh the advantages in enrollment time and effort offered by a first‐come‐first‐served selection strategy. By contrast, the advantages of a lottery or weighted lottery lose significance when high‐demand trials require frequent lottery draws (as patients with better hospital access would still be favored), when patients need to be enrolled as soon as they are determined to be eligible, or when trials do not offer significant opportunities to enhance social value or other ethically relevant trial dimensions through participant selection.

Whichever selection strategy principal investigators ultimately choose, it should be justified based on the four ethically relevant trial dimensions we have identified. Finally, we recommend that principal and local investigators be transparent about how eligible and interested participants are selected for enrollment in order to preserve public trust in individual trials and the broader research enterprise.s

ACKNOWLEDGMENTS

Iyer and Hendriks contributed equally to this work.

The authors thank Luke Gelinas, Joseph Millum, David Wendler, and an anonymous reviewer for helpful feedback on earlier drafts of this paper. This study was supported by the Intramural Research Program of the National Institutes of Health Clinical Center's Department of Bioethics.

DISCLAIMER

The views expressed are the authors’ own and do not necessarily reflect those of the National Institutes of Health, the Department of Health and Human Services, or the U.S. government.