REPRESENTATION AND EXTRAPOLATION: EVIDENCE FROM CLINICAL TRIALS

Abstract

This article examines the consequences and causes of low enrollment of Black patients in clinical trials. We develop a simple model of similarity-based extrapolation that predicts that evidence is more relevant for decision-making by physicians and patients when it is more representative of the group being treated. This generates the key result that the perceived benefit of a medicine for a group depends not only on the average benefit from a trial but also on the share of patients from that group who were enrolled in the trial. In survey experiments, we find that physicians who care for Black patients are more willing to prescribe drugs tested in representative samples, an effect substantial enough to close observed gaps in the prescribing rates of new medicines. Black patients update more on drug efficacy when the sample that the drug is tested on is more representative, reducing Black-white patient gaps in beliefs about whether the drug will work as described. Despite these benefits of representative data, our framework and evidence suggest that those who have benefited more from past medical breakthroughs are less costly to enroll in the present, leading to persistence in who is represented in the evidence base.

As a physician caring for patients in an urban safety-net setting and wanting to provide the best evidence-based preventive care...I would spend as much time on the science as I devoted to reinforcing with patients why they should still trust these guidelines and the process, despite the unrepresentative populations in the evidence base.

—Kirsten Bibbins-Domingo (NASEM 2022, xii)

I. Introduction

Innovation does not benefit everyone equally (Jones and Kim 2018; Aghion et al. 2019; Kline et al. 2019). Research investments skew toward developing technologies appropriate for more profitable groups (Kremer and Glennerster 2004; Cutler, Meara, and Richards-Shubik 2012; Jaravel 2019; Michelman and Msall 2021), and diffusion often occurs faster among the well-connected or well educated (Skinner and Staiger 2005, 2015; Glied and Lleras-Muney 2008; Foster and Rosenzweig 2010; Papageorge 2015; Agha and Molitor 2018; Hamilton et al. 2021). In this article, we explore a third dimension of innovation and inequality. We ask whether the low enrollment of certain groups in the research and development process (Koning, Samila, and Ferguson 2021) creates gaps in how much group members use those technologies. Put differently, does how a technology is developed affect who adopts it?

Our context is new drug approval in the United States, where information on drug safety and efficacy—generated from clinical trials on human subjects—must be submitted to the U.S. Food and Drug Administration (FDA) before the drug can be sold. Racial disparities in the production of clinical evidence and the eventual diffusion of products is common (Wang et al. 2007; Jung and Feldman 2017; McCoy et al. 2019; Ding and Glied 2022; Elhussein et al. 2022). As Figure I documents, Black patients are consistently underrepresented in clinical trials relative to their share in the U.S. population (Panel A) and are similarly underrepresented in prescriptions for newly approved medications (Panel B). Population is the implied benchmark in Figure I and we note that Black patients are often even more underrepresented relative to their disease burden (Green et al. 2022). Although other groups have also been historically underrepresented, we focus on Black Americans for several reasons, including the history of racial discrimination and associated distrust, persistent racial disparities in health outcomes, and continued underrepresentation in research.¹

Figure I. Racial Disparities in the Development and Distribution of New Drugs.

Panel A plots the median enrollee percentage by race (Black and white) for pivotal clinical trials, studies that support new-drug applications to the FDA, over time. Panel B plots the median new drug prescription percentage by race in each year relative to its approval. Straight lines in Panels A and B plot population shares by race in the United States as reported in the 2020 census (Black population share is 13.6% and non-Hispanic white population share is 59.3%; U.S. Census Bureau 2021). Panel A is drawn from the FDA Drug Trials Snapshots data, and Panel B is from the Medical Expenditure Panel Survey data (Agency for Healthcare Research and Quality 2022). Online Appendix Figure B1 plots Panel A using a longer time series from ClinicalTrials.gov. Online Appendix Figure B2 plots the distribution of race in trials using both the ClinicalTrials.gov and FDA Drug Trials Snapshots data sets. Online Appendix Figure B4 plots prescribing rates of new drugs per 1,000 individuals in each racial group.

While gaps in trial enrollment are well-documented, the consequences, if any, have not been rigorously studied. Two natural questions emerge. First, does representative data matter to physicians and patients? Second, if so, why are such data not (endogenously) supplied by the market? To address the first question, we conduct two survey experiments designed to understand physician and patient reactions to trial evidence. To address the second question, we turn to a theoretical framework that sheds light on how underrepresentation may persist, even if representative data would lead to higher drug demand. It also identifies potential levers for policy intervention, which we assess in the context of case studies.

Our framework models how physicians and patients interpret the evidence that supports new technologies when making decisions about whether to adopt them. Through instruction in evidence-based medicine (EBM), physicians are trained to consider whether a new product would work similarly well in their patients as those in its trial. A typical question from EBM training is: “Are the participants in the study similar enough to my patient?” (Masic, Miokovic, and Muhamedagic 2008, 222). Inspired by this process and the role of reasoning by similarity and analogy in belief formation (e.g., Gilboa and Schmeidler 1995; Mullainathan, Schwartzstein, and Shleifer 2008; Bordalo, Gennaioli, and Shleifer 2020; Bordalo et al. 2022; Malmendier and Veldkamp 2022), we develop a model of similarity-based extrapolation. We assume that people update more readily from evidence when their patients (in the case of doctors) or people like them (in the case of patients) have more in common with the experimental sample. Our framework incorporates this assumption in a simple way: it assumes doctors and their patients have in mind a model where a given group characteristic (e.g., race) could be correlated with drug efficacy and they update model parameters using Bayes’s rule. A key result of our framework is that—conditional on trial data—the perceived benefit of a drug will be increasing not only in the average reported efficacy but also increasing at a decreasing rate in the share of one’s own group in the trial.

To empirically assess whether representation affects clinical decisions and health behavior, we designed and conducted two survey experiments among patients and physicians. After completing a short module eliciting patient panel characteristics, physicians viewed profiles of diabetes drugs, including the drug’s mechanism of action and the design of the supporting clinical trials. For each profile, the share of Black trial subjects and average drug efficacy in trials were cross-randomized from distributions of values collected in a comprehensive search of clinical literature. To introduce sufficient variation in sample demographics and efficacy within the mechanism of action of a given drug, the drugs shown were hypothetical but were based on recently developed drugs to treat diabetes.² After viewing each profile, physicians were asked to indicate their intent to prescribe the drug to patients in their care.

A separate experiment was designed for patients because they must fill and adhere to a prescription to realize any health gains. We recruited 275 patients with diagnosed hypertension who identified as either white or Black. We assessed their interest in a novel therapy to treat hypertension that had been tested in a real clinical trial at two separate sites with varying shares of Black participants. Other product characteristics, including drug efficacy in lowering blood pressure, were held constant.

We find that physicians are more willing to prescribe drugs tested on representative samples. A 1 standard deviation increase in the share of Black trial participants increases physician prescribing intention for a given drug by 0.11 standard deviation units. The magnitude of this effect on prescribing is medically meaningful and equivalent to roughly half the standardized effect of the drug’s efficacy. It also correlates strongly with donation behavior to campaigns aimed at boosting trial participation for underrepresented minority communities measured a few weeks after the initial intervention. In prespecified heterogeneity analyses, we find that the effect of increasing Black representation in a clinical trial sample on prescribing intention is close to zero for doctors who do not routinely see Black patients and rises steeply in the share of a physician’s own patients who are Black.

In our patient experiment, when Black respondents were presented with a representative trial, they viewed the drug in question as significantly more relevant for their own blood pressure control and were 20 percentage points more likely to state that the drug will work as well for them as it was shown to work in the trial. We also find in a separate but similar survey experiment that Black patients exposed to a representative trial were more likely to indicate that they want to participate in future clinical trials, and that they viewed the researchers as more trustworthy. This suggests that increasing representation might be one tool to help address medical mistrust. In contrast, and consistent with the model’s prediction of diminishing returns to representation, we do not find significant effects associated with the trial composition for white patients. The combination of physician and patient results suggest that doctors are broadly acting as agents for their patients.

Survey experiments are important tools for uncovering people’s mental models and perceptions (Stantcheva 2021, 2022) but are also subject to critiques, such as experimenter demand and social desirability bias. Our experiments were designed to mitigate such concerns. First, we used neutral recruitment materials stating that our goal was broadly to understand views on medical research, mirroring language from a nonprofit dedicated to the same whenever feasible.³ Second, we recruited both white and Black patients. If the response to sample representation was solely attributable to social desirability, we might expect to find similar effects for both groups (we do not). Third, survey responses correlate with actual donation behavior in a follow-up study.

A related concern is that our experiment may have informed patients and doctors about something that they did not already know about—that is, the composition of clinical trials. If so, our results might overstate the degree to which trial representation influences treatment choices. Indeed, the order of questions and salience of race might have played a role in the magnitudes of our effects. To better understand baseline knowledge in our study populations, we reviewed literature on how doctors evaluate trials and obtained data on patients’ knowledge regarding medical research. Physicians educated at accredited medical colleges in the United States are explicitly taught to consider the applicability of trial findings to their own patients through EBM training (Blanco et al. 2014).

In our survey, 72% of physicians reported that they have been asked by patients whether a new medicine will “work in people like me.” Data from the nonprofit Research!America reveal that Black and white respondents are aware of clinical trials (80% and 88%, respectively). However, Black respondents are less likely to believe that science benefits them and less likely to consent if invited to participate in clinical trials than white respondents. Two additional pieces of evidence suggest that trial representation is taken into account by (at least some) doctors and patients: one comes from stakeholder quotes compiled in the writing of a recent National Academies of Science Engineering and Medicine report (NASEM 2022) and another comes from the association between more representative clinical trials and higher prescribing rates for new drugs among Black patients (see Section VI.B and Online Appendix Table D2).

Turning to mechanisms, we find that doctors—and to a greater extent, patients—lack confidence in extrapolating from samples that are not representative of them or their patients. This is true of both Black patients (when extrapolating across racial groups) and white patients (when extrapolating across countries). One question is whether this hesitancy to extrapolate, especially among doctors, is a mistake. Given the current state of the literature and data availability, this does not seem to be a clear mistake. Manski, Mullahy, and Venkataramani (2022) show, under mild assumptions regarding doctors’ objective function, that including any predictive factor in clinical decision making is welfare enhancing. Is race in fact predictive of treatment effects? First, precisely because representation is so low, clinical trials offer limited direct evidence on this question. Green et al. (2022) review 290 new drug approvals in the FDA Drug Trials Snapshots data, and approximately 80% did not report treatment effects for Black patients separately; among those that did, 91.4% and 98.1% found no difference in side effects and benefits, respectively. Ramamoorthy et al. (2015) report a higher rate of heterogeneous effects in a review of post marketing analyses, finding such effects for nearly 20% of all new drugs. Second, because the medication mechanisms of action (i.e., a drug’s pharmacodynamics) are often incompletely specified and evolving, it is difficult to provide assurances that the findings will extrapolate across patients with different characteristics without trial evidence. Third, there is a strong relationship between social class and race in the United States that could affect pharmacokinetics, or how the drug is metabolized. Indeed, in our experiments, respondents cited the possibility of biological, socioeconomic, and environmental differences that could alter drug performance as rationales for their lack of confidence. Fourth, even if physicians believe findings do extrapolate, they might internalize patients’ lack of confidence for a variety of reasons (Ellis and McGuire 1986), including that it might affect patient adherence. Our qualitative findings from doctors explaining why they care about representation include concerns regarding treatment effect heterogeneity and concern for patients’ views.

Importantly, we find that increasing the representativeness of medical research can reduce prescription gaps. Physicians treating Black patients are considerably less willing to prescribe drugs approved on the basis of unrepresentative trials—at all levels of drug efficacy—compared with physicians who treat white patients, mirroring the racial prescription gap observed in the Medical Expenditure Panel Survey. When clinical trial samples are more representative of Black patients, this gap disappears. The difference between the share of Black and white patients who believe that the drug will work as well for them as it did in clinical trials is also eliminated when respondents are shown results generated from more representative data. These findings suggest that policies that increase representation in the evidence base for new technologies could narrow gaps in their adoption.

These findings then also imply that a firm could increase sales by recruiting a more representative sample. The trade-off in doing so is cost—our framework and evidence suggest that a history of underrepresentation in (voluntary) research leads Black patients to anticipate lower benefits of trial enrollment, making recruitment more costly. With the status quo recruitment infrastructure, representation of Black patients remains low—perpetuating doubt about whether trial findings extrapolate to them and generating a cycle of underrepresentation.

Although policies that break this cycle of underrepresentation may take many forms, we discuss case studies of successful investments in what we call inclusive infrastructure. We document considerable variation in trial representation across diseases and contrast two especially different cases: cancer and HIV/AIDS. Although research into both diseases is supported by large, coordinated networks with substantial federal investment, Black patients are well represented in HIV/AIDS trials and poorly represented in cancer trials, relative to population share and disease burden benchmarks. To understand the origins of these differences, we draw on interviews with clinical trials networks, qualitative research, and administrative data. We highlight two key features that differentiated HIV/AIDS trials: engagement with priority population communities from protocol design to recruitment, and site selection in and around safety net hospitals. These differences may explain its more representative evidence base and, more suggestively, its higher diffusion rates of new products.

Our work contributes to a growing literature that seeks to understand the role of innovation in creating or exacerbating inequality. Previous studies have focused on how endogenous (demand-pull) investment can affect the composition of resulting technologies. Most closely related is Cutler, Meara, and Richards-Shubik (2012), who find that allocation of NIH grant funding disproportionately flows toward majority groups when physicians “treat what they see,” widening health gradients in settings where disease burden differs across groups. Michelman and Msall (2021) highlight the harm from regulatory restrictions on women’s participation in early-stage clinical trials, which dampens patent activity for women-specific conditions. Other scholarship focuses attention on how product characteristics affect diffusion. Papageorge (2016) develops a dynamic structural model of demand for medical treatment when patients trade off health and work experience, illustrating how side effects associated with HIV medication could affect treatment decisions among employed persons. Hamilton et al. (2021) extend this model, describing more generally how patient preferences exert a demand externality, tilting innovation toward less efficacious drugs and lowering overall experimentation. We build on these important contributions by developing and testing an alternative link between innovation and inequality: we ask whether unequal representation in the R&D process can induce inequality directly by making it more difficult for people to extrapolate from the data to their situation.

We also contribute to a literature on race and trust. People from different backgrounds may have different experiences (i.e., different data to readily extrapolate from), and these experiences can lead to increased or decreased levels of trust that a variety of institutions are effective for them. Previous research has shown that differential beliefs in the returns to investment opportunities (Boerma and Karabarbounis 2023) contribute substantially to the persistence of the racial wealth gap (Derenoncourt et al. forthcoming). Research also indicates that historical exploitation, violence, and discrimination have led to distrust in the medical system and medical research (Alsan and Wanamaker 2018; Eli, Logan, and Miloucheva 2019), declines in home ownership (Albright et al. 2021), and reduced participation in political processes (Williams 2022). Our article provides a way to think about the consequences of these different experiences for trust more broadly, as the cycle of underrepresentation result applies to any process that includes a participation decision.

The remainder of the article proceeds as follows. Section II provides background information on clinical trials and relevant history. In Section III, we formalize how representative clinical trials may matter to patients and physicians. Section IV describes our two experiments. Section V presents our experimental results. We conclude by drawing lessons from case studies of successful efforts to improve representation in medical research.

II. Background

This section discusses the institutional context of clinical research, including trial financing and costs, the regulatory review process, and factors that shape enrollment. We describe how doctors and patients learn about new drugs and trial results. The features are incorporated into our framework. Online Appendix G provides additional details.

II.A. Clinical Trials Landscape

1. The Drug Development Process.

Before a new drug may be marketed in the United States, the FDA must deem it to be safe and effective. Sponsors seeking to obtain FDA approval typically conduct clinical trials—randomized evaluations of the new drug relative to a placebo or current standard of care (National Institutes of Health 2017). Data drawn from ClinicalTrials.gov, the largest global registry of clinical trials, suggest that private firms are the most frequent single primary sponsor of clinical trials (36%), an order of magnitude more frequent than U.S. federal agencies (3%).⁴ The remainder of clinical trials are sponsored by academic institutions, hospitals, and nonprofit organizations.⁵

The drug approval process begins when sponsors identify a promising lead compound—the core component of what will become a drug. Sponsors typically file initial patent applications on the drug just before beginning Phase I clinical trials.⁶ When firms begin clinical testing, they also file investigational new drug (IND) applications, which draw on data from preclinical testing. Patent terms are 20 years long, though firms may receive other forms of market exclusivity that can extend effective patent life.

Drug sponsors must complete three stages of clinical testing before applying for marketing approval. Phase I trials are intended to establish safety, determine appropriate dosages, and identify side effects. Phase II and III trials test efficacy, monitor safety, and compare the product to existing alternatives. Whereas Phase I trials often recruit a small number of healthy volunteers, Phase II and III trials recruit from the target patient population and may enroll thousands of people. Drug approval hinges on so-called pivotal trials, which are typically Phase III trials that aim to demonstrate efficacy.

2. The Cost of Clinical Trials.

Clinical research is expensive. Recent estimates suggest that the median cost of a pivotal clinical trial providing evidence of efficacy to the FDA is about $19 million (Moore et al. 2018).⁷ Industry reports suggest that the most expensive step of the clinical trial process is recruiting patient participants in Phases II and III (Sertkaya et al. 2014). Accrual rates—the speed with which a trial can recruit eligible patients—are cited as the most common reason for trial delays and, in some cases, failure. Slower accrual rates can lengthen clinical trial periods and erode patent life (Budish, Roin, and Williams 2015). Thus, trial sponsors aim to identify and enroll patients as quickly as possible, often contracting with third parties that specialize in clinical trial enrollment, and sometimes moving operations overseas where recruitment costs tend to be lower (Qiao, Alexander, and Moore 2019).

The cost—in terms of money and time—of enrolling a new patient in a trial also varies across demographic groups. Obtaining proprietary information of these costs is difficult; however, several published studies and our own qualitative interviews with stakeholders provide corroborating evidence that white patients tend to require fewer resources and are thus much lower cost to recruit (see Online Appendix A.2 for details). Efforts to reach out to non-white communities typically involve additional staff, tailored recruitment materials, and new relationships with health care networks—all of which contribute to a comparatively high cost per enrollee (Marquez et al. 2003).

Two additional pieces of evidence provide some quantitative information on the size of these cost differences. First, consider the case of Moderna, which ran one of the highest-stakes clinical trials in recent history for its first-generation SARS-CoV-2 vaccine. In September 2020, the company announced that enrollment was going to be slowed for the explicit purpose of improving representation of patients from racial and ethnic minorities in the trial. Moderna’s stock price fell 8% upon the announcement (Online Appendix Figure B5) (Tirrell and Miller 2020). A second illustration is the cost of recruiting experimental subjects for online surveys. In Online Appendix Figure B6, we plot price quotes for U.S.-based respondents that we received for our own study from three large survey firms. All three firms quoted higher prices to recruit Black respondents as compared to white respondents—with prices ranging from 4% to 130% more to recruit a Black respondent. We endogenize these cost differences and explore their effects in our conceptual framework (Sections III and VI).

3. Enrollment Patterns and Barriers to Participation.

The cost differences described above may play a role in explaining the trial enrollment patterns observed in Figure I. Black patients make up just 5% of trial enrollees in the median clinical trial—far less than the 13.6% of the U.S. population that they make up (U.S. Census Bureau 2021). This level has remained flat since data collection efforts began (Online Appendix Figure B1). Based on the Research!America survey data, Black Americans are less likely to have confidence in research institutions, to believe science benefits them, or to enroll in clinical trials (Table I).8 These findings mirror those of our own survey data: an analysis of open-text responses reveals that Black patients are more likely to cite trust, privacy, and racism as reasons not to enroll, whereas white patients cite logistical barriers and comorbidities (Online Appendix Figure B7).

TABLE I.

Views on Science and Clinical Trials among U.S. Respondents

	Black Respondents	White Respondents	Difference
	(1)	(2)	(3)
Confidence in research institutions	2.829	3.082	−0.253***
	(0.963)	(0.822)
Heard of clinical trial	0.796	0.875	−0.079***
	(0.374)	(0.339)
Would enroll in clinical trial if doctor recommends	0.783	0.837	−0.054***
	(0.384)	(0.379)
Trust not reason for lack of enrollment	0.432	0.536	−0.104***
	(0.463)	(0.514)
Science is beneficial	0.284	0.383	−0.099***
	(0.419)	(0.493)
Would get FDA-approved vaccine	2.907	3.069	−0.163
	(1.024)	(1.099)

Notes. The table reports the survey responses from Black and white U.S.-based respondents for a set of questions regarding science. Data are from a national survey conducted by the nonprofit Research!America over 2013, 2017, and 2021. Heard of Clinical Trial, Trust, and Science is Beneficial are dichotomous variables. Other variables are on an ordinal scale. See Online Appendix H for details on variable construction. Standard deviations are in parentheses.

^*, **, *** refer to statistical significance at the 10%, 5%, and 1% levels, respectively.

4. Clinical Trials Data.

Upon successful completion of the three phases of clinical trials, sponsors submit new drug applications (NDAs) to the FDA. Based on these data, the FDA determines whether the drug will be approved for sale in the United States and for which specific indications. Currently, the FDA only requires that a drug is proven efficacious for the “target population,” which in practice translates to patients with the targeted condition. Most trials are therefore powered to detect a mean difference in the primary endpoint between treatment and control groups and not to detect subgroup-specific treatment effects, which are not commonly reported (Green et al. 2022). The most common statistic reported in abstracts and quoted in advertisements is therefore a drug’s average treatment effect, as demonstrated in the trial. Demographic characteristics of the sample are typically provided in the first table (the balance table) of journal articles or in the short description of the study population in drug advertisements.⁹

5. The Market for New Drugs.

Although analogous approval processes occur worldwide, approval in the U.S. market is critical for pharmaceutical firms: U.S. sales were projected to account for nearly 50% of the $1.2 trillion in global pharmaceutical revenues earned in 2020 (IQVIA 2015) and a disproportionate share of pharmaceutical net income (Goldman and Lakdawalla 2018; Ledley et al. 2020). In particular, the United States currently lacks the price controls that other countries use to curtail spending and is permissive with respect to marketing. Given these features of the market, we focus on demand in the United States, among physicians and patients.

II.B. Demand for New Drugs in the United States

1. How Physicians Learn about New Drugs.

Randomized controlled trials are considered the gold standard for causal inference in medicine and have been since their popularization by the British Medical Research Council and subsequent adoption by the FDA in 1962 (Cochrane 1972). EBM is a step-by-step process that facilitates the “reasonable use of modern best evidence in making decisions about the care of individual patients” (Martí-Carvajal 2020, 1). EBM’s five steps aim to integrate clinical experience, patient values, and research findings (Blanco et al. 2014).¹⁰

After physicians complete their formal training, trial information is often accessed via multiple sources. These sources include ClinicalTrials.gov, which as of April 2019 received more than 215 million page views per month and 145,000 unique visitors daily. (See their website for additional details.) They also include academic journals, society or national practice guidelines, pharmaceutical representatives, medical conferences, and, more informally, online and in-person social networks. To maintain an active medical license, many primary care doctors participate in continuing medical education (CME). In addition to meeting requirements set by professional associations, doctors might wish to stay up to date with the literature for other reasons, including a desire to help their patients (Doximity 2014).

2. How Patients Learn about New Drugs.

Patients learn about new drugs mainly through their physicians and via advertisements. The United States and New Zealand are the only two countries that allow firms to market medications directly to patients (Schwartz and Woloshin 2019). Between 2016 and 2018, firms spent $17.8 billion on direct-to-consumer advertising (DTCA) associated with 553 unique drugs (U.S. Government Accountability Office 2021). Ads can be precisely targeted based on people’s search history and sometimes include links to clinical information. Patient advocacy groups in the United States are also key in disseminating information about new drugs—lists of trials and summaries of evidence exist for nearly all major categories of disease.

Perhaps in part because of this outreach, data from Research!America show that 80% of Black respondents and 88% of white respondents had heard of clinical trials (Table I). Moreover, we document in our survey of primary care physicians that 72% report having ever been asked by their patients about whether a new medication will “work in people like me.” The share of physicians asked this question on a regular basis is higher among those that treat Black patients (Online Appendix Figure B8).

Our theoretical framework considers beliefs and behavior of U.S.-based patient-physician dyads with access to information on average treatment effects and demographics from trials; we then report results from experimentally manipulating these two features of trials in Section IV.

III. Organizing Framework

The framework presented here formalizes how representation in the trial process affects perceived benefits of new drugs for patients and their doctors, yielding predictions we can then test experimentally. After presenting experimental tests of these predictions, we return to the framework in Section VI to try to understand why the underrepresentation of Black patients in clinical trials is so persistent.

III.A. Physicians and Patients

Physicians and patients use clinical trial information to understand the benefits of a new treatment and to inform decisions about participation in clinical research. Both agents are important end users of clinical trial information: physicians are the gatekeepers of prescriptions, whereas patients’ adherence behavior determines whether prescribed drugs will have the intended salubrious effect. To abstract from strategic interactions between physicians and patients and instead focus on the core issues surrounding consequences and causes of low representation, we make two assumptions that guarantee that a doctor’s decision of whether to prescribe a treatment (or recommend trial participation) aligns with a patient’s decision to adhere to the prescription (or participate in the trial). First, we follow the standard assumption that everyone shares a common prior. Second, we assume doctors are agents for patients and share their objective function.¹¹

1. Physician and Patient Beliefs.

The assessments of patient-doctor dyad $i$ are influenced by the current and historical trial data. Suppose the benefits to treatment for the patient in dyad $i$ equal $b_i\in \{0,\tilde{b}\}$ for $\tilde{b}>0$, where benefits are measured relative to not getting treatment. That is, the treatment either doesn’t work $\left(b_i=0\right)$ or works $\left(b_i=\tilde{b}\right)$, and $\tilde{b}$ parameterizes the stakes of the disease-treatment combination. The likelihood that the treatment works for a patient with characteristics $x_i$ is given by $\theta \left(x_i\right)\equiv \mathrm{Pr}\left(b_i=\tilde{b}\mid x_i\right)\in [0,1]$. Overall, the perceived benefit of treatment, ${\widehat{b}}_i$, is:

$${\widehat{b}}_i=\tilde{b}\times {\mathbb{E}}_i\left[\theta \left(x_i\right)\mid \text{trial data}\right],$$

where ${\mathbb{E}}_i[\cdot ]$ is the expectation of dyad $i$ on whether the treatment will work and this expectation is conditioned on data available at the time of the decision. The assumption that everyone applies the same (explicit or implicit) model of inference allows us to simplify the presentation of the model in two ways. First, the expectation operator is identical across all dyads and we can write ${\mathbb{E}}_i[\cdot ]$ as $\mathbb{E}[\cdot ]$. Second, the perceived benefit of treatment ${\widehat{b}}_i$ only depends on $i$ through $i$ s characteristics $x_i$ (i.e., it is not heterogeneous conditional on $x_i$), so whenever it does not cause confusion we write ${\widehat{b}}_i$ as a function of $x_i$ and the available data $h:{\widehat{b}}_i=\widehat{b}\left(x_i;h\right)$.

To focus and simplify the exposition, assume $x_i$ is unidimensional and in {0, 1}, where $x_i=0$ corresponds to “white” and $x_i=1$ to “Black.” As noted already, clinical trials rarely report subgroup analyses. Instead, data from a given trial $t\in \{1,\dots ,T\}$ consist of the combination of the average reported efficacy and fraction of Black participants, $\left({\overline{b}}_t,{\overline{x}}_t\right)$. Average efficacy is defined as ${\overline{b}}_t\equiv {\tilde{b}}_t\times \frac{k_t}{N_t}$, where ${\tilde{b}}_t$ denotes the benefits of the treatment if successful, $k_t$ the number of trial participants for whom the treatment was in fact successful, and $N_t$ the number of trial participants.¹² The fraction of Black trial participants simply equals $\sum _j\frac{x_j}{N_t}$, where the summation is taken over the trial participants. The complete history of trial data $h$ equals $h^{T-1}={\left({\overline{b}}_t,{\overline{x}}_t\right)}_{t=1}^{T-1}$ before treatment $t=T$’s trial is run and equals $h^T={\left({\overline{b}}_t,{\overline{x}}_t\right)}_{t=1}^T$ after. Our focus will be on beliefs about this treatment $t=T$ and, when it does not cause confusion, we omit the $t$ subscript when referring to it.

The key assumption underlying patients’ and doctors’ model of inference $\widehat{b}(\cdot )$ is that, in assessing the likelihood of treatment success for patients with characteristics $x_i$, they extrapolate more from data on patients with those characteristics than from data on patients with different characteristics. For patients, this could reflect learning from similarity, central to a wide variety of evidence-backed frameworks in psychology and economics.13 For doctors, this is consistent with evidence-based medicine (see Section II.B). Formally, people form beliefs about $\theta \left(x_i\right)$ and hence $\widehat{b}\left(x_i;h\right)$ by attaching probability $m$ to characteristic $x_i$ mattering.¹⁴ We then have

$$\widehat{b}\left(x_i;h\right)=m\times \left(\tilde{b}\times \mathbb{E}\left[\theta \left(x_i\right)\mid h,x_i\text{matters}\right]\right)$$

$$+(1-m)\times \left(\tilde{b}\times \mathbb{E}\left[\theta \left(x_i\right)\mid h,x_i\text{doesn′t matter}\right]\right).$$

To generate simple closed-form expressions for the above expectations, we assume priors over $\theta$ are in the beta family. If $\theta \left(x_i\right)$ is distributed according to beta distributions prior to the trial data for treatment $T$, with parameters $(\alpha \left(x_i;h^{T-1}\right)$, $\beta \left(x_i;h^{T-1}\right))$ conditional on $x_i$ mattering and parameters $(\alpha \left(h^{T-1}\right)$, $\beta \left(h^{T-1}\right))$ conditional on $x_i$ not mattering, then:

$$\widehat{b}\left(x_i;h^{T-1}\right)=m\times \underset{\text{posterior estimate of}\widehat{b}\text{conditional on}{x}_i\text{mattering}}{\underbrace{\left(\tilde{b}\times \frac{\alpha \left(x_i;h^{T-1}\right)}{\alpha \left(x_i;h^{T-1}\right)+\beta \left(x_i;h^{T-1}\right)}\right)}}$$ (1)

$$+(1-m)\times \underset{\text{posterior estimate of}\widehat{b}\text{conditional on}{x}_i\text{not mattering}}{\underbrace{\left(\tilde{b}\times \frac{\alpha \left(h^{T-1}\right)}{\alpha \left(h^{T-1}\right)+\beta \left(h^{T-1}\right)}\right)}}.$$

We set initial conditions for these parameters such that $\alpha \left(x_i,h^0\right)=\beta \left(x_i,h^0\right)=\alpha \left(h^0\right)=\beta \left(h^0\right)$ (i.e., in the absence of trial data agents assess the likelihood of treatment success as 0.5).

If clinical trial data are available, people form priors on the efficacy of novel treatments under investigation (more on this later), and update their beliefs once trial data on those treatments become available. We assume people attribute fraction ${\overline{x}}_T\left(x_i\right)$ of the overall number $k_T$ of successes reported in the trial to study participants with $x_i$, where ${\overline{x}}_T\left(x_i\right)$ equals the fraction of trial participants with characteristics $x_i$.¹⁵

Given this assumption, they update their beliefs from trial data on treatment $T$ according to Bayesian updating (see Online Appendix F for precise equations). As is standard, people end up placing some weight on the prior (given by $\frac{\alpha }{\alpha +\beta }$) and some on the empirical success probability in the trial $\left(\frac{k}{N}\right)$.

Proposition 1.

Supposing $m>0$ is fixed and average trial efficacy $\left(\frac{k_T}{N_T}\right)$ exceeds prior-belief ratios $(\frac{\alpha \left(x_i;h^{T-1}\right)}{\alpha \left(x_i;h^{T-1}\right)+\beta \left(x_i;h^{T-1}\right)}$ and $\frac{\alpha \left(h^{T-1}\right)}{\alpha \left(h^{T-1}\right)+\beta \left(h^{T-1}\right)})$, then

1. $\frac{\partial \widehat{b}\left(x_i;h^T\right)}{\partial k_T}>0$: the perceived benefit of a treatment to a patient is increasing in efficacy, as measured in the clinical trial.

2. $\frac{\partial \widehat{b}\left(x_i;h^T\right)}{\partial {\overline{x}}_T)}>0$: the perceived benefit of a treatment to a patient is increasing in the representation of patients with similar characteristics in the clinical trial.

3. $\frac{{\partial }^2\widehat{b}\left(x_i;h^T\right)}{\partial {\overline{x}}_T{\left(x_i\right)}^2}<0$: the degree to which increasing representation in a clinical trial positively impacts perceived benefits for group members is decreasing in the group’s existing trial representation.

Proof. All proofs can be found in Online Appendix F.5.

The intuition is straightforward: when a treatment works better than expected in the trial, people update their beliefs upward on treatment efficacy.¹⁶ But the degree to which they update depends on the (effective) sample size of the trial. Given that people place positive probability on characteristic $x_i$ mattering, the effective sample for patients with characteristics $x_i$ is increasing in their trial representation. Diminishing returns to representation follows from diminishing returns to sample size in (e.g., Bayesian) models of updating.

We assume posteriors from the most similar previous treatment become the prior for a novel drug.¹⁷ That is, letting the most similar past treatment to $T$ come in period $Z<T,\alpha \left(x_i;h^{T-1}\right)=\alpha \left(x_i;h^Z\right),\beta \left(x_i;h^{T-1}\right)=\beta \left(x_i;h^Z\right),\alpha \left(h^{T-1}\right)=\alpha \left(h^Z\right)\text{, and}\beta \left(h^{T-1}\right)=\beta \left(h^Z\right)$. Given this assumption, even when all groups begin with the same prior beliefs on efficacy at the beginning of time (in period 0), the underrepresentation of a given group will lead to a divergence in the perceived benefit of treatment over time (see Online Appendix F.4 for a numerical example). This divergence has important implications for behavior, described next.

2. Patient and Doctor Behavior.

Suppose that a patient with characteristics $x_i$ participates in a trial for treatment $T$ when she is invited to participate and

$$\widehat{b}\left(x_i;h^{T-1}\right)-n_T^{\text{trial}}+{\epsilon }_{iT}^{\text{trial}}⩾0,$$

where $n_T^{\text{trial}}$ equals the nonprice costs of participating in the trial (or convincing a patient to do so) and ${\epsilon }_{iT}^{\text{trial}}$ is a stochastic shock that is i.i.d. across $i$ according to a differentiable cumulative distribution function $F_{\epsilon }(\cdot )$.

Similarly, after a successful trial, a patient is treated for treatment $T$ when indicated and

$$\widehat{b}\left(x_i;h^T\right)-n_T-p_T+{\epsilon }_{iT}⩾0,$$

where $n_T$ refers to the non price costs of prescribing or adhering to treatment $T$, $p_T$ is the price (i.e., copay) for $T$, and ${\epsilon }_{iT}$ is a stochastic shock that is i.i.d. across $i$ according to $F_{\epsilon }(\cdot )$. Let

$$d\left(x_i;h^{T-1}\right)=\mathrm{Pr}\left(-{\epsilon }_{iT}^{\text{trial}}⩽\widehat{b}\left(x_i;h^{T-1}\right)-n_T^{\text{trial}}\right)$$

be the likelihood that a patient with characteristic $x_i$ participates in a trial when invited. Similarly, let

$$d\left(x_i;h^T\right)=\mathrm{Pr}\left(-{\epsilon }_{iT}⩽\widehat{b}\left(x_i;h^T\right)-n_T-p_T\right)$$

be the likelihood a patient with characteristic $x_i$ is treated for treatment $T$ when the treatment is indicated.

COROLLARY 1.

Given Proposition 1, a patient’s demand to participate in a given trial (or a physician’s decision to recommend a trial) is increasing in the degree to which patients who shared their (their patients’) characteristics were represented in previous trials $Z$ for which the average trial efficacy exceeded prior-belief ratios. Formally, for such trials $Z$,

$$\frac{\partial d\left(x_i;h^{T-1}\right)}{\partial {\overline{x}}_Z\left(x_i\right)}>0.$$

This result implies that a failure to represent groups in a trial today creates an intertemporal externality, as it becomes more difficult to recruit those groups in a trial tomorrow. Such less-represented group members perceive limited benefits from novel treatments relative to members of more-represented groups.¹⁸

Online Appendix F.3 formalizes two additional results on how beliefs affect behavior. First, Corollary F.2 shows that the comparative statics which Proposition 1 establishes for beliefs also hold for behavior: the demand for a new medication is increasing in the efficacy observed in the clinical trial and the representation of patients with similar characteristics in the clinical trial, with diminishing returns to the latter.¹⁹ Second, Corollary F.3 shows how historical and contemporaneous underrepresentation of Black patients in clinical trials creates a gap in the perceived benefits and demand for novel drugs between white and Black patients, where white patients have higher perceived benefits and demand relative to Black patients. It goes on to show how increasing Black representation in clinical trials closes these gaps. Table II summarizes our theoretical predictions and how they connect to our empirical results, which we turn to next.

TABLE II.

Summary of Theoretical Predictions and Empirical Results

Theory	Predictions	Exhibits	Result Summary
Prop. 1.1; Cor. F.2.1	Perceived benefits and demand for a new medication are increasing in trial-reported efficacy	Table III	A 1 std. dev. increase in efficacy increases physician prescribing intention by 0.28 std. dev.
Prop. 1.2; Cor. F.2.2	Perceived benefits and demand for a new medication are increasing in representation of similar patients in clinical trials	Table III	• For physicians, a 1 std. dev. increase in representation increases prescribing intention by 0.11 std. dev. • For Black patients, being assigned to the representative treatment increases self-reported relevance for their own care (“relevance”) and the likelihood that their posterior on efficacy is within a small neighborhood of the reported clinical-trial results (“loading on the signal”) by 0.78 std. dev. and 19.9 percentage points, respectively.
Prop. 1.3; Cor F.2.3	Diminishing returns to representation	Figure II, Panel D, Table III	• For physicians treating white patients (PWP), we fail to reject the null hypothesis that a decrease in white representation (from existing high levels) does not change prescribing intention. • For white patients, we fail to reject the null hypothesis that a decrease in white representation (from existing high levels) does not change relevance or loading on the signal.
Cor. F.3	• There are white-Black gaps in perceived benefits and demand for a new medication. • Increasing Black representation in clinical trials narrows these gaps.	Figure III; Figure IV; Figure V	• PWPs have a mean prescribing intention of 6.46, while PBPs who are exposed to nonrepresentative trials have a mean prescribing intention of 4.90. The prescribing intention of PBPs who are exposed to representative trials increases to 6.26 and is statistically indistinguishable from that of PWPs. • Black patients who are shown the low representation trial are 26 percentage points less likely to load on the signal than white patients. Black patients shown the representative trials are only 1 percentage points less likely to load on the signal than white patients and this difference is statistically indistinguishable from zero.
Cor. 1	Groups that were historically underrepresented in successful trials have a lower propensity to participate in trials today than historically well-represented groups.	Online Appendix Table C11; Section VI.B	• For Black patients, being assigned to the representative treatment increases their stated willingness to participate in similar future blood pressure studies by 0.39 std. dev. • Historically, HIV/AIDS trials were more representative than cancer trials. Recent HIV/AIDS trials are associated with a higher percent Black representation than recent cancer trials.

Notes. Formatting of the exhibits indicate the type of evidence: italics indicates causal evidence; boldface indicates descriptive evidence.

IV. Experimental Design

IV.A. Experimental Design

To test predictions from our theoretical framework, we conducted survey experiments—one with a sample of primary care physicians, and one with a sample of patients.²⁰ The experiments differed in important ways reflective of the different subject pools. Physicians, who are familiar with the task of evaluating new medications as part of standard practice, were asked to rate several hypothetical drugs. In each drug profile, the racial composition of the trial and efficacy were cross-randomized.²¹ Drug efficacy was used as a “numeraire” because it is widely considered the most important characteristic of a new medication. Prescribing intention and relevance for own patients for each medication were assessed. When surveying patients, a simpler exercise was presented: respondents were shown trial evidence associated with a single actual drug. Primary outcomes for patients included beliefs on the drug’s efficacy, relevance for own health, and willingness to “ask their doctor” about the new medication.²² We describe the experiments below and discuss common critiques of survey experiments and how we endeavored to overcome them in Section V.B.

1. Physician Survey Experiment.

We recruited physicians who met the following criteria: (i) actively practicing in primary care, (ii) practicing in an outpatient setting (i.e., excluding hospitalists), and (iii) holding either an MD or DO. We worked with a licensed vendor of the American Medical Association’s (AMA) physician masterfile to identify and contact eligible physicians. We verified that survey respondents met all three criteria with a set of screening questions at the outset of the experiment. We prespecified that the representativeness of the trial sample could interact positively with the demographic composition of the physician’s patient panel. Thus, to ensure suitable variation in the panel, we split ZIP codes into deciles by Black population, weighting each ZIP code by its total population, and requested that half of all physician contacts be pulled from the top decile, one-quarter from the bottom decile (these two deciles account for 15% of all primary care physicians), and one-quarter from the remaining deciles.²³ This sampling approach was motivated by the fact that the distribution of Black patients across geographies and providers tends to be highly concentrated (Bach et al. 2004; Chandra, Frakes, and Malani 2017).

We sent each physician a personalized email (to their professional email address) inviting them to participate in a study. The email originated from a Harvard email account. We embedded a message as email text, which noted that the purpose of the study was to collect physician views on clinical trials research, that the study had received IRB approval, that their data would be securely stored, and that the study was not funded by industry but for academic purposes (see Online Appendix Exhibit E1). The letter explained that the physician respondents would be asked to rate eight hypothetical drugs and would be compensated $100 for their participation.²⁴

Although the vignettes were hypothetical, the drugs were based on recently developed therapies to treat diabetes. We chose to focus on diabetes because it is a common condition that is typically managed by primary care providers, and several new therapies with novel mechanisms of action have recently been developed (American Diabetes Association 2020). There are no established guidelines that encourage different prescribing by race or ethnicity for patients with diabetes (Golden et al. 2012). However, there is a debate (as with other conditions) about the role that genetic ancestry plays in its incidence (Parcha et al. 2022).

After confirming eligibility and answering questions about their practice, physicians were shown eight unique drug profiles. Profiles were selected randomly without replacement (i.e., physicians never saw an exact duplicate) and drug names were selected from 15 alternatives.²⁵ At the top of each profile, we listed the generic name of a hypothetical drug, which we developed by following standard naming conventions (e.g., suffixes and prefixes) that convey information about a drug’s type. Profiles also included the drug’s mechanism of action, the study type, sample size, and sample demographics (see Online Appendix Exhibit E2 for an example of a profile and Online Appendix Exhibit E3 for a table listing the hypothetical drugs shown to participants). Profiles were randomly assigned an efficacy value ranging uniformly from a 0.5%–2.0% average reduction in A1c, conforming to typical values of FDA-approved oral antiglycemics (e.g., metformin typically reduces A1c by 1–2 percentage points) (Nathan et al. 2009; Wexler 2022), and a percent Black of trial subjects value ranging from 0% to 35%, with lower values oversampled as trial diversity is typically low (Knepper and McLeod 2018; Dornsife et al. 2019).²⁶ Note that only efficacy and percent Black varied across the profiles, with all else held fixed.²⁷ In each case, the trial type was listed as a double-blind active comparator trial, and the sample size was fixed at 1,500 participants.²⁸

After viewing each profile, physicians were asked to rate how relevant the findings from the trial were for their patients (akin to the EBM step) and how likely they would be to prescribe the drug for patients with poorly controlled diabetes in their care. Both outcomes were on a scale from 0 to 10.²⁹ After reviewing all drug profiles, respondents were asked about their confidence in extrapolating trial findings across demographic groups or geographies. In the final survey section, we asked questions about risk aversion, time preference, and altruism. We posed open-text questions used in sentiment analyses.

We sent a follow-up survey to physicians one to three weeks after they initially completed the survey. In the follow-up survey, we allocated $5 to each physician and asked how they would like to divide the amount between two real-world campaigns supporting recruitment efforts for clinical trials (see Online Appendix Exhibit E6). The first campaign aimed to boost trial participation among the American public at large, while the second campaign aimed to boost trial participation for underrepresented minority communities. Both campaigns were run by a nonprofit, the Center for Information and Study on Clinical Research Participation (CISCRP).

2. Patient Survey Experiments.

Patients were recruited from Lucid, an online survey platform frequently used in social science research and marketing (see Online Appendix H for more information on this platform). Respondents were told that the survey was designed to solicit their views on health care and understand the factors that affect their interest in health research. Eligibility criteria included: (i) self-reported non-Hispanic white or non-Hispanic Black race/ethnicity, (ii) at least age 35, and (iii) endorsement of a diagnosis of high blood pressure (alone or comorbid with other conditions). To verify that respondents had, in fact, been diagnosed previously with hypertension, they were asked to enter their latest systolic and diastolic blood pressure readings in an open-text field.³⁰ Any respondent entering nonsensical values for blood pressure was deemed ineligible. We focused on high blood pressure instead of diabetes because a larger share of adults in the United States suffer from hypertension (45%) than diabetes (15%), thus facilitating recruitment (Ostchega et al. 2020; Centers for Disease Control and Prevention 2021). For the experiment assessing the new medication, we introduced consequentiality by explicitly encouraging patients to answer truthfully, and noting that their responses would be used to generate a personalized report they could download and share with their primary care provider. Approximately 42% of patient respondents downloaded the personalized report.

We began the experimental module by providing basic details about the clinical trial process. Before randomization, we informed respondents that new medications to treat blood pressure are frequently studied by researchers. We noted that these new therapies typically aim to improve blood pressure control, reduce complexity, or decrease side effects from medication. We added that new medications may not be an improvement over previous therapies, and thus must be tested before they are widely available. Patients were then shown details about a new medication: a combination antihypertensive medication. We asked each patient whether they had heard of the new drug before (95% had not) and what they anticipated the effect of the medication would be on their systolic blood pressure (in units of mmHg).

Patients were then shown findings from an actual clinical trial. We randomly assigned respondents to see trial data from studies that enrolled different shares of Black patients. The medication we presented was tested in two separate locations: in one setting, the percent Black in the trial was less than 1%—approximately one-third of trials in the ClinicalTrials.gov database meet such a criterion—and in the second, the percent Black in the trial was 15%. Efficacy was strong and comparable in both settings, lowering systolic blood pressure by about 15 mmHg.³¹ We thus randomized only the percent Black in the trial, holding efficacy and all other parameters of the trial constant.

After being shown information on the drug’s efficacy and the randomized racial composition of the study, in text and graphic form, patient respondents were again asked to provide their beliefs about the drug’s efficacy. In addition, respondents reported how relevant the findings of the trial were to patients like them and whether they would be interested in “asking their doctor” about the medication.³² We also asked patients the same question we had posed to doctors about extrapolating from trials generically. If patients indicated that they were not confident in extrapolating, we asked them to describe the reasons for this limited confidence.

In the final sections of the survey, we inquired about trust, risk aversion, altruism, and time preferences. We also asked respondents to provide details about their current primary health care provider and current regimen for blood pressure management and medication adherence. We concluded with open-text questions and a reference to learn more about clinical trials.

Our survey experiment on clinical trial participation followed the above design but occurred several months later, using a separate group of patients. In this second study, the outcome of interest was a respondent’s stated willingness to participate in a new trial that was similar to the one they had been shown. After respondents provided this information, we asked multiple-choice questions designed to elicit views on the financial or medical consequences of trial participation, on whether the trial would produce new or relevant knowledge, on data privacy, and on researcher trustworthiness.

V. EXPERIMENTAL DATA AND RESULTS

V.A. Sample Characteristics

We invited 12,192 physicians to participate in the study.³³ Among those who passed the screening questions, 87% completed the survey (137 physicians); completion rates did not vary significantly across strata. Potential respondents were most commonly screened out if they were not practicing primary care physicians or if they were hospitalists (i.e., not outpatient providers). On nearly all dimensions, the characteristics of physicians in our sample are comparable to those of physicians in the same ZIP code strata in the AMA Masterfile (see Online Appendix Table C3), with the following exceptions: sample physicians from the top Black share decile stratum tend to be older and from higher-ranked medical schools, and physicians in other ZIP codes tend to have a higher share white population and a lower share Hispanic population.34

We recruited 275 patients diagnosed with hypertension to provide views on a novel treatment: 139 Black and 136 white respondents. Respondents are comparable to individuals with hypertension in the Medical Expenditure Panel Survey (MEPS); in Online Appendix Table C5, we document that Black and white respondents in our survey are broadly similar to MEPS respondents by age, geography, income, and insurance status, although there were relatively more female respondents. Black respondents had slightly higher levels of college education, and white respondents were less educated than in the MEPS data (Blewett et al. 2019).35 We recruited another 272 participants to the clinical trials participation experiment. There was no significant imbalance or differential attrition across arms for any survey (see Online Appendix Tables C2, C7, C8, C9, and C10).

V.B. Estimation and Results

To test whether increasing representation of Black patients (which we refer to simply as “representation”) in trials affects how physicians view study results and make prescribing decisions, we estimate the following equation:

$$Y_{jk}={\alpha }_0+{\alpha }_1{\text{Representation}}_{jk}+{\alpha }_2{\text{Efficacy}}_{jk}$$ (2)

$$+{\rho }_k+{\mu }_j+{\sigma }_{jk}+{ϵ}_{jk},$$

where $j$ denotes a drug and $k$ denotes a unique physician respondent, $Y_{jk}$ denotes our primary outcomes of interest: relevance for one's own patients and willingness to prescribe. Representation is the share of patients in a given trial who are Black. Efficacy captures the percentage point drop in measured hemoglobin A1c. Both efficacy and representation were cross-randomized in each profile. Our prespecified main estimating equation includes physician fixed effects $\left({\rho }_k\right)$, mechanism of action fixed effects $\left({\mu }_j\right)$, and indicators for the order in which profiles were shown $\left({\sigma }_{jk}\right)$, though we also present results without any controls. The outcome and randomized attributes are standardized. Standard errors are clustered at the physician level. We also prespecified heterogeneity, interacting trial demographics with those of the doctor's panel.

To test whether the racial composition of clinical trials affects patient beliefs and behavior, we estimate for patient $i$ of race $r$ the following:

$$Y_{i(r)}={\beta }_0+{\beta }_1{\mathrm{𝟙}}_{i(r)}^{\text{Representative}}+X_{i(r)}^{\prime }\Omega +{ϵ}_{i(r)},$$ (3)

where the indicator variable captures the difference between receiving the information that the percent Black of trial participants was 15% versus less than 1%. Recall that efficacy was held fixed, and all respondents saw the same drug. We estimate equation (3) separately by patient race for three outcomes: relevance, efficacy beliefs, and asking one’s doctor. Relevance (of the drug for oneself) is transformed from a Likert scale (0 to 10) to standard deviation units. Loading on Signal is an indicator equal to 1 if patients’ beliefs about personal efficacy are within 1 mmHg of the reported treatment effect in the trial.36 Ask Doctor is an indicator variable equal to 1 if patients indicate a desire to talk to their doctor about the drug.

1. Main Findings.

Table III presents our main results for both experiments: Panel A reports findings for physicians and Panel B for patients. In Panel A, columns (1) and (2) include only the randomized components of drug profiles. A 1 standard deviation increase in the reported efficacy of the drug—a reduction in A1c of roughly 0.44 percentage points—increases relevance and willingness to prescribe a medication by 0.165 and 0.229 standard deviation units, respectively. Conditional on the drug’s efficacy, a 1 standard deviation increase in percent Black—about a 10 percentage point increase in Black trial participants—increases relevance for patients by 0.163 standard deviation units and willingness to prescribe the drug by 0.179 standard deviation units. Columns (3) and (4) present our main specification (equation (2)). We find that representation affects both relevance and intent to prescribe, increasing them by approximately 0.11 standard deviation units.

TABLE III.

Physician and Patient Experimental Results on Effects of Increasing Representation

	Panel A: Primary care physicians
	Relevance	Prescribing	Relevance	Prescribing	Relevance	Prescribing
	No controls		Main specification		Share Black interactions
	(1)	(2)	(3)	(4)	(5)	(6)
Representation	0.163***	0.179***	0.109***	0.107***	0.007	−0.005
	(0.039)	(0.036)	(0.029)	(0.029)	(0.038)	(0.039)
Efficacy	0.165***	0.229***	0.189***	0.281***	0.179***	0.285***
	(0.038)	(0.039)	(0.029)	(0.032)	(0.036)	(0.043)
Representation × Patient percent Black					0.004***	0.004***
					(0.001)	(0.001)
Efficacy × Patient percent Black					0.000	−0.000
					(0.001)	(0.001)
p-value: Representation = efficacy			.057*	< .001***
p-value: Representation = $\frac{1}{2}$ (efficacy)			.655	.314
Doctor Fes	No	No	Yes	Yes	Yes	Yes
Profile order FEs	No	No	Yes	Yes	Yes	Yes
Rx mechanism FEs	No	No	Yes	Yes	Yes	Yes
Observations	1,096	1,096	1,096	1,096	1,096	1,096
Panel B: Patients
	Relevance		Ask Doctor		Loading on Signal
	Black patients	White patients	Black patients	White patients	Black patients	White patients
	(1)	(2)	(3)	(4)	(5)	(6)
Representative treatment	0.781***	0.172	0.021	0.006	0.199**	−0.057
	(0.167)	(0.159)	(0.077)	(0.079)	(0.083)	(0.086)
p-value: Black patients = white patients		.008***		.893		.030**
Control mean	−0.26	−0.23	0.70	0.70	0.33	0.59
Observations	139	136	139	136	139	136

Notes. Panel A reports OLS estimates for the outcomes of Relevance and Prescribing Intention on the sample of primary care physician respondents. Representation refers to the randomized percent Black in the trial unless otherwise indicated. Efficacy refers to the randomized percentage point drop in A1c. Prescribing Intention, Representation, and Efficacy are standardized to a mean of zero and a standard deviation of one. Columns (3) and (4) report results from the main specification (equation (2)). Columns (5) and (6) interact Representation and Efficacy with the reported percent of patients that are Black in the physician’s panel and the main effect is included but not reported. One hundred thirty-seven physicians participated in the experiment each assessing eight oral antiglycemic medications. Standard errors clustered at the physician level are in parentheses. Panel B reports OLS estimates from equation (3) on the sample of patient respondents. Relevance refers to relevance for own care and is standardized to a mean of zero and standard deviation of one. Loading on Signal is an indicator equal to 1 if the respondent’s posterior was within 1 mmHg of the signal (i.e., between 14 and 16) and 0 otherwise. Robust standard errors are in parentheses.

^*, **, *** refer to statistical significance at the 10%, 5%, and 1% levels, respectively.

The p-values displayed in the bottom rows of these last two columns indicate that—although we reject that the coefficients on representation and efficacy are equal—we cannot reject that representation has about half the effect of efficacy. In other words, physicians are approximately half as responsive to who was in the trial as they are to how well the drug works. The results in columns (5) and (6)—in which we include interaction terms between experimentally manipulated measures of representation and efficacy with each physician’s Black patient share—are key in understanding our results: the effect of increased Black representation on prescribing behavior is attributable to doctors who treat at least some Black patients. We observe no comparable (significant) interaction between doctors’ patient demographics and efficacy.

In Table III, characteristics of the physician’s patient panel enter linearly. Figure II explores these relationships nonparametrically by interacting quartiles of patient percent Black with the treatment and plotting the total effect (main effect plus interaction). Panel A shows the results for efficacy, demonstrating a relatively constant effect on relevance and prescribing across the percentage Black of patients. By contrast, in Panel B representation has a nearly linear and upward-sloping relationship: the higher percentage Black in a doctor’s patient panel, the more they respond to the inclusion of Black patients in the trial. Note that this line naturally begins at zero over the domain we test: there is simply a null effect (not a strong negative effect) of increasing Black representation among physicians who care mostly for white patients.

Figure II. Heterogeneity among Physicians by Racial Composition of Patient Panel.

The figure plots OLS estimates for two outcomes—Relevance (Panels A and C) and Prescribing Intention (Panels B and D)—from specifications estimated with interaction terms between each quartile of patient percent Black and either Representation or Efficacy. Fixed effects are residualized before estimating equation (2). The figure plots the linear combination of the main effect and the interaction with each quartile; quartile one is defined as the reference. Robust standard errors are clustered at the physician level. Ninety-five percent confidence intervals are displayed.

To provide further assurance that it is specifically the racial composition of the panel that is driving the heterogeneity, Online Appendix Figure B10 presents an omnibus test, in which physician-specific representation coefficients are regressed on panel demographic characteristics. A significant association exists only between the magnitude of the coefficient and the panel percent Black, with no strong relationship between representation and percent female, Hispanic, foreign-born, or senior citizen. Moreover, there is no significant relationship between physician-specific efficacy coefficients and panel percent Black, nor between the other demographic categories. Online Appendix Figure B11 demonstrates few associations between physician-specific responses to representative trials and their own background characteristics.

We next turn to findings from patients in Table III, Panel B. Recall that in this specification (equation (3)), the treatment is an indicator variable. We split the sample by patient race, with findings from Black patients displayed in the odd columns, results from white patients in the even columns, and a p-value of the difference between the two samples in the bottom even rows. Column (1) reports that Black patients with hypertension assess clinical trials with 15% Black participants as 0.781 standard deviation units more relevant than trials with less than 1% Black participants—holding drug name, mechanism, and reported efficacy constant. This result is statistically significant at the 1% level. Column (3) indicates that these higher assessments translate into a positive but statistically insignificant willingness to ask their physician about the medication. Column (5) reports that the representative arm is associated with a 19.9 percentage point increase in believing the drug would perform as well on oneself as in the trial. The results from white patients with hypertension are mixed in sign and never statistically significant (columns (2), (4), and (6)).

Results from our patient sample are also broadly consistent with the model’s prediction of diminishing returns to representation: representation matters for Black hypertensive patients, and does not (over the domain tested) for white patients, similar to what we find for prescribing intentions in Figure II. Taken together, the results suggest physicians are acting as good agents for their patients—combining the evidence on efficacy while also taking patient views into account (Ellis and McGuire 1986; Barnato 2017).

Last, we turn to our main results from the follow-up experiment, which investigated the relationship between beliefs about trial representation and willingness to participate in future clinical trials. Results are reported in Online Appendix Table C11, Panels A and B, column (1). We find that exposure to the treatment—data on a more representative trial—increases Black patients’ stated willingness to participate in similar future blood pressure studies by 0.385 standard deviation units. There was no significant effect for white patients and the difference in treatment effects across the two groups was significant (p-value = .038). We discuss potential mechanisms for these results later.

2. Representation and Disparities.

We assess whether increased racial representation in clinical trials can close gaps similar to those documented in Figure I. Figure III documents that— when the share Black of the trial is low—a gap emerges between Black and white patients shown identical information on drug efficacy. For Black hypertensive patients, beliefs about how much the drug will lower blood pressure are within 1 mmHg of the range of the reported clinical effect for 33% of respondents, compared to almost 60% of white hypertensive respondents. This difference is large and statistically significant. When the trial is more inclusive of Black patients, this gap closes. While the change for Black patients is dramatic, the effect on white patients is negligible. This result is also observed when plotting the distributions of prior and posterior views on drug efficacy—the latter under the different interventions. Before the information treatment, the prior distributions for Black and white patients are indistinguishable (see Figure IV, K-S test p-value = .960). Regardless of the trial arm they are assigned, white patients update substantially on trial results, reporting a perceived effectiveness for their own health that is similar to the study finding. In contrast, Black patients are more willing to accept that reported efficacy under study conditions captures the drug’s effectiveness for their own health when the sample is more representative (K-S test p-value = .026).

Figure III. Loading on Signal by Race and Treatment Status.

The figure plots the share of respondents who “Load on Signal”—whose posteriors are within 1 mmHg of the reported drug efficacy in our intervention (15 mmHg)—by race and treatment group. Load on Signal is an indicator variable that takes a value of 1 if the respondent’s posterior was between 14 and 16, and 0 otherwise. The x-axis reports values for two groups of respondents: nonrepresentative trials with < 1% Black patients and representative trials with 15% Black patients. Results are plotted separately by respondent race. Ninety-five percent confidence intervals are included.

Figure IV. Prior and Posterior Beliefs on Drug Efficacy by Patient Race and Trial Representation.

The figure plots the prior and posterior distribution of beliefs about the perceived efficacy of the new antihypertensive medication for the patient’s own condition by respondent’s race and assigned treatment status (trial shown is either nonrepresentative or representative). The signal on efficacy shown to patients (15 mmHg) is displayed as a black vertical line and was revealed to patients following elicitation of priors. A Kolmogorov-Smirnov test fails to reject the null that the priors are identical across race (p-value = .960). For Black patients, a Kolmogorov-Smirnov test rejects the null that the posteriors are identical across arms (p-value = .026). For white patients, a Kolmogorov-Smirnov test fails to reject the null that the posteriors are identical across arms (p-value = .789).

Our results can be visualized by examining the gaps in prescribing intention across physicians who treat different categories of patients. We divide the sample of physicians into two groups: physicians who treat Black patients (PBP) and physicians who treat white patients (PWP). We define these categories by using the reported characteristics of each physician’s reported panel and whether they treat above or below the sample median for the relevant racial group.

Figure V plots prescribing intentions across the physician types. Efficacy, as measured by A1c reduction, is shown on the x-axis and the mean prescribing intention for each efficacy bin is plotted on the y-axis. The upward-sloping line indicates that physicians serving all types of patients are more likely to prescribe medications that were randomly assigned higher rates of efficacy. If a trial has less than 5% Black representation (the current median share of Black participation in clinical trials) prescribing intention of physicians treating more Black patients lies below that of physicians treating white patients at every efficacy level. However, when trials become more representative, this gap is erased.

Figure V. Physician Prescribing Intention by Patient Composition and Trial Representation.

The figure plots the relationship between Efficacy and Prescribing Intention (on a 0–10 scale) by patient composition and percent Black of trial subjects in the profiles shown to physicians. PBP (physicians treating Black patients) denotes physicians who report above the median percent Black patients in their patient panel. PWP (physicians treating white patients) is defined similarly with respect to white patients. NR indicates nonrepresentative (< 5% Black in trial) whereas R indicates representative (≥ 5% Black in trial). Note that 5% is the median percent Black in clinical trials (see Figure I).

3. Understanding Mechanisms: Extrapolation.

Why does representation matter? The model in Section III.A captures the idea that extrapolation from trial data is facilitated by the similarity between patient characteristics and the trial sample. We probe that assumption by asking physicians and patients how confident they are that a drug found to be safe and effective in a study of white patients would be safe and effective for Black patients. Confidence is measured on a scale of 0 to 3 ranging from “Not confident at all” to “High confidence.” As such a question is likely to be less informative for white patients, who are typically well represented in clinical trial evidence, we also asked respondents about how confident they are about the effectiveness of a drug approved on the basis of evidence generated entirely outside of the United States. Such a scenario mirrors a recent trend of “offshoring” clinical trials (Petryna 2009).

For all respondents who were not highly confident about extrapolating—which turns out to be the vast majority—we sought to understand the rationale for their beliefs. In particular, we asked why they believed that a drug tested on one sample would not work equally well in a different context. We provided a set of multiple-choice responses that allowed respondents to indicate concerns about biological factors, socioeconomic and environmental factors, or trust in the trial. Participants were also allowed to select “other” and asked to provide open-text answers.

Results are reported in Table IV. Panel A presents views from Black patients and doctors who treat them regarding extrapolation across race. Panel B presents views from white patients and doctors who treat them regarding confidence in extrapolating across geography. Each cell demonstrates the percentage of respondents who fall into that category. We find three broad patterns. First, few people fall into the highest confidence category for this exercise: ranging from 7.0% among PBP to 15.4% among PWP. Second, patients are less confident extrapolating on average than physicians: the mean level of confidence for Black and white patients is 1.0 (std. dev. 0.97) and 1.3 (std. dev. 0.91), respectively. For physicians treating these groups, the values are 1.72 (std. dev. 0.65) and 1.91 (std. dev. 0.65), respectively. In both instances, confidence among white patients and their doctors (Panel B) is slightly higher than their counterparts in Panel A. Third, when providing a rationale for why a drug might work differently across samples, a nontrivial share selected biological factors, though the most commonly chosen answer was socioeconomic and environmental factors.

TABLE IV.

Extrapolation from Clinical Trial Data among Physicians and Patients

	Panel A: Black patients and their physicians (PBPs)
White to Black patients	Confidence				Rationale
	Not at All	Some	Moderate	High	Perceived Biol. Factors	Perceived Social & Envir. Factors
	(1)	(2)	(3)	(4)	(5)	(6)
Black patients	39.6%	28.1%	25.2%	7.2%	31.0%	45.7%
PBPs	3.5%	28.1%	61.4%	7.0%	32.1%	45.3%
		Panel B: White patients and their physicians (PWPs)
Offshored to U.S. patients	Confidence				Rationale
	Not at All	Some	Moderate	High	Perceived Biol. Factors	Perceived Social & Envir. Factors
	(1)	(2)	(3)	(4)	(5)	(6)
White patients	21.3%	36.8%	32.4%	9.6%	19.5%	43.9%
PWPs	1.5%	21.5%	61.5%	15.4%	10.9%	70.9%

Notes. The table reports clinical trial data extrapolation confidence and rationale among patients and physicians. Panel A reports confidence in extrapolation across race among Black Patients and PBPs. Panel B reports confidence in extrapolation across geography among White patients and PWP. Columns (1)–(4) report the percentage of respondents at each confidence level. If a respondent did not select “High” confidence in extrapolation, they were asked to provide a rationale. Column (5) reports the percentage of respondents who cite perceived biological factors as the rationale for not having “high” confidence in extrapolation. Column (6) reports the percentage of respondents who cite perceived social and environmental factors as the rationale for not having “high” confidence in extrapolation. For each subgroup (Black patients, white patients, PBPs, PWPs), Online Appendix Table C14 reports confidence and rationale for both extrapolation questions (race and geography). PBP denotes physicians who report above the median percent Black patients in their patient panel. PWP is defined similarly with respect to white patients.

Several doctors selected “other,” and their open-text responses are reproduced in Online Appendix Table D1. When discussing extrapolation across race, doctors mention external validity, skepticism with results not obtained from representative samples, or a normative desire for the inclusion of diverse populations. With respect to foreign trial data, similar concerns were raised, though physicians also wondered about standards for studies performed abroad. One respondent noted that the ease of extrapolation depends on where the study took place, stating, “It would depend upon the country. I would expect Western European and Canadian trials to be similar to my particular patient population.”

Returning to the experimental results, we find that Black patients who view others as trustworthy were significantly more likely to want to ask their doctor about the new medication (Online Appendix Table C13, column (3)). In addition, we find that the representative treatment increases Black patients’ willingness to participate in future clinical trials, as well as their views on the trustworthiness of the trial researchers (Online Appendix Table C11, Panel A, column (2)). The same pattern does not hold for white patients (Panel B, column (2)).

4. Threats to Internal Validity.

Concerns with survey responses as outcomes include social desirability or experimenter demand effects. As mentioned already, we added consequentiality to both the physician (i.e., reporting findings on trial preferences to federal agencies) and patient (i.e., sharing personalized reports with their doctors) experiments. The majority of physicians and nearly half of all patient respondents requested access to these reports, suggesting that participants indeed valued them. For the patient survey, all respondents had been diagnosed with hypertension and thus had limited incentives to distort their responses to information about a new drug of potential health benefit for their specific condition. Our results on subsamples of respondents who asked for the reports are similar to those presented already (see Online Appendix Tables C15 and C6 for experimental results from physicians and patients, respectively). Online Appendix Tables C4 and C16 show that patients and doctors who downloaded or requested the report are statistically similar to other respondents.

The second key feature that reduces concerns about social desirability or experimenter demand effects is that we prespecified heterogeneous effects by the patient’s race and the racial composition of the provider’s patient panel. If social desirability was playing a large role, patterns might be similar across Black and white patient respondents and across doctors treating all types of patients. In terms of experimenter demand, the patients were only shown one trial so it would have been difficult for them to discern the rationale for the study. Indeed, a word cloud of responses to the open-ended question “What do you think this study was about?” shows only limited references to race or diversity (see Online Appendix Figure B12), with the dominant response being “blood pressure.” Similarly, information presented in our physician survey closely resembled the demographic information presented in biomedical publications and regulatory publications (e.g., the FDA Drug Trial Snapshots database).

We follow Kuziemko et al. (2015) and Elías et al. (2019), who use donations and petitions to validate survey responses, and ask physicians to make a decision about a donation in a follow-up survey.³⁷ Our follow-up donation survey finds that the amount physicians allocate to the enrollment campaign targeting underrepresented minorities is strongly and significantly associated with physician-specific coefficients on representation (Table V) and not with physician-specific responsiveness to efficacy. Because the donation question was fielded to physicians as a follow-up question released one to three weeks after they completed the survey experiment, the results also suggest that our findings are unlikely to be driven by experimenter demand.

TABLE V.

Association between Physician-Specific Coefficients and Trial Donations

	(1)	(2)
Coefficient on representation	1.279***	1.229***
	(0.449)	(0.436)
Coefficient on efficacy		0.199
		(0.621)
Constant	3.534	3.485
Observations	82	82

Notes. The table reports OLS estimates from a regression of physician-specific coefficients for representation and efficacy on dollars donated to a campaign to increase the representativeness of clinical trials. Physicians were asked to indicate, out of a possible $5, how many dollars they would like the research team to donate to a campaign that advocates for increases in clinical trial representation versus a campaign that advocates for increases in participation in clinical trials more generally. Observations are at the physician level. Robust standard errors are in parentheses.

^*, **, *** refer to statistical significance at the 10%, 5%, and 1% levels, respectively.

5. Threats to External Validity.

There are several potential concerns about mapping our survey results to real-world behavior. First, we may prime people to think about something obviously bad, which might affect their survey responses. Second, we may induce patients to construct beliefs on-the-fly about something (clinical trials) they are not well informed about. Third, features of trials may not alter real-world prescribing or medication adherence decisions, even if people do know about clinical trials.

Regarding the notion that we used an obviously negative prime (underrepresentation) for Black respondents, this presumes that ex ante we had access to our ex post results. Recall that our null hypothesis was that representation did not matter, which is precisely what we can now reject. Thus, we view our design as making underrepresentation—a widely known aspect of medical research—especially salient in the context of the survey experiment. We also ask an open-text question to our patient respondents immediately after the intervention about the rationale for their responses; sentiment analysis reported in Online Appendix Table C18 indicates no significant difference in positive affect across race groups. Furthermore, the time spent on the survey does not differ across those groups.

Of course, if patients are unaware of clinical trials and our surveys elicit responses that then do not map onto real behaviors, our findings are less relevant. However, data from Research!America and our own follow-up survey indicate that patients are, in fact, aware of clinical trials and that Black patients believe that they are not well represented in trial samples. Returning to the Research!America data in Table I, column (1) indicates that on average, 80% of Black respondents report that they have heard of clinical trials.

Regarding whether information on trial representation matters in practice, we document that it affects prescribing intention and updating from trial results. In settings outside of our experiments, evidence that Black Americans are skeptical of research institutions and medical technologies—FDA-approved and investigational—is widespread. We tabulate survey respondents consistent with these patterns in Table I. Qualitative comments from physicians in our study, as well as those drawn from a recent NASEM report, also suggest that representation plays a role in how doctors practice medicine (see Online Appendix Tables D1 and D2 and Online Appendix Figure B13).

6. Robustness.

We probe the robustness of our findings for physicians in Online Appendix Table C15. Columns (1) and (2) indicate that we obtain similar results when we use nonstandardized versions of the outcomes. We replicate our main findings with standardized prescribing as the outcome in column (3) and show that our findings are largely unchanged when restricting the sample either to physicians who answer our follow-up donation question or to those who request a copy of our report to NIH and NASEM (see columns (4) and (5)). Column (6) shows that findings on representation are not sensitive to the addition of controls selected using double-selection LASSO linear regression (Chernozhukov et al. 2018). We also find that the order of profiles presented to physicians does not substantially affect how they respond to the treatment (Online Appendix Figure B14).

Additional results from our physician sample are presented in Online Appendix Table C17. Column (1) reports our main results from equation (2), while column (2) assesses whether representation and efficacy are substitutes or complements by adding an interaction term; we find no evidence of either.38 Columns (4)–(6) indicate that our finding of substantial heterogeneity by Black patient representation in one’s panel is insensitive to varying definitions of physicians who treat Black patients. Our finding of a strong interaction between representation and reported patient percent Black (from Table III and replicated in column (3)) is robust to dichotomizing patient percent Black at the median and to defining physicians treating Black patients using ZIP code–level statistics obtained from the U.S. Census Bureau. In Online Appendix Figure B.15, we present further tests of robustness, including results from alternative specifications and on the sample of observations with at least one efficacy duplicate, and show that our finding of a significant coefficient on representation withstands all these tests.

We report robustness checks for our patient experiment in Online Appendix Table C6. Panel A demonstrates that results across our three outcomes are unchanged when we restrict to patients who requested the personalized report we offered, whereas Panel B shows that our findings are robust to weighting patients using person weights obtained from MEPS. Panel C indicates that our results are robust to including LASSO-selected controls.

VI. Discussion and Conclusion

The theoretical and experimental analysis sheds light on the potential benefits of increasing representation of Black patients in clinical trials to patients and pharmaceutical companies. Given these benefits, why does such underrepresentation persist?

One hypothesis would be that this underrepresentation persists because of a combination of a relative lack of information and distrust of doctor recommendations between Black and white patients. However, the racial participation gap in clinical trials is much larger than would be implied by the observed gaps in trust and information reported in Table I.³⁹ This section uses a combination of theory and case studies to analyze why this gap is so persistently large, extending the earlier theoretical and experimental analysis to study the costs and benefits to firms conducting clinical trials. In the process, this section fleshes out a potentially important intertemporal externality associated with a history of underrepresentation.

VI.A. Why Might Underrepresentation Persist?

Suppose pharmaceutical firms seek to maximize the expected profit from a given experimental drug trial and can choose their recruitment strategy (see Online Appendix F.2 for details). They have access to a status quo technology for recruiting patients to clinical trials. Under this technology, a racial gap in perceived treatment benefits increases the racial gap in trial participation relative to the gap in trial recruitment (Proposition F.3). In other words, firms using the status quo technology anticipate a higher refusal rate from Black versus white patients. Firms could choose to incur a fixed cost $f>0$ to increase Black representation from its level under the status quo by making investments that reduce the marginal costs of inviting more Black participants. We refer to these investments as building “inclusive infrastructure.” Our theoretical and empirical results suggest firms would see value from such investment: due to diminishing returns to representation, it could increase demand among Black patients and their doctors without significantly decreasing demand among white patients or their doctors. However, the returns to such investment may not be completely internalized by any given firm: it increases perceived benefits for all similar treatments in the future, including those developed by other firms.⁴⁰ The externalities a firm’s current recruitment decisions have on other firms’ future recruitment costs enables a cycle of underrepresentation.

Proposition 2.

Suppose the most similar treatment $Z$ to $T$ out-performed patients’ prior expectations. When the fixed costs $f$ to deviating from the status quo recruitment technology to inclusive infrastructure are sufficiently large, then underrepresentation of Black patients in the historical trial leads to further underrepresentation of Black patients in the current trial:

$$\frac{\partial {\overline{x}}_T}{\partial {\overline{x}}_Z}>0.$$

This result flows from the externality described above and is illustrated with a numerical example in Online Appendix F.4.

Together, the theoretical and empirical results (summarized in Table II) are suggestive of a cycle of underrepresentation. (i) Trials in the past have not been representative of Black patients. (ii) The lack of representation decreases the perceived benefits of treatments for Black patients and physicians who treat them. (iii) The first two items make it more costly for firms to increase trial representation actively. (iv) Trials today are not representative for Black patients.⁴¹ (v) The cycle continues.

VI.B. Case Studies

The theoretical analysis suggests that investments in inclusive infrastructure may help break such a cycle of underrepresentation. Here, we combine quantitative and qualitative evidence, including insights drawn from informal interviews with experts in trial design, to tighten the links between our theoretical and empirical findings and real-world practice.

Figure VI, Panel A plots the median percent Black in pivotal trials across the most common diseases or conditions in the United States.42 Black patients are underrepresented relative to their population share across most conditions, and underrepresented relative to disease burden as well (see Online Appendix Figure B16), although there is significant variation across conditions. In Panel B, we document that higher representation of Black patients in clinical trials is associated with higher outpatient prescriptions of new drugs to Black Americans across various conditions.

Figure VI. Trial Representation by Condition and Association with New Drug Prescribing.

Panel A plots the median share of Black patients in trials across HIV/AIDS and the 10 leading causes of death (excluding unintentional injuries and suicide) in the United States (Heron 2021). Data on trial composition are from ClinicalTrials.gov. Panel B plots the correlation between the prescription rate of new medications to Black Americans and the median percent Black in pivotal trials. We construct the prescription rate as the percentage of newly marketed drugs (on the market for five or fewer years) received by Black Americans in each major condition category. In Panel B, the y-axis value of Cancer includes outpatient cancer supportive therapies. CLRD, Diabetes, Heart, Kidney, and Flu/PNA indicate chronic lower respiratory diseases, diabetes mellitus, diseases of heart, kidney diseases, and influenza and pneumonia, respectively. Prescription data are from the Medical Expenditure Panel Survey. Observations associated with cancer and HIV/AIDS are denoted with diamonds (purple). See Online Appendix H for details.

Next, we focus on cancer and HIV/AIDS (purple diamonds in Figure VI, Panel B), which are instructive to compare for several reasons. Both disease areas benefit from decades of federal investments into research networks across the United States by the National Cancer Institute (NCI) and National Institute of Allergy and Infectious Diseases (NIAID), respectively.⁴³ Federal investments into these networks are comparable, totaling $6.54 billion into NCI and $6.05 billion into NIAID in 2021 (Congressional Research Service 2022).

The history of these research networks—and their specific forms of investment—shed light on differences in contemporary outcomes across disease areas. Investment in cancer research has historically been driven by top-down investments into academic medical centers, including efforts in the “war on cancer” that began with the National Cancer Act of 1971 (Mukherjee 2010). Beginning in 1972, motivated by a Howard University study documenting “an astounding increase in cancer mortality among the nation’s Black population in recent years,” the NCI invested in efforts to understand the burden of cancer mortality across racial groups (Henschke et al. 1973; Wailoo 2011). After the passage of the 1993 NIH Revitalization Act, investigators receiving NCI funding reported that they were struggling to comply with new rules regarding minority representation in clinical trials because NCI funding could not be used for “ancillary” study costs, including reimbursements for patient expenses, resources for advertising and outreach, and funding for patient navigators and counselors.

In contrast to the top-down development of federal cancer research infrastructure, research into HIV/AIDS has been shaped by community involvement and activism. Activists pushed researchers to alter standard protocols for research, calling for accelerated approvals and emergency access to medicine, introduction of surrogate endpoints that could proxy for other clinical markers, and greater emphasis on representation in trial recruitment (Epstein 1996). In parallel, political, religious, and community leaders worked to combat the stigma associated with links between HIV/AIDS and homosexuality, especially in Black communities, thus creating opportunities for individuals to seek access to experimental therapies (Robertson 2006; Royles 2020). At a 1990 community forum on clinical trials held in San Francisco, ACT UP/San Francisco member Michelle Roland called for a “revolution in clinical trial design,” in which activists and scientists designed “realistic clinical trials that do a better job of meeting people’s needs” (as recounted in Epstein 1996, ch. 7). In response to demands from activists, the AIDS Clinical Trial Group and the NIAID adopted the practice of seeking community involvement at each trial site when developing protocols, prioritizing long-term relationships outside of academic medical centers (Kagan et al. 2012).

Table VI substantiates these anecdotes and makes clear how site selection shapes trial composition. Among U.S.-based trial sites listed in the ClinicalTrials.gov database, sites that enroll for HIV/AIDS are approximately 11 (16) percentage points more likely to be located at a safety net hospital than sites that recruit for cancer (Alzheimer’s Disease Related Dementias ADRD). Unsurprisingly, the demographic characteristics of the trial sites also differ. Online Appendix Tables C20 and C21 report information on the demographics of HIV/AIDS, cancer, and ADRD research centers at the hospital service area level for all clinical trials and for specific networks.44 Trial sites recruiting for cancer have, on average, a 10.5 percentage point higher share of non-Hispanic white population and a 3.0 percentage point higher share of those with private health insurance than trial sites recruiting for HIV/AIDS.45

TABLE VI.

Trial Sites and Safety Net Hospitals

	DSH Index		UCMP Care
	(1)	(2)	(3)	(4)
HIV/AIDS (cancer comparison)	0.110***		0.019***
	(0.008)		(0.007)
HIV/AIDS (ADRD comparison)		0.161***		0.054***
		(0.012)		(0.010)
Constant	0.475	0.423	0.176	0.141
Observations	197,240	6,804	182,929	5,997

Notes. The table reports OLS estimates from a regression of an indicator for whether a trial site is located at a safety net hospital. Each observation represents a specific site associated with a unique clinical trial and the data are limited to cancer, HIV/AIDS, and Alzheimer’s disease and related dementias (ADRD) trials. Following Popescu et al. (2019), we define a safety-net hospital as a hospital in the state’s top quartile of Medicaid and Medicare Supplemental Security Income inpatient days historically used to determine Medicare Disproportionate Share Hospital (DSH) payments (columns (1) and (2)); and uncompensated (UCMP) care costs (as a percentage of total operating expenses) (columns (3) and (4)). See Online Appendix H for more detailed definitions of these variables. HIV/AIDS (cancer comparison) is an indicator variable equal to 1 if a trial site studies HIV/AIDS and 0 if a trial site studies cancer. HIV/AIDS (ADRD comparison) is an indicator variable equal to 1 if a trial site studies HIV/AIDS and 0 if a trial site studies ADRD. See Online Appendix Table C19 for a Cancer (ADRD comparison). Trial site information is drawn from Clinical-Trials.gov. See Online Appendix H.1.1 and H.3.8 for details. Robust standard errors are in parentheses.

^*, **, *** refer to statistical significance at the 10%, 5%, and 1% levels, respectively.

Site selection is just one part of the R&D process: protocol development is another important step and also differs across conditions. Since 1990, the Division of AIDS (DAIDS) at the NIAID has required that trial protocols include explicit community engagement plans, developed in conjunction with standing community advisory boards (Strauss et al. 2001).⁴⁶ The advisory boards meet regularly with trial investigators and consult on proposed protocols. Our discussion with HIV Vaccine Trials Network leadership suggests that DAIDS requirements have important spillover effects: although firms are not obligated to comply, industry sponsors often engage with communities to benefit from existing recruitment networks.

The stark differences in trial composition for cancer and HIV/AIDS highlight the extent to which active, large-scale investments in inclusive infrastructure, in addition to incentives, can be important for improving health disparities. Figure VI, Panel B demonstrates a positive relationship between greater representation in trials and prescribing rates.⁴⁷ This descriptive finding is robust to dropping HIV/AIDS (see Online Appendix Figure B18), although the main takeaway from this section is that HIV/AIDS is an “outlier” on many dimensions and therefore a potentially useful template for industry and regulators.

VI.C. Concluding Comments

Motivated by persistent, substantial racial disparities in both clinical trial enrollment and prescriptions for new drugs, we investigated the consequences and causes of underrepresentation of Black patients in medical research. Consistent with a theoretical model of similarity-based extrapolation, Black patients, and the physicians who treat them, find trial evidence less relevant for their care, and are less likely to prescribe medications, when experimental samples are not representative. However, when the evidence base is more racially representative, these gaps close. The results suggest that a feedback loop exists between representation in a process and subsequent decision-making. Such a cycle of underrepresentation could apply more widely to any data-driven participation or take-up decision.

DATA AVAILABILITY

The data underlying this article are available in the Harvard Dataverse, https://doi.org/10.7910/DVN/VB5MDJ (Alsan et al. 2023).