Black representation in critical care randomized controlled trials: a meta-epidemiological study

Abstract

Purpose

The under-representation of Black people within critical care research limits the generalizability of randomized controlled trials (RCTs). This meta-epidemiologic study investigated the proportionate representation of Black people enrolled at USA and Canadian study sites from high impact critical care RCTs.

Source

We searched for critical care RCTs published in general medicine and intensive care unit (ICU) journals between 1 January 2016 and 31 December 2020. We included RCTs that enrolled critically ill adults at USA or Canadian sites and provided race-based demographic data by study site. We compared study-based racial demographics with site-level city-based demographics and pooled representation of Black people across studies, cities, and centres using a random effects model. We used meta-regression to explore the impact of the following variables on Black representation in critical care RCTs: country, drug intervention, consent model, number of centres, funding, study site city, and year of publication.

Principal findings

We included 21 eligible RCTs. Of these, 17 enrolled at only USA sites, two at only Canadian sites, and two at both USA and Canadian sites. Black people were under-represented in critical care RCTs by 6% compared with population-based city demographics (95% confidence interval, 1 to 11). Using meta-regression, after controlling for pertinent variables, the country of the study site was the only significant source of heterogeneity (P = 0.02).

Conclusion

Black people are under-represented in critical care RCTs compared with site-level city-based demographics. Interventions are required to ensure adequate Black representation in critical care RCTs at both USA and Canadian study sites. Further research is needed to investigate the factors contributing to Black under-representation in critical care RCTs.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12630-023-02462-x.

Black people and other racialized populations disproportionately experience critical illness.¹ Rates of sepsis and heart failure are also higher in Black people, even after adjusting for differences in poverty and region of residence.^2,3 Black people who are hospitalized for chronic conditions experience worse clinical outcomes than people who are not Black do.³ Although there has been an overall decline in stroke mortality since the 1950s, mortality after stroke in Black patients has remained higher than in non-Black patients.⁴ Within the context of the COVID-19 pandemic, Black Americans were 1.1 times more likely to contract COVID-19 compared with the national rate, 2.8 times more likely to be hospitalized, and 1.9 times more likely to die.⁵ Compared with White patients, Black patients admitted to the intensive care unit (ICU) were younger and had a higher severity of disease.⁶ Despite this, Black patients had a shorter adjusted length of stay and less resources were used for their care during their first seven days of hospitalization compared with White patients.⁶ Addressing these disparities in critical illness is an urgent imperative for communities, health systems, and national critical care organizations.^1,7,8

High-level evidence would aid in addressing the disparities in critical care. Structured randomized controlled trials (RCTs) coupled with adequate representation would generate generalizable high-level evidence to inform real-world clinical practice. Randomized controlled trials are the gold standard for producing trustworthy evidence to improve health care and outcomes for critically ill adults.⁹ Randomized controlled trials, however, cannot optimally inform care for patient populations who are not adequately represented in the trial population. The external validity of RCTs in critical care has been criticized for the under-representation of certain populations, including pregnant patients, people experiencing multiple medical conditions, and notably, racial and ethnic minorities.^10–12 According to a 2020 Health Information National Trends Survey, Black respondents had 72% decreased odds of clinical trial participation than White respondents did.¹³ The Committee on Comparative Effectiveness Research has declared racial and ethnic disparities in health care delivery as one of the top research priorities in the USA.¹⁴

Under-representation of racial or ethnic minorities in critical care RCTs is a multifaceted issue. One way to address this concern is to ensure adequate representation in critical care clinical trials as it maximizes external validity and informs optimal care. Nevertheless, before investigating ways to achieve adequate representation, we must establish the extent of under-representation, if any, within the racial group of interest. As such, the primary objective of this meta-epidemiological study was to determine the proportionate representation of Black people in recently published high-impact critical care RCTs performed at USA or Canadian sites. To achieve this objective, we conducted a systematic review and meta-analysis comparing site-based enrolment demographics to corresponding city-based demographics. As a secondary objective, we used meta-regression to explore the impact of study characteristics on Black representation.

Methods

This meta-epidemiological study examined representation of Black clinical trial participants across high-impact ICU trials using meta-analysis. We used meta-regression to assess heterogeneity and explore potential subgroup effects. We established a study protocol a priori (Electronical Supplementary Material [ESM] eAppendix 1). As a meta-epidemiological study, the predeveloped protocol was not eligible for external registration on repositories such as PROSPERO. While the protocol was not registered, we have included it in the ESM for transparency.¹⁵ The search, screening, and data processing were conducted according to the Cochrane Handbook for Systematic Reviews.¹⁶

Data sources and searches

With the assistance of an experienced health sciences librarian, we developed a search strategy to include key terms pertaining to critical care literature (ESM eAppendix 2). We searched the following journals for any RCT published between 1 January 2016 and 31 December 2020: New England Journal of Medicine (NEJM), Lancet, American Journal of Respiratory and Critical Care Medicine, Journal of the American Medical Association (JAMA), Intensive Care Medicine (ICM), CHEST, and Critical Care Medicine. These journals were selected to include both general medicine journals and ICU-specific journals and capture the highest impact critical care RCTs published within the five-year period.

Study selection

We included English language RCTs that examined ICU interventions and were published between 1 January 2016 and 31 December 2020. Randomized controlled trials had to enrol only adult patients (> 16 yr old) and include at least one study centre in the USA or Canada. Studies had to report site-by-site breakdown of racialized data for enrolled patients to meet eligibility criteria. We excluded trials performed outside the ICU and study designs other than RCTs. We also excluded pilot and cluster RCTs.

Two reviewers screened all citations independently and in duplicate in two stages: first titles and abstracts, then full texts using Covidence software (C. T. Y., T. G., I. N. S., I. O., S. Y., A. P.).¹⁷ In the first stage, any citation selected as potentially relevant by either reviewer was advanced to stage 2. In stage 2, any discrepancies were resolved by discussion and third-party adjudication as necessary (B. R. or A. K.). We captured reasons for exclusion at the second stage only.

If site-by-site breakdown of race-based data for study patients was not included in the manuscript or supplementary material, further information was solicited from trial authors by email. A single follow-up email was sent two weeks after the first email. If site-level data on racial demographics was not included in publicly available materials and could not be obtained from authors after two attempts, the study was excluded (ESM eAppendix 3).

Data extraction

Data were extracted independently and in duplicate by two reviewers with discrepancies resolved through discussion. We used a prepiloted data extraction template to collect key study details, including study citation, year of publication, demographic data, racialized site-by-site breakdown, and study design. For multicentre studies with international sites, we only included USA and Canadian study sites. For each trial study site, we captured the total number of patients enrolled and the number enrolled from each racial identity based on self-identification. For analysis, patients were classified as Black vs non-Black. For each study site, we recorded the city in which the site was located. City-specific demographic data were obtained using USA Census and Statistics Canada data.^18,19

Data analysis

We analyzed the site-by-site demographic data to calculate proportionality ratios of Black study participants compared with the total number of participants enrolled at each study site. We then calculated the same proportionality ratio of Black people using city-based population demographic data. We used these two measures to calculate a risk difference (RD) comparing the ratio of Black people in study site-based cohorts to the ratio of Black people in the corresponding cities. A negative RD signified under-representation.

Data analysis was conducted using Stata version 16.0 (StataCorp LLC, College Station, TX, USA).²⁰ We pooled risk differences examining the representation of Black people across individual studies, cities and centres using a random effects model reporting pooled RD and 95% confidence intervals (CIs). We explored subgroup effects using meta-regression to determine the study-level predictors of Black under-representation. Predetermined variables included the country within which the study site was located (USA, Canada, or both), study site city, whether the RCT evaluated a drug intervention (yes/no), consent model (written informed consent vs waiver of informed consent), centre status (multicentre vs single centre), industry funding (yes/no), and year of publication (as a continuous variable). The dependent variable for meta-regression was RD, the percent of over/under representation. We used a P value < 0.05 as a threshold for subgroup effect modification. We conducted meta-analyses using Stata and a meta-regression using the restricted maximum likelihood method. We assessed heterogeneity between studies using the Wald Chi squared test for homogeneity, the I² statistic, and visual inspection of the forest plots.

Results

Study identification

Of an initial 730 citations, we excluded 601 during the title and abstract screening and 102 during full-text screening, leaving 27 studies that met the preliminary eligibility criteria (Fig. 1). Four had complete site-based racial demographic information. We emailed investigators of the 23 other studies for missing site-based racial demographic data. Of these, 17 provided all the necessary data, five responded but did not have or were not unable to provide the necessary demographic information, and one failed to respond.^21–26 In the end, 21 RCTs met full eligibility criteria and were included in the analysis.^27–47

Fig. 1.

PRISMA flow diagram demonstrating the identification, screening and inclusion of eligible critical care RCTs

Study characteristics

Table 1 provides an overview of the included trials. Of the included studies, eight (38%) were single-centre trials, while the remaining 13 (62%) trials were multicentred. Seventeen (81%) studies had sites only in the USA, two (10%) had sites only in Canada, and two (10%) had sites both in the USA and Canada. None of the included trials were international trials with study sites outside the USA and Canada. Of the 21 included RCTs, 12 (57%) involved a drug trial intervention and nine (43%) were pragmatic trials that used a waived consent model for study participants. Overall, the 21 studies included sites spread across 59 USA and Canadian cities.

Table 1.

Key characteristics of included studies

Study year	Site	Critical care condition	Intervention	Consent model	Funding	Multicentre	Number of North American sites
Billings 201641	USA	Acute kidney injury	Drug intervention	No waived consent	Industry	Single centre	1
Casey 201927	USA	Tracheal Intubation	Non-drug intervention	Waived consent	Industry	Multicentre	7
Curtis 201646	USA	End of life care	Non-drug intervention	No waived consent	Industry	Multicentre	2
Delorme 201738	Canada	Acute respiratory failure	Drug intervention	No waived consent	Non-industry	Single centre	1
Festic 201745	USA	Acute respiratory distress syndrome	Drug intervention	No waived consent	Non-industry	Multicentre	5
Girard 201940	USA	Delirium in critical illness	Drug intervention	No waived consent	Non-industry	Multicentre	19
Heyland 202043	Canada	Long stay ICU patients	Drug intervention	No waived consent	Not reported	Multicentre	12
Jaiswal 201944	USA	Postoperative deliruim	Drug intervention	No waived consent	Non-industry	Single centre	1
Janz 201630	USA	Endotracheal intubation	Non-drug intervention	Waived consent	Non-industry	Single centre	1
Janz 201829	USA	Endotracheal intubation	Non-drug intervention	Waived consent	Non-industry	Multicentre	6
Janz 201928	USA	Cardiovascular collapse	Non-drug intervention	Waived consent	Non-industry	Multicentre	9
Limaye 201737	USA	Critical illness	Drug intervention	No waived consent	Non-industry	Multicentre	14
Moss 201634	USA	Acute respiratory failure	Non-drug intervention	No waived consent	Non-industry	Multicentre	5
Schell-Chaple 201739	USA	Febrile critically ill	Drug intervention	No waived consent	Industry	Multicentre	3
Semler 201631	USA	Endotracheal intubation	Non-drug intervention	Waived consent	Non-industry	Single centre	1
Semler 201733	USA	Endotracheal intubation	Non-drug intervention	Waived consent	Non-industry	Multicentre	6
Semler 201832	USA	Critically ill	Drug intervention	Waived consent	Industry	Single centre	1
Sims 201936	USA	Trauma and Hemorrhagic stroke	Drug intervention	Waived consent	Non-industry	Single centre	1
Skrobik 201842	USA	ICU delirium	Drug intervention	No waived consent	Non-industry	Multicentre	2
STARRT-AKI 202047	Canada and USA	Acute kidney injury	Non-drug intervention	No waived consent	Non-industry	Multicentre	23
Swan 201635	USA	Hospital-acquired infections	Drug intervention	Waived consent	Non-industry	Single centre	1

ICU = intensive care unit

Representation of Black people across RCT populations

The proportion of Black study participants ranged from 0% to 82% and is summarized in Table 2. The median [interquartile range] number of Black participants enrolled per study was 19 [9–72] compared with a median total sample size of 160 [120–334] participants across included studies. Compared with population-level, city-based demographics, Black people were under-represented in the trial population of critical care RCTs trials by 6% (95% CI, 1 to 11) (Fig. 2). Black people were under-represented in 17 of the 21 RCTs. The highest rate of under-representation was 23%,⁴⁰ followed by two studies with under-representation rates of 19% (Fig. 2, Table 2).^41,45 The highest proportion of Black trialist over-representation among included RCTs was 38% (RD, 38%).³⁶

Table 2.

Level of representation

Study year	Number of Black patients enrolled	Number of patients enrolled	% Black	Difference between study enrolment and city-based demographic (%)
Billings 201641	26	615	4	Under-representation (−23%)
Casey 201927	108	398	27	Neutral representation (0%)
Curtis 201646	12	219	6	Under-representation (−2%)
Delorme 201738	0	12	0	Under-representation (−2%)
Festic 201745	3	82	4	Under-representation (−19%)
Girard 201940	76	566	13	Under-representation (−19%)
Heyland 202043	2	155	1	Under-representation (−5%)
Jaiswal 201944	11	120	9	Over-representation (+3%)
Janz 201630	19	149	13	Under-representation (−15%)
Janz 201829	72	291	25	Under-representation (−4%)
Janz 201928	75	334	23	Over-representation (+4%)
Limaye 201737	17	160	11	Under-representation (−14%)
Moss 201634	9	120	8	Under-representation (−4%)
Schell-Chaple 201739	2	41	5	Under-representation (−1%)
Semler 201631	19	149	13	Under-representation (−15%)
Semler 201733	72	291	25	Under-representation (−4%)
Semler 201832	2,165	15,802	14	Under-representation (−14%)
Sims 201936	82	100	82	Over-representation (+38%)
Skrobik 201842	1	100	1	Under-representation (−13%)
STARRT-AKI 202047	27	869	3	Under-representation (−4%)
Swan 201635	42	325	13	Under-representation (−10%)

Fig. 2.

Meta-analysis for Black patients’ underrepresentation in critical care RCTs.

RD = risk difference

Subgroup analysis and meta-regression

A subgroup analysis to determine the study-level predictors of Black under-representation based on predefined variables found no significant subgroup effects (ESM eAppendix 4–8). Subgroup analysis by city of enrolment yielded important variations in representation although small numbers of patients were enrolled in some cities, contributing to imprecision and limiting our ability to draw conclusions (ESM eAppendix 9).

Using meta-regression—after controlling for the country (USA vs Canada), drug intervention, consent model, multicentre vs single centre, industry funding, and year—country was the only significant source of heterogeneity, with greater under-representation in USA-based study sites (B = –0.13; 95% CI, –0.24 to –0.02; P = 0.02) (Table 3).

Table 3.

Random-effects meta-regression assessing sources of heterogeneity in Black representation in critical care randomized controlled trials

Predictor	Beta coefficient	95% confidence interval	P value
USA site	-0.129	-0.239 to -0.020	0.02
Drug	0.003	-0.084 to 0.089	0.96
Pragmatic trial	0.059	-0.063 to 0.181	0.34
Multicentre	-0.015	-0.143 to 0.114	0.82
Industry funding	0.03	-0.026 to 0.086	0.29
Year of publication	-0.012	-0.036 to 0.012	0.34
City	0.001	-0.0003 to 0.003	0.12

Discussion

Our meta-epidemiological study of 21 critical care RCTs found that Black people were under-represented in trial populations by 6%, compared with the populations of the cities in which the trials were conducted. In addition, while representation varied by study and site, USA-based sites had higher degrees of under-representation than Canadian sites, an observation which deserves further investigation. Although we did not examine contributing factors, these could include the impact of privatized health care or differential rates of systemic racism. Privatized health care may restrict accessibility to health care services as patients with low socioeconomic status may wait longer before accessing health care because of cost. These patients may subsequently have higher burdens of illness by the time they reach critical care, impacting their eligibility for clinical trials. Systemic racism within health care may be more prevalent in the USA than in Canada and can manifest in several ways including differences in resources allocated to racialized patients or their length of stay in hospital. There may also be greater mistrust within marginalized communities facing perpetual systemic racism, which would be a barrier to study enrolment. Furthermore, there was no evidence of effect modification based on prespecified subgroups.

Our study results revealed heterogeneity in Black representation in clinical trials. Despite this heterogeneity, we opted to pool results. As opposed to interventional studies, and similar to prognosis studies, there are known challenges in interpreting I² values when pooling proportions or risk differences.⁴⁸ Guidance in this setting suggests examining variation in point estimates to assess statistical heterogeneity as more important than the I² value.⁴⁸ Pooled proportions can be interpreted as a weighted average and using a random-effects model accounts for these heterogeneous effects.⁴⁹

Black representation may vary depending on the context and intervention being examined. For example, an RCT conducted in Philadelphia examined whether low-dose supplementation of arginine vasopressin in patients with trauma and hemorrhagic shock increases the need for transfused blood products during resuscitation.³⁶ The study adopted a pragmatic waived consent model; and 82 of 100 (82%) enrolled participants were Black—double the estimated Black population in Philadelphia (42%). This discrepancy is likely explained by the disproportionate prevalence of gun violence in minority communities living in under-resourced neighbourhoods, especially in urban cities like Philadelphia.⁵⁰ In contrast, similar discrepancies in representation are not observed in studies investigating other medical conditions disproportionately affecting Black communities, such as cardiovascular disease and respiratory failure. Another trial investigated whether short-term, high-dose perioperative atorvastatin reduced acute kidney injury following cardiac surgery.⁴¹ Although Black patients have a higher incidence and prevalence of heart failure, along with worse clinical outcomes for cardiac conditions, the study under-represented Black people by 23% compared with city-based demographics. Randomized controlled trials should produce information generalizable to all, notably the communities disproportionately impacted by the health condition of interest.

Similar findings of under-representation have been reported in oncology and COVID-19 clinical trials.^51,52 In the context of COVID-19, although Black, Latinos, and Indigenous Americans were over-represented in COVID-19 populations and related deaths, these groups were substantially under-represented in many of the studies examining the condition.⁵¹ Several reasons for Black under-representation (and other minority groups) have been identified previously. Major barriers to recruiting Black clinical trial participants include time commitment (i.e., difficult work schedules), confidentiality concerns, lack of interest, educational disadvantage, lack of compensation, lack of accessible transportation options, and distrust in medical establishments.^53,54 Our results highlight site selection as another key consideration to enrolling Black participants in critical care trials. Even within cities, there is variability in racial breakdown by neighbourhood and hospital coverage. For example, because of inter-hospital variability in patient demographics, a trial could enrol every patient in a single ICU and perfectly represent the population of the hospital but still under-represent the city’s Black population. For example, according to Matthew Semler, MD (written communication, January 2022), while approximately 27% of Nashville’s population is Black, 16% of inpatients at Vanderbilt’s academic site are Black. In contrast, 85% of the inpatients at Meharry, a historically Black University Hospital, are Black. This inter-institution variability, even within cities, must be considered when interpreting the results of our analysis.

To overcome under-representation, noteworthy recommendations include culturally competent community outreach, culturally specific information sessions, verbal marketing, targeting specific clinics, newsletters, respect, and rapport building as key strategies for informed consent, encouraging study participation and mitigating participation barriers.⁵³ Benefits to participation (i.e., adequate compensation or free medical services) and low risk in participation (i.e., noninvasive treatment) are two additional strategies to mitigate accessibility barriers and encourage participation.⁵⁵ An NEJM editorial noted that individuals are often more trusting of research led by or involving an individual belonging to their ethnic group thereby facilitating informed consent, trust, participant screening, and retention.⁵⁴ Nevertheless, there is a shortage of investigators from minority groups due to a long history of marginalization.⁵⁴ Research teams should aim for diverse composition of researchers and individuals from minority groups, and support these researchers, given that representation among health care professionals can promote representative study cohorts.

The results of this paper should inform policy in terms of ensuring adequate representation of Black people in critical care research and highlight the importance of reporting study cohort racial and ethnic demographic data. Like the NEJM, journals should implement policies requiring that every article submitted for journal publication includes demographic data and disease-specific background information as an important first step to improving Black representation, especially in studies that disproportionately impact the Black community.⁵⁴ Similarly, funding agencies should mandate disclosing demographic data when applying for funding. Critical care trials should be designed so that participants reflect those who may benefit from the results. Sustainable recruitment strategy guidelines have been developed through Trial Forge’s INCLUDE Ethnicity Framework, aiming to improve the inclusion of under-represented groups in research studies.⁵⁶ Furthermore, documenting diversity in reporting RCTs is important because reporting the demographic data of eligible but not recruited patients may provide important insight into the representativeness of the study population. Optimal representation cannot be achieved unless it is adequately captured and studied. Our study may also help facilitate research into the representation of other historically under-represented populations, such as Latinos or Indigenous Americans.^57,58 A similar type of review examining representation could be replicated for other groups and patient populations beyond the critical care setting.

Our review has several strengths. We included a comprehensive search with a clear pre-established protocol and search strategy (ESM eAppendix 1). We screened and extracted data in duplicate and we were able to access additional missing data from study authors with a high degree of engagement, ensuring generalizability of findings. This study also has limitations. Several studies were not eligible for final inclusion even though they met preliminary eligibility criteria, because site-based racial demographics were unavailable. The lack of race-based ICU patient demographic data may further conceal under-representation. We hypothesize that published trials without accessible demographic data were likely to be less representative, as researchers who have successfully recruited a racially diverse cohort may be more likely to share demographic data. As such, the exclusion of such trials may bias the overall findings towards underestimating the true degree of under-representation.

We chose to compare data sets with city-based racial demographics as these were the most reliable demographic data available. Nevertheless, as discussed earlier, city-based demographics do not necessarily equate to hospital demographics. We also note that race and ethnicity are social constructs that may not adequately capture underlying genetic variability and response to specific treatments. Furthermore, dichotomizing Black and non-Black may miss important racial variation within the non-Black population; however, we were unable to access this level of detail. Another limitation is heterogeneity in study characteristics that influence the overall certainty in the pooled representation estimate. Nevertheless, the overall estimate showing under-representation remains an important finding, and can serve as a call to action for investigators and researchers. In addition, we limited our search to critical care RCTs published in high-level journals, and the generalizability to other populations is uncertain. Finally, we made a practical decision to include only ICU studies done in adults and at North American study sites, which limits the generalizability of our results. A follow-up study examining other jurisdictions and other populations (i.e., children, outside the ICU) would be valuable.

Conclusion

This study suggests that Black people are under-represented in critical care RCTs compared with site-level city-based demographics. Critical care research in the USA and Canada should improve representation of Black people to increase equity, improve generalizability, and facilitate the implementation of results. Interventions are required to ensure adequate Black representation in critical care RCTs in both the USA and Canada. Further research is needed to investigate the full extent of Black under-representation in critical care trials, identify other contributing factors, and identify effective interventions to improve Black representation.

Supplementary Information

Below is the link to the electronic supplementary material.