Enrollment of women and minorities in NINDS trials

Abstract

Objective:

To determine policy-associated changes over time in 1) the enrollment of women and minorities in National Institute of Neurological Disorders and Stroke (NINDS)–funded clinical trials and 2) the trial publication reporting of race/ethnicity and gender.

Methods:

All NINDS-funded phase III trials published between 1985 and 2008 were identified. Percent of African Americans, Hispanic Americans, and women enrolled in the trials was calculated for those trials with available data. Z tests were used to compare reporting and enrollment data from before (period 1) and after (period 2) 1995 when NIH enacted their policies regarding race, ethnicity, and gender. Percent of main trial publications reporting enrollment of African Americans, Hispanic Americans, and women was also calculated.

Results:

Of the 56 trials identified, 100%, 48%, and 25% reported enrollment by gender, race, and ethnicity. Women constituted 42.1% of the trial population. Enrollment of women increased over time (36.9% period 1; 49.0% period 2, p < 0.001). African Americans constituted 19.8% of the enrollees in trials with available data and enrollment increased over time (11.6% period 1; 30.7% period 2, p < 0.001). Hispanic Americans constituted 5.8% of subjects in trials with available data and enrollment decreased over time (7.4% period 1; 5.0% period 2, p < 0.001).

Conclusions:

Improvements in reporting of race/ethnicity in publications and enrollment of Hispanics in NINDS trials are needed. While African American representation is above population levels, Hispanic Americans are underrepresented in NINDS trials and representation is declining despite Hispanics' increasing representation in the US population.

Despite the increased burden of stroke in women and minorities^1–4 and neurologic disease in general in minorities,^5–7 there are no data on enrollment of women and minorities specific to trials in neurologic diseases. Historically, women and minorities have been underrepresented in clinical trials.^8,9 More recently, overall representation of women and African Americans, but not Hispanic Americans, in NIH trials is improving.¹⁰ Despite overall improvement, underrepresentation of women and minorities persists in a number of disease states.^9,11

Since the 1994 NIH Revitalization Act, NIH has mandated annual reporting of race, ethnicity, and gender by study investigators to NIH in the form of progress reports, but does not mandate race and ethnicity reporting in primary publications.¹² In addition, NIH created standards for participation of women and minorities. These requirements vary based on pretrial evidence of differential treatment effects for subgroups. Prior studies have suggested that this policy has changed neither the reporting of minority participation in primary publications¹³ nor participation of minorities in cancer or heart failure trials.^9,11

The purpose of this study was to assess 1) the reporting of race, ethnicity, and gender in trial publications and 2) the enrollment of women and minorities in NINDS-funded phase III clinical trials with published results. Representation of women and minorities in stroke-related trials was assessed separately to allow for comparison with the relatively well-described epidemiology of stroke.

METHODS

NINDS phase III clinical trials were identified using 2 mechanisms. First, trials completed before 2000 (n = 28, start dates between 1977 and 1998) were identified from a previous publication.¹⁴ One trial was excluded because no associated publication could be identified. Second, trials completed after 2000 (start dates between 1992 and 2004) were identified through clinicaltrials.gov, using “NINDS” and “phase III” as search criteria. This search identified 76 additional trials. Forty-seven trials were excluded because they were ongoing (n = 36), the primary publication indicated the trial was not phase III (n = 6), the trial was prematurely terminated (n = 3), or no associated publication could be identified (n = 2) at the time of the search (October 2009). For included trials, whether any associated publication reported information about race, ethnicity, and gender was recorded. Next, for trials that reported race and ethnicity, enrollment of African Americans, Hispanic Americans, and women were abstracted from the publication. If the data were not present in the primary publication, prior publications, such as methods papers, were used where available. If enrollment data were not identified through published literature, we contacted the corresponding author of each primary trial publication by e-mail. Three sequential methods were used to identify e-mail addresses for the corresponding author: 1) the primary publication was searched, 2) more recent publications by that author were sought, and 3) a general Web search was conducted. Through this approach, we identified corresponding author e-mail addresses for 42/44 trials for which some enrollment data were unavailable. After identifying an address, initial e-mails were sent to all authors. If authors did not reply to the initial e-mail, a second e-mail was sent 2 weeks later. We received e-mail responses from 31/42 authors contacted within 2 weeks of the second e-mail.

Data analysis.

Reporting of gender and race/ethnicity information in publications of included trials was summarized with frequencies and percents. The percent of women and minorities (i.e., African Americans and Hispanic Americans) enrolled was calculated by summing the number of women or the number of minorities and dividing by the total number of patients enrolled in the trials. The percent of trials reporting gender and race/ethnicity data and the percent of women, African Americans, and Hispanic Americans from the trials were compared before and after January 1, 1995 (period 1 vs period 2) with respect to start date using binomial Z tests. This date was selected because the initial NIH policy regarding enrollment of women and minorities was published in 1994.¹² Comparisons were repeated excluding gender-specific (n = 5) and race-specific trials (n = 1). Finally, gender and race/ethnicity reporting and enrollment calculations were repeated in the subset of trials that were stroke-related. Stroke trials were defined as trials where the primary intervention targeted stroke treatment, rehabilitation, or prevention. Analyses were performed using R: A Language and Environment for Statistical Computing (Vienna, Austria).

We performed a sensitivity analysis to assess the extent to which lack of available race/ethnicity data for some trials could affect the estimated minority enrollment across all trials. To do this, we re-estimated the overall minority enrollment under a range of assumed values of minority enrollment in the trials without available data. The range of values included zero to double the enrollment in the trials with available data.

RESULTS

The 56 trials available for analysis included a total of 42,388 subjects and addressed the following conditions: stroke (n = 22), Parkinson disease (n = 5), traumatic brain injury (n = 4), epilepsy (n = 3), spinal cord injury (n = 3), cardiac arrest (n = 3), multiple sclerosis (n = 3), amyotrophic lateral sclerosis (n = 2), and other conditions (n = 11) (table e-1 on the Neurology® Web site at www.neurology.org). Thirty-one trials began recruiting patients in period 1 and 25 in period 2. Five trials exclusively recruited members of a single gender, 4 of which only included women. Two of these trials focused on pregnant women while assessing fetal outcomes. One trial exclusively included African Americans and there were no ethnicity-specific trials. Unpublished enrollment data were obtained directly from the trial investigators for 18 trials: race data only (n = 2), ethnicity data only (n = 5), and both race and ethnicity data (n = 11). Of the 13 other authors who responded, most reported that race/ethnicity data were not available either because they were not collected or because the data were no longer available. Net enrollment of African Americans (4.0% vs 22.0%, p < 0.001) and Hispanics (3.4% vs 7.3%, p < 0.001) were lower in trials where enrollment data were obtained through author e-mail as opposed to trial publications.

Reporting and enrollment of women.

Of the 56 trials identified, all reported the number of women included in the primary publication. Women constituted 42.1% of the entire trial population, which decreased to 39.1% when the 5 gender-specific trials were excluded (table 1). Women comprised 51% of the US population in 2000.¹⁵ Enrollment of women improved over time (36.9% period 1; 49.0% period 2, p < 0.001), a trend that persisted when gender-specific trials were excluded (33.6% period 1; 42.6% period 2, p < 0.001) (table 2).

Table 1.

Reporting and enrollment by race, ethnicity, and gender in NINDS-funded phase III studies published from 1985–2008 with and without specific race/gender enrollment criteria

Abbreviation: NINDS = National Institute of Neurological Disorders and Stroke.

Table 2.

Comparison of enrollment and reporting of race, ethnicity, and gender in NINDS phase III trials before and after January 1, 1995: all trials and those trials without specific race/gender enrollment criteria

Abbreviation: NINDS = National Institute of Neurological Disorders and Stroke.

African American reporting and enrollment.

Most trials (80.4%) reported at least some race information in trial publications (table 1). This percentage did not change over time (80.6% period 1; 80.0% period 2, p > 0.99) (table 2). Eighteen trials (32.1%) reported race solely by dichotomizing between white and nonwhite subjects, a proportion that did not change over time (29.0% period 1; 36.0% period 2, p = 0.79). Approximately half (48.2%) of trials reported the number of African Americans participating. This proportion did not change over time (51.6% period 1; 44.0% period 2, p = 0.77). Of trials with available African American enrollment data, African Americans constituted 19.8% of the entire trial population, and 14.5% if the single race-specific trial was excluded. African American enrollment increased over time (11.6% period 1; 18.9% period 2, p < 0.001 excluding the race-specific trial). Sensitivity analysis (figure) revealed that even if zero African Americans were enrolled in trials without available race data (n = 16), their trial representation (13.5%) would be greater than their representation in the US population in 2000 (12.9%).¹⁶

Figure. Sensitivity analysis.

Results of one-way sensitivity analysis to assess the extent to which nonreporting of race/ethnicity in some trials could affect the estimated minority enrollment across all National Institute of Neurological Disorders and Stroke phase III trials. Assumed enrollment rates for nonreporting trials are varied from 0% to 200% of the average enrollment of all trials that reported race/ethnicity information. 1) The African American percentage of the population as of the 2000 US census was 12.9%.²⁴ 2) The Hispanic percentage of the population as of the 2000 US census was 12.5%.⁸

Hispanic American reporting and enrollment.

A quarter (25.0%) of trials reported the number of Hispanic American subjects in trial publications (table 1). This proportion did not change over time (table 2; 25.8% period 1; 24.0% period 2, p > 0.99). Hispanic Americans constituted 5.8% of the enrolled population in trials with available ethnicity data. This proportion decreased over time (7.4% period 1; 5.0% period 2, p < 0.001). In sensitivity analysis (figure), even if the proportion of Hispanic Americans enrolled in trials without available ethnicity data were twice as high (11.7%) as in trials with available data (5.8%), their overall trial representation (8.6%) would fall far short of their representation in the US population in 2000 (12.4%).¹⁶

Stroke trials.

Stroke trials tended to be relatively large trials; consequently, more patients were enrolled in the 22 stroke-related trials (26,421; 62.3%) than in the 34 non–stroke-related trials (15,967; 39.2%) (table 3). African American race was reported in 68.2% of stroke-related trial publications and Hispanic ethnicity in 22.7%. Women constituted 40.2% of the stroke trial population and 38.7% of the population of non–gender-specific trials. Of stroke trials with available data, African Americans constituted 22.7% and Hispanic Americans 5.0% of the population.

Table 3.

Comparison of enrollment and reporting of race, ethnicity, and gender in stroke and non–stroke-related NINDS phase III trials published from 1985–2008

Abbreviation: NINDS = National Institute of Neurological Disorders and Stroke.

DISCUSSION

This analysis of NINDS-funded phase III studies demonstrates that reporting of race and ethnicity information in trial publications has been poor and furthermore has not improved since the implementation of the NIH Revitalization Act. In addition, while actual enrollment of women and African Americans in NINDS trials has improved over time, the poor enrollment of Hispanic Americans in these trials has worsened with time. This is consistent with general trends in NIH trial enrollment where representation of women (58% of NIH trial enrollees in 2007¹⁰ vs 51% of the US population¹⁶) and African Americans (13% of NIH trial enrollees in 2007¹⁰ vs 14% of the US population¹⁶) roughly parallels population representation, while Hispanics (7% of NIH trial enrollees¹⁰ vs 15% of the US population¹⁶) are underrepresented relative to their population representation. This underrepresentation has occurred while the Hispanic population has been rapidly growing,¹⁷ raising concerns about the adequacy of measures to ensure Hispanic American participation and pointing to a potentially worsening problem of generalizability of trial results to this subgroup.

In spite of NIH mandates that trials track and report the number of women and minority enrollees, this information is frequently omitted from trial publications; African American race was reported in 44% of trials and Hispanic ethnicity in 24% of trials since 1995 in our sample. These omissions limit the ability of readers of the medical literature to assess whether results generalize to all racial or ethnic groups and for researchers to generate and test hypotheses related to racial or ethnic differences in outcomes. Limited reporting of race and ethnicity in trial publications has been observed repeatedly.^9,13,18 For older trials (period 1), it is unclear whether nonreporting is a consequence of failure to collect race/ethnicity data or a failure to report collected data in trial publications. For more recent trials (period 2), given the requirement to report race/ethnicity data to NIH,¹² it is most likely that these data were collected, but not included in trial publications. Several explanations have been posited to explain nonreporting, such as reticence of subjects to disclose race/ethnicity, lack of confidence in methods of ascertaining race/ethnicity,¹³ or publication bias. Given that the NIH Revitalization Act is well over a decade old, our results are troubling, although it is possible that there have been more recent improvements in reporting and enrollment performance as we were not able to separately analyze the impact of the 2001 policy amendment¹⁹ because only 6 trials started enrollment after 2001.

While African Americans have historically been underrepresented in clinical trials, our analysis found representation in trials (19.8%) greater than in the US population (12.9%).¹⁶ Much of this overrepresentation is attributable to a single large trial that exclusively enrolled African Americans.²⁰ If this trial were excluded, African American representation would fall to 14.5%. Overrepresentation may also be due to the higher incidence of neurologic disorders among African Americans.^3,5–7 It appears that reporting bias explains some of the overrepresentation in this sample, as trials reporting race information through direct author correspondence had lower African American enrollment (4.9%) than trials that reported race in trial publications (22.0%). Nonetheless, if trials without available data had no African American participants, African Americans would still have constituted 13.5% of the study population.

While African American representation appears to be adequate in NINDS trials, Hispanic Americans appear to be underrepresented. Hispanic Americans represented only 5.8% of the trial population compared to 12.5% of the US population.¹⁶ The underrepresentation of Hispanic Americans was also identified in stroke trials, where Hispanic Americans made up only 5.0% of the study population despite the well-established increased stroke burden in this population.^2,4 In spite of Hispanic American population growth and the NIH policy, Hispanic American enrollment declined over time. As Hispanic American ethnicity is known for only a subset of trials (53.6%), definitive conclusions should not be drawn about the appropriateness of representation in NINDS trials. These conclusions should be further tempered by the challenges of ethnicity classification in general and the inconsistent ethnicity ascertainment methods used in these trials.²¹ However, even if trials without available Hispanic American enrollment data enrolled twice the number of Hispanic Americans as trials with available data, a highly unrealistic assumption, Hispanic Americans would still have constituted only 8.6% of the trial population. While a significant body of literature has analyzed reasons for underrepresentation of African Americans in clinical trials,²² there has been less exploration of reasons for Hispanic American underrepresentation. Hispanic Americans appear to be as likely or slightly more likely to participate in clinical trials than non-Hispanic whites if given the opportunity.²³ Also, a survey of Hispanic Americans with cancer found that they are more interested in learning about clinical trials than non-Hispanic whites.²⁴ Despite this willingness to participate, underrepresentation of Hispanic Americans has been repeatedly documented.^9,11,25 Several hypotheses have been postulated to explain this phenomenon.^11,26 For example, language barriers,²⁷ potential concerns about documentation status,²⁸ and the geographic distribution of trial enrollment sites relative to the Hispanic American population²⁹ may be issues in this subgroup. Unless measures are taken to increase Hispanic American enrollment, the magnitude of underrepresentation will likely increase as the Hispanic population is rapidly growing; Hispanic Americans are estimated to make up 16% of the US population in 2010 and 19% by 2020.¹⁷

The discordance between Hispanic American underrepresentation and African American overrepresentation in our series has not been reported in disease-specific series,^11,25 but is consistent with the overall trend in NIH trial enrollment.¹⁰ There are several potential explanations. First, the magnitude of the difference is amplified by a single large race-specific trial,²⁰ which accounted for almost one-third of enrolled African Americans. If this trial is excluded, overall African American representation falls to 14.5% of enrollees. Second, this set of trials was heavily weighted toward stroke trials (60.7% of all enrollees were in stroke trials) and while both groups have increased stroke incidence relative to non-Hispanic whites, the increased risk is higher in African Americans than in Hispanic Americans.^2–4 The high proportion of stroke trials may further contribute to this difference as stroke incidence increases dramatically with age¹ and the Hispanic American population in the United States is younger than the African American population as a whole.¹⁷ Finally, recent increased awareness of cardiovascular disparities in African Americans compared to Hispanic Americans may have contributed to this difference as our series included more recent trials than previous series.^11,25

Women have made up the majority of participants in NIH trials every year since 1995.¹⁰ This effect appears to be driven largely by gender-specific trials. In National Heart, Lung and Blood Institute–funded cardiovascular trials, women made up 54% of the entire population, but only 38% of patients in trials enrolling both women and men.³⁰ Our results found similar trends. While women were likely underrepresented prior to 1995 (36.9% of enrollees), representation improved after 1995 (49.0% of enrollees), an effect largely driven by gender-specific trials; women made up 42.6% of the non–sex-specific trial population. While overall enrollment of women has improved, underrepresentation has persisted in a number of specific disease states.^9,11,18,31 Our findings suggest that women are likely underrepresented in stroke trials, constituting 37.8% of the participants in trials enrolling both men and women. This level of representation is likely attributable in part to age effects as stroke incidence is higher in men than women in the younger cohorts participating in clinical trials.³¹ However, age does not appear to completely explain the lower proportion of women in these trials. With the pragmatic assumption that gender-specific enrollment should roughly parallel stroke incidence in that gender, the 37.8% participation proportion in these trials would translate to a female/male enrollment ratio of 0.6, a ratio lower than the female/male incidence ratio in any age cohort.¹

In the 15 years since implementation of the NIH Revitalization Act, representation of women and African Americans in clinical trials has improved significantly.¹⁰ Despite clear progress, underrepresentation persists in a number of specific disease states,^9,11,18,32 including stroke and other neurologic diseases, as demonstrated in the current analysis, particularly for Hispanic Americans. Additional mechanisms to increase participation of women and minorities in trials may be necessary. Currently, controlled data favoring specific recruitment approaches are sparse and there are no data specific to recruitment in neurologic disease.³³ However, several promising tactics have generated higher levels of minority recruitment. At the level of individual trials, these tactics include actively targeting recruitment to regions with racially and ethnically diverse populations,³⁴ partnering with community and church-based organizations,³⁵ and minimizing logistical hurdles to participation that disproportionately affect minorities.³⁶ Improving disease-specific knowledge³⁷ and limiting trial exclusion criteria that may differentially exclude minorities³⁸ have also been suggested as methods of increasing minority enrollment. More policy-driven approaches may include mandating that trials be designed to enroll population-representative samples even when no clear differential treatment effect exists and mandating reporting of the enrolled number of women and minorities in primary publications. Reporting could be similarly improved through the efforts of standards organizations, such as CONSORT, or organizations of journal editors³⁹ who should consider the addition of race/ethnicity as mandatory reporting requirements and the imposition of standardized race and ethnicity classifications and means of ascertainment. While journal editors have broadly accepted the importance of race/ethnicity reporting, the challenges in measuring and classifying race and ethnicity appear to be barriers to adequate reporting.⁴⁰

AUTHOR CONTRIBUTIONS

Statistical analysis was conducted by Dr. Burke and Dr. Sanchez. Dr. Burke participated in the design, analysis, and interpretation of the data as well as drafting the manuscript. Dr. Brown participated in the conceptualization, analysis, and interpretation of the study as well as extensive revisions of the manuscript. Dr. Lisabeth participated in conceptualization, analysis, and interpretation of the study as well as revisions of the manuscript. Dr. Sanchez participated in the design and analysis of the study as well as revisions of the manuscript. Dr. Morgenstern participated in the conceptualization and interpretation of the study as well as revisions of the manuscript.

DISCLOSURE

Dr. Burke reports no disclosures. Dr. Brown serves on the editorial board Neurology; is supported by an NINDS Career Development Award (K23 NS051202); and has received research support from CVR Global, Inc., the University of Michigan, Michigan Department of Community Health, Blue Cross Blue Shield of Michigan Foundation, the Diocese of Corpus Christi, and the NIH. Dr. Lisabeth has received speaker honoraria from the Hazel K. Goddess Fund for Stroke Research in Women; and receives research support from the NIH (NINDS and NHLBI). Dr. Sanchez receives research support from the NIH. Dr. Morgenstern has served on scientific advisory board for Genentech, Inc. and on a medical adjudication board for Wyeth; has served as a consultant for the Alaska Native Medical Center; and receives research support from AGA Medical Corporation and the NIH/NINDS.