Characterization of Clinical Study Populations by Race and Ethnicity in the Biomedical Literature

Abstract

Objective

The importance of race and ethnicity in biomedical research has long been a subject of debate, recently heightened by data revealed by the completion of the sequencing of the human genome and the mapping of human genetic variation. We aimed to determine whether and how the reporting of race has changed over the last three decades and how the practice may differ given study location, where the journal of publication is based, and decade of publication.

Design

We analyzed a sample of studies published in the Journal of the American Medical Association, the Lancet, and the Canadian Medical Association Journal from 1980 to 2009.

Main Outcome Measures

The number of articles that reported race by journal and decade and the descriptors used.

Results

Of 1,867 publications analyzed, 17.30% reported race. The reporting of race and number of populations reported increased over time for all three journals. In addition, the diversity of race/ethnicity descriptors increased, with increased use of race/ethnicity combinations and nationality of research subjects.

Conclusion

Though it has increased over the past few decades, the reporting of race/ethnicity of study populations is relatively low, ambiguous and inconsistent, likely influenced by the uncertain relevance of these variables to the study’s outcomes, study location, researcher views, and the policies of journals and funding agencies. Thus, due to the inconsistent and ambiguous practice of reporting race/ethnicity, comparison of study outcomes can result in misleading conclusions. Thus, improvements in standardization of terms and new approaches to characterize research subjects related to race/ethnicity are imperative.

INTRODUCTION

There has been much debate about the biological validity of race, particularly spurred by recent findings from genomic analyses of human populations.^{1, 2} Overall, it has been found that there are greater genetic differences between individuals of the same racial group than between individuals of different groups.³ Although some rare genetic variants are only found in one population, the majority of genetic variations are found in all populations.^{4, 5} Despite the uncertainty of the biological validity of race, the construct of race is still used in biomedical research as a primary descriptor of study participants. However, its continued use may contribute to confounding outcomes instead of narrowing or identifying factors associated with study outcomes.

One of the primary reasons for the continued use of race in the literature are requirements enforced by funding agencies, regulatory agencies and biomedical journals. For example, in 2001, the U.S. National Institutes of Health (NIH) issued a policy specifying characterization of human subjects by race and ethnicity for research proposals and annual progress reports.⁶ The racial and ethnicity classifications recommended are based on the 1997 Office of Management and Budget (OMB) Directive.⁷ The U.S. Food and Drug Administration (FDA) has issued similar guidelines recommending that racial and ethnicity data should be collected for participants in U.S. clinical trials.⁸ Furthermore, the International Committee of Medical Journal Editors (ICMJE) developed a set of guidelines for reporting such descriptors and categorizations, which are followed by more than 500 journals. The guidelines state “when authors use variables such as race or ethnicity, they should define how they measured the variables and justify their relevance.”⁹ Some journals have developed additional guidelines on how race and ethnicity should be reported.¹⁰

Descriptors for race and ethnicity have changed over time as evidenced from the continually evolving categories in the U.S. Census since its inception.^{11, 12} Changes in terminology and differences between countries may lead to confusion and misinterpretation of conclusions involving race/ethnicity when making comparisons between studies.¹³ This study aimed to document the characterization of the race and ethnicity of clinical study populations in three top medical journals based in three different countries over the last three decades. In particular, we were interested in investigating the use of race/ethnicity descriptors in journals based in different English-speaking countries given the different descriptors used in Census-taking and the national make-up of each country. In addition to surveying the prevalence and types of descriptors used to describe the race and ethnicity of study populations, we compared the terms used in publications to those used in national census.

METHODS

Generation of Dataset

To assess the evolution of the use of race/ethnicity categories in biomedical research over the past three decades, we selected a sample of publications for analysis in three top medical journals based on impact factor, country of publication, and online access from our library: the Journal of the American Medical Association (JAMA; based in the United States), The Lancet (based in the United Kingdom) and the Canadian Medical Association Journal (CMAJ; based in Canada). A PubMed search of each journal was performed in April 2009 to identify a sample of articles published between 1981 and 2008. The PubMed search criteria included original research articles and clinical trials of human subjects and specifically excluded editorials, letters, reviews, commentaries and guidances. To achieve a feasible sample size, articles from every third year of publication from 1981 onwards were selected for our final dataset for analysis. This yielded a total of 1,867 journal articles: 464 from the JAMA, 1,318 from The Lancet, and 85 from CMAJ. Articles were then manually curated to ensure that the aforementioned criteria were met.

A numerical code was created for each data category that was captured to facilitate data entry and analysis. Each article was identified by its PubMed identification number. Several data-points were extracted from each article for analysis: journal, year/decade of publication, term(s) used to report race, ethnicity or heritage of the study subjects, the study trial location, and the country of the corresponding author. Papers with a single study population with respect to race were included in the dataset and coded accordingly to terminology used to describe population. Each article was reviewed and coded by one coder (P.K.). A random sample of articles (10%) was reviewed (S.B.H.) and agreement was greater than 95%. All data were recorded and stored in an Excel file for analysis.

Data Analysis

Frequency distributions, t-tests, and two proportion Z-tests (using α = 0.05) were conducted to determine the statistical significance of the bivariate distributions between the independent variables (e.g., journal and decade) and the dependent variable (race/ethnicity descriptors). More specifically, the usage of specific terms describing race/ethnicity was analyzed and compared within and between journals and each decade studied. In addition, we compared Census categories and use of race/ethnicity descriptors to explore the influence of the evolving Census categories on race/ethnicity descriptors used in the biomedical literature for each decade and country.

RESULTS

Prevalence of Articles Reporting Race/Ethnicity

Of the 1,867 journal publications analyzed overall, 17.2% of articles reported race/ethnicity: 39.22% of JAMA articles, 10.32% of The Lancet articles and 5.88% of CMAJ articles (Table 1). The reporting of race increased through the decades in JAMA from 14.08% of articles reporting in the 1980s to 40.66% in the 1990s (p<0.0001) and 46.45% in the 2000s. Similarly, the reporting of race/ethnicity increased in The Lancet from 3.68% of articles in the 1980s to 7.69% in 1990s (p=0.004) and 24.15% in 2000s (p<0.0001). In CMAJ, no articles reported race in the 1980s or 1990s, but 13.89% of articles reported race in the 2000s.

Table 1.

Prevalence of articles reporting race by journal and decade.

Journal/Decade	Percentage Reporting Race (actual)
JAMA
1980s (n=71)	14.08% (10)
1990s (n=182)	40.66% (74)*
2000s (n=211)	46.45% (98)
Overall (n=464)	39.22% (182)
The Lancet
1980s (n=462)	3.68% (17)
1990s (n=533)	7.77%** (41)
2000s (n=323)	24.15%*(78)
Overall (n=1,318)	10.32% (136)
CMAJ
1980s (n=22)	0.00% (0)
1990s (n=27)	0.00% (0)
2000s (n=36)	13.89% (5)
Overall (n= 85)	5.88% (5)
TOTAL (1,867)	17.30%

^*Significantly greater than previous decade (p-value<0.0001)

^**Significantly greater than previous decade (p-value<0.01)

Prevalence of Populations Reported

The number of study populations (based on the number of race/ethnicity descriptors reported) was calculated for all articles that described the race/ethnicity of study participants. The average number of populations reported per article in JAMA that reported race increased significantly from 2.40 to 2.70 between the 1980s and 1990s (p=0.004), and to 3.49 populations reported per article in the 2000s (p=0.04) (Figure 1). We also found significant differences in the number of populations described in The Lancet, with an average of 1.00 population per article reported in the 1980s, significantly increasing to 1.88 in the 1990s (p=0.03) and 2.14 in the 2000s (p = 0.01). The difference in the number of populations described between JAMA and The Lancet was statistically significant only in the 1990s (p = 0.02). A significant difference in the number of populations described in JAMA and CMAJ (p = 0.005) and The Lancet and CMAJ (p=0.01) in the 2000s was observed.

Figure 1.

Average number of populations reported by articles by journal and decade.

Variability of Race/Ethnicity Descriptors

Overall, there were a total of 104 descriptors used to describe race/ethnicity in the articles analyzed (60 of these were in JAMA; 61 in Lancet; 6 in CMAJ). However, after grouping of synonymous terms (based on presumed ancestry), there were 9, 12, and 5 unique groups in JAMA, Lancet, and CMAJ, respectively. The descriptor ‘Black’ was used most often to describe populations of African descent and ‘White’ was used to describe populations of European descent. In JAMA articles, the most common descriptor used overall was ‘White’ (100% of articles in the 1980s; 75.68% in the 1990s; 81.63% in the 2000s), followed by ‘Black’, ‘Hispanic’, and ‘Other’, throughout all three decades studied (Table 2). In The Lancet, the top descriptors used in the 1980s were ‘Gambian’ and ‘Chinese’, ‘White’ and ‘Caucasian’ in the 1990s were ‘White’ (65.85%), ‘Black’, ‘Other’, ‘African American’ and ‘Asian’; and in the 2000s were ‘White’ (78.21%), ‘Black’, ‘Hispanic’ and ‘Other’. In CMAJ in the 2000s, the most common descriptors were ‘White’ (100%), ‘Black’, ‘Asian’, ‘Other’, ‘Nonwhite’ and ‘South Asian’. The descriptors used in The Lancet became increasingly descriptive, combining race and ethnicity, and describing populations by nationality or region, such as ‘Arab’, ‘Chinese’, ‘Bangladeshi’, ‘Indian’ and ‘North African’. The use of these descriptors also aligns with the UK Census categories, which includes many nationalities in contrast to the U.S. Census categories.

Table 2.

Top race/ethnicity descriptors reported per journal and decade.

		JAMA			Lancet			CMJ
	Population	Total	%*	Population	Total	%*	Population	Total	%*
1980s	(Total number of articles reporting race/ethnicity=10)			(Total number of articles reporting race/ethnicity=17)
	White	10	100.00%	Gambian	4	23.53%
	Black	7	70.00%	Chinese	4	23.53%
	Hispanic	4	40.00%	White	2	11.76%
	Other	2	20.00%	Caucasian	2	11.76%
1990s	(Total number of articles reporting race/ethnicity=74)			(Total number of articles reporting race/ethnicity=41)
	White	56	75.68%	White	27	65.85%
	Black	32	43.24%	Black	10	24.39%
	Hispanic	25	33.78%	Other	10	24.39%
	Other	21	28.38%	African American	4	9.76%
				Asian	4	9.76%
2000s	(Total number of articles reporting race/ethnicity=98)			(Total number of articles reporting race/ethnicity=78)			(Total number of articles reporting race/ethnicity=5)
	White	80	81.63%	White	61	78.21%	White	5	100.00%
	Black	45	45.92%	Black	25	32.05%	Black	1	20.00%
	Hispanic	42	42.86%	Hispanic	14	17.95%	Asian	1	20.00%
	Other	37	37.76%	Other	18	23.08%	Other	1	20.00%
							Nonwhite	1	20.00%
							South Asian	1	20.00%

^*Percentages may be greater than 100% due to reporting of more than one population per article. Percentages were calculated based on total number of articles reporting race/ethnicity per journal and decade.

Relationship between Reporting of Race and Study Location

Studies published in JAMA and conducted in the U.S. wholly (N=123) or in part (international multi-site studies; N=14) were more likely to report race/ethnicity than non-U.S. studies (N=7) (p<0.0001.). However, studies published in the Lancet and conducted in the US in whole (N=21) or in part (N=11) were less likely to report race than studies that did not include a U.S. site (N=82) (p<0.0001.). No studies conducted in the U.S. published in CMAJ were included in our dataset.

DISCUSSION

Our findings suggest that while the use of race/ethnicity descriptors in the biomedical literature has increased, the high but ambiguous number of descriptors used to describe study populations may lead to misleading conclusions across studies. The role of race in biomedical research continues to be debated, yet the practice of reporting race/ethnicity is both accepted and expected, if not required. Some have expressed concerns that the excessive focus on differences in race and ethnicity may reify the racial divide and inadvertently ascribe to the biological variations between races.^{14, 15} Others argue that the social, economic, environmental and perhaps genetic factors that surround race and ethnicity can be useful for testing hypotheses regarding medical outcomes.¹⁶ Despite the FDA and OMB statements that race and ethnicity are not scientific, the organizations have upheld the belief that such descriptors are important for public health surveillance, research and collection of health-related data.⁸ Our analysis of a sample of biomedical articles published in three leading medical journals over the last three decades shows that a significant amount of disparity exists with respect to the reporting of race/ethnicity and that the addition of ethnicity may reflect greater prominence of cultural heritage as well as recognition of multi-racial heritage.

In summary, this study demonstrated three major findings: 1) the reporting of race/ethnicity of clinical trial participants has increased over the past three decades, though remains quite low; 2) the number of populations described per article is increasing, but continues to be dominated by “White” populations; and 3) when race/ethnicity is reported, study populations are described inconsistently between and within journals. Our finding regarding the overall low reporting of race/ethnicity (17.30%) is supported by several previous analyses,^17–19 though higher rates (>50%) have been reported in pediatric,^20–22 nursing,²³ and public health^{24, 25} research. Similarly, the increase in the number of articles reporting race over the past decades is supported by earlier work.^{21, 26} Despite the large number of terms used to describe study populations observed here and other studies,^{17, 25, 27} the most common group described and represented in clinical studies continues to be White.^{21, 27} The apparent homogeneity of clinical trial populations has been a continuing challenge over the past several decades,^{28, 29} though evidence suggests that the representation in U.S. cancer trials is consistent with disease burden in minority populations.³⁰ Corbie-Smith et al.¹⁹ found that studies of diseases associated with racial or ethnic disparities often did not report the race/ethnicity of participants or analyses based on such descriptors. Despite the reporting of race/ethnicity of study participants in some studies, it has been reported that few publications actually discuss race/ethnicity.²⁰

The inconsistent reporting of race/ethnicity poses a challenge for comparing clinical trials globally and, in some instances, even those conducted in the same country. Many journals have policies in place regarding the use of race/ethnicity in human studies, however, there is no clear standard for race/ethnicity descriptors or defining study populations.^{10, 31} The wide range of researchers’ views on the use and definition of race/ethnicity also likely contributes to the variation in practice observed.^{10, 32} The large number of terms used to describe seemingly similar groups in publications in JAMA, The Lancet and CMAJ may lead to erroneous comparisons of outcomes of presumably similar study populations. For example, in the U.K., ‘Asian’ could refer to persons of Central, South, or East Asian descent whereas in the U.S., the term refers to persons primarily of East or South Asia.^{33, 34} If race/ethnicity descriptors serve as surrogates for other variables potentially affecting health outcomes such as environment, the lack of a clear description of the population studied may lead to inaccurate comparisons. In addition, the difference in prevalence of reporting race between countries (higher in U.S.-based studies)^{26, 35} limits comparisons with global studies. Describing population groups based on nationality (e.g., Gambian) appears more prevalent for studies conducted outside of the U.S. and published in the Lancet, perhaps due to greater diversity and cultural propensity to describe nationality than use broad race descriptors.

Based on the sample of biomedical literature analyzed in this study, racial/ethnicity descriptors appear to have become more specific over time, incorporating ethnicity in the 1990s. Earlier publications from JAMA predominantly used terms that were used as the race categories in the U.S. Census. In contrast, publications in The Lancet, while also using terms such as ‘White’ and ‘Black’, included other terms not listed in their Census such as ‘Hispanic’ and ‘African American’. Compounding the problem of inconsistent reporting, many publications do not explain their criteria or methods for assigning race or ethnicity as an independent variable, despite the fact that many articles report conclusions about relationships between race/ethnicity, genetics and health outcomes.^{17, 25, 36, 37}

This ongoing ambiguity supports the development of a standard vocabulary as suggested by many others,^{13, 17, 24, 38–47} however, the strong national social and political values influencing race/ethnicity terminology would seriously impede global consensus. Alternatively, a new practice to accurately measure factors for which race serves as a surrogate could be developed and implemented such as collecting data on ancestry through grand-parental birthplace or using genetic ancestry markers.^48–50 The use of genetic markers would allow for consideration of admixture (mixed parentage),^51–53 however, it too is not an option without its own challenges.⁵⁴ First, genetic analysis of all clinical study populations would not be economically feasible until sequencing or genotyping technologies costs decline further and technical expertise becomes widespread and routine. Second, no standard set of markers has been developed, which could result in similar ambiguity in defining study participants’ ancestry. Third, by replacing race with genetic markers, we would potentially be overlooking environmental factors associated with or causative of health outcomes. Thus, genetic characterization alone would not be sufficient to describe the biological and/or social factors that may be important to a study’s outcomes and some combination of factors enabling more detailed characterization of research subjects’ ancestry and environment are needed.

In conclusion, even if reporting of study participants’ race/ethnicity continues to become more common, the lack on uniformity of the terms may pose problems for understanding the factors that affect health outcomes and lead to erroneous and potentially harmful conclusions. Thus, it becomes unclear how to interpret differences in study outcomes across populations in different countries, cultures, health systems and the like, beyond the U.S., the U.K. and Canada. Greater enforcement of existing journal policies regarding how populations are defined and clarification of the relevance of race/ethnicity to the study’s outcomes as well as development of international consensus of race/ethnicity descriptors or consideration of alternative methods to characterize study populations are needed.