Skip to main content
PLOS Medicine logoLink to PLOS Medicine
. 2021 Sep 14;18(9):e1003758. doi: 10.1371/journal.pmed.1003758

Evaluating the frequency of English language requirements in clinical trial eligibility criteria: A systematic analysis using ClinicalTrials.gov

Akila V Muthukumar 1, Walker Morrell 2,3, Barbara E Bierer 2,3,4,*
Editor: Margaret E Kruk5
PMCID: PMC8439488  PMID: 34520467

Abstract

Background

A number of prior studies have demonstrated that research participants with limited English proficiency in the United States are routinely excluded from clinical trial participation. Systematic exclusion through study eligibility criteria that require trial participants to be able to speak, read, and/or understand English affects access to clinical trials and scientific generalizability. We sought to establish the frequency with which English language proficiency is required and, conversely, when non-English languages are affirmatively accommodated in US interventional clinical trials for adult populations.

Methods and findings

We used the advanced search function on ClinicalTrials.gov specifying interventional studies for adults with at least 1 site in the US. In addition, we used these search criteria to find studies with an available posted protocol. A computer program was written to search for evidence of English or Spanish language requirements, or the posted protocol, when available, was manually read for these language requirements. Of the 14,367 clinical trials registered on ClinicalTrials.gov between 1 January 2019 and 1 December 2020 that met baseline search criteria, 18.98% (95% CI 18.34%–19.62%; n = 2,727) required the ability to read, speak, and/or understand English, and 2.71% (95% CI 2.45%–2.98%; n = 390) specifically mentioned accommodation of translation to another language. The remaining trials in this analysis and the following sub-analyses did not mention English language requirements or accommodation of languages other than English. Of 2,585 federally funded clinical trials, 28.86% (95% CI 27.11%–30.61%; n = 746) required English language proficiency and 4.68% (95% CI 3.87%–5.50%; n = 121) specified accommodation of other languages; of the 5,286 industry-funded trials, 5.30% (95% CI 4.69%–5.90%; n = 280) required English and 0.49% (95% CI 0.30%–0.69%; n = 26) accommodated other languages. Trials related to infectious disease were less likely to specify an English requirement than all registered trials (10.07% versus 18.98%; relative risk [RR] = 0.53; 95% CI 0.44–0.64; p < 0.001). Trials related to COVID-19 were also less likely to specify an English requirement than all registered trials (8.18% versus 18.98%; RR = 0.43; 95% CI 0.33–0.56; p < 0.001). Trials with a posted protocol (n = 366) were more likely than all registered clinical trials to specify an English requirement (36.89% versus 18.98%; RR = 1.94, 95% CI 1.69–2.23; p < 0.001). A separate analysis of studies with posted protocols in 4 therapeutic areas (depression, diabetes, breast cancer, and prostate cancer) demonstrated that clinical trials related to depression were the most likely to require English (52.24%; 95% CI 40.28%–64.20%). One limitation of this study is that the computer program only searched for the terms “English” and “Spanish” and may have missed evidence of other language accommodations. Another limitation is that we did not differentiate between requirements to read English, speak English, understand English, and be a native English speaker; we grouped these requirements together in the category of English language requirements.

Conclusions

A meaningful percentage of US interventional clinical trials for adults exclude individuals who cannot read, speak, and/or understand English, or are not native English speakers. To advance more inclusive and generalizable research, funders, sponsors, institutions, investigators, institutional review boards, and others should prioritize translating study materials and eliminate language requirements unless justified either scientifically or ethically.


Akila Muthukumar and coauthors, systematically analyze ClinicalTrials.gov to evaluate the frequency of English language requirements in clinical trial eligibility criteria.

Author summary

Why was this study done?

  • Some clinical trials in the US exclude individuals who do not read, speak, or write English.

  • While requiring English language proficiency for entry into clinical trials may sometimes be scientifically or ethically justified, often it is not, raising concerns of equity and justice.

  • We sought to establish the frequency of English language requirements in clinical trials in the United States.

What did the researchers do and find?

  • We reviewed 14,367 US clinical trials registered on ClinicalTrials.gov in 2019 and 2020.

  • Of the 14,367 clinical trials, 18.98% (n = 2,727) had English language requirements, and 2.71% (n = 390) mentioned accommodation of a language other than English.

  • Clinical trials funded by the federal government were more likely to require English than clinical trials funded by the life sciences and pharmaceutical industries.

  • Compared to all clinical trials in 2019 and 2020, clinical trials related to COVID-19 and other infectious diseases were less likely to have English language requirements.

  • In a separate analysis of clinical trials in 4 therapeutic areas (depression, diabetes, breast cancer, and prostate cancer), trials related to depression were the most likely to require English proficiency (52.24%).

What do these findings mean?

  • A meaningful percentage of clinical trials in the US exclude individuals who do not speak, read, or write English.

  • Clinical trials can be made more inclusive by eliminating language requirements that are not scientifically or ethically justified.

Introduction

The populations enrolled in clinical trials should optimally reflect the characteristics of the populations for whom the knowledge gained from the research is applicable [1]. While restricting clinical trial participation for scientific or ethical reasons, such as safety concerns or protection of vulnerable populations, is justified, the intentional or systematic exclusion of subgroups of the population for whom the research is intended is problematic. Scientifically, such exclusion limits the generalizability of the research findings, resulting, inter alia, in an insufficient evidence base for the safe and effective use of medicines and other interventions. Such exclusion is also ethically problematic in that, as a principle of justice, the selection of research participants is not equitable; exclusion may also contribute to distrust of the clinical research enterprise and healthcare systems, institutions, and providers. Finally, the exclusion contributes to health inequities, a problem that is particularly salient when applied to individuals prohibited from participating simply for their lack of English proficiency in the absence of other reasons.

Requiring English proficiency in a clinical trial may be scientifically justified, such as when a trial involves surveys, assessments, or outcome measures that have been validated only in English, at least in an effort not to delay the initiation of research due to translation and validation. Research evaluating or using mobile health technologies available only in English will also require English proficiency. However, no such scientific justification exists for the exclusion of non-English speakers from clinical trials for pragmatic reasons such as the cost of translation, or inadequate training or availability of language-concordant staff or interpreters [2]. And in interventional therapeutic trials, where there is a possibility of direct benefit to the participant, the exclusion of individuals based on language alone is itself inequitable.

A number of prior studies have demonstrated that research participants with limited English proficiency in the United States are routinely excluded from clinical trial participation; eligibility criteria require that trial participants be able to speak, read, and/or understand English [35]. These studies have routinely called for a reexamination of this requirement and the translation of study materials to different languages. The term “limited English proficiency” is used here to describe individuals who have a limited ability to read, speak, write, or understand English. In the US, an estimated 57 million people do not speak English, and an additional 25 million people may be defined as having limited English proficiency [6].

We sought to establish the frequency with which English proficiency is required in clinical trials in the US. Using the eligibility criteria of clinical trials registered on ClinicalTrials.gov, and posted study protocols when available, we evaluated the frequency of language requirements in eligibility criteria, and we further characterized the requirement by funding source and across several therapeutic areas. We also reviewed the frequency with which non-English languages are affirmatively accommodated in clinical trials in the US. We discuss the common—and discriminatory—exclusion of individuals who do not speak, read, or understand English from clinical trials in the US and the ethical challenges raised by this exclusion.

Methods

We used the advanced search function on ClinicalTrials.gov to search for interventional studies for adults (18–64 years) and older adults (65+ years) with at least 1 site in the US with an actual start date between 1 January 2019 and 1 December 2020. We performed additional searches for interventional studies for adults (18–64 years) and older adults (65+ years) with at least 1 site in the US across 4 therapeutic areas (depression, diabetes, breast cancer, and prostate cancer). We restricted these additional searches to studies that had posted a study protocol to ClinicalTrials.gov, and searched over a 4-year time interval, or longer if needed, so that the search results contained at least 50 trials. The criteria for each search can be found in Table A in S1 Tables.

The comma-separated values (CSV) files of the search results were downloaded. The funding source data element for each trial was coded as “federal,” “industry,” “federal/industry,” or “other” using the categories outlined in Table B in S1 Tables.

Data analysis followed a prospective analysis plan. We wrote a computer program in R to parse each trial’s ClinicalTrials.gov webpage entry for the terms “English” and “Spanish” for all trials with an actual start date between 1 January 2019 and 1 December 2020. If the program identified either term on a trial’s ClinicalTrials.gov webpage entry, it printed the line of text in which the term was found.

One person (AM or WM) manually reviewed the printed text and categorized each trial as one that (1) requires English, (2) accommodates translation, (3) mentions that participants must be able to provide informed consent and/or communicate with study staff via phone call or email, or (4) does not mention any language requirement for participation (“no mention”). After reviewing a sample of clinical trials, we determined that a statement about the ability to provide informed consent is a common inclusion criterion and independent of language. Since there was no relevant distinction between categories 3 and 4, these 2 categories were pooled for further analysis. Further, since Spanish is the second most common language in the US, we noted the number of trials that specifically mentioned the accommodation of Spanish as a subcategory of category 2.

Details on the specific entries that informed language categorization are provided in Table 1.

Table 1. Key phrases used to code language requirements in clinical trials.

Classification ClinicalTrials.gov entry
Requires English Native English speakers
Ability to read English
Ability to speak English
Ability to understand English
Legally authorized representative must read, speak, understand English
We have no non-English speaking patients in this population
Accommodates translation Mentions another language by name: Spanish, American Sign Language, Korean, Xhosa, Swahili, Luganda, French, Arabic, Mandarin, Cantonese, etc.
Non-English speakers will be accommodated
If applicable, [informed consent] will be provided in a certified translation of the local language
Non-English language speaking participants for whom an Institutional Review Board (IRB) approved short form is available
The informed consent form must be written in a language fully comprehensible to the prospective patient
Any oral or written information will be provided to participants in their own language
Patients unable to read/write English are eligible to participate in the overall study but will not be required to participate in the Patient-Reported Outcome questionnaires
Self-reported questionnaires will be administered in countries where the questionnaires have been translated into the native language of the region and linguistically validated
Standard Operating Procedures (SOPs) are in place pertaining to how to approach and consent participants of limited English proficiency

If the terms “English” or, for accommodation of language translation, “Spanish” were not found by the computer program and a protocol was posted, one person categorized the trial based on the study protocol. If no protocol was posted, the trial was categorized as “no mention.” This method was chosen so that findings of English language requirements were conservative and not overestimated.

To verify reliability of the computer program, we manually reviewed all trials in 1 therapeutic area (diabetes, n = 85). We found 100% agreement with the results generated by the computer program. The computer program is available at https://github.com/akilamuthukumar/languagesearchfunctions and https://rpubs.com/akilamuthukumar/709377. The data are available upon request.

Four therapeutic areas (depression, diabetes, breast cancer, and prostate cancer) for which study protocols had been uploaded to ClinicalTrials.gov were further examined. Two independent coders (AM and WM) manually reviewed and classified the ClinicalTrials.gov webpage entry for each trial. If language requirements were not identified on a trial’s webpage entry, the 2 coders reviewed the trial protocol and categorized the trial as one that (1) requires English, (2) accommodates translation, or (3) does not mention any language requirement for participation. The inter-rater reliability was 95.3%, and entry errors (n = 4) were corrected. Discrepancies (n = 9) were reviewed by an independent third party (BEB) and discussed until consensus was reached.

The frequencies of language requirements were determined, and 95% confidence intervals were calculated. Comparisons were made between proportions of trials by calculating the relative risk (RR). 95% confidence intervals for these comparisons were calculated on the natural log scale and converted to a linear scale through exponentiation. p-Values were calculated using a 2-tailed z-test of 2 proportions at significance level 0.001.

In order to determine whether there was a relationship between the accommodation of a non-English language and increased diverse representation in clinical trials, we reviewed the ethnicity data in the trials with an actual start date between 1 January 2019 and 1 December 2020 that had posted results to ClinicalTrials.gov.

Results

In total, 14,367 interventional clinical trials met the advanced search criteria of having at least 1 site in the US, being intended for adults/older adults, and being registered on ClinicalTrials.gov between 1 January 2019 and 1 December 2020 (Table 2). Of these clinical trials, having the ability to read, speak, and/or understand English, or being a native English speaker, was required by 18.98% (95% CI 18.34%–19.62%; n = 2,727), while 2.71% (95% CI 2.45%–2.98%; n = 390) specifically mentioned accommodation of translation to another language. The majority (78.3%; 95% CI 77.63%–78.98%; n = 11,250) did not mention any language requirement.

Table 2. Language requirements in clinical trials.

Trial type Total Number (percent, 95% CI)
Requires English Accommodates translation Accommodates Spanish Does not mention any language requirement
All trials 14,367 2,727 (18.98, 18.34–19.62) 390 (2.71, 2.45–2.98) 328 (2.28, 2.04–2.53) 11,250 (78.30, 77.63–78.98)
Infectious disease trials 1,023 103 (10.07, 8.22–11.91) 32 (3.13, 2.06–4.19) 31 (3.03, 1.98–4.08) 888 (86.80, 84.73–88.88)
COVID-19 trials 611 50 (8.18, 6.01–10.36) 15 (2.45, 1.23–3.68) 15 (2.45, 1.23–3.68) 546 (89.36, 86.92–91.81)
Trials with posted protocol 366 135 (36.89, 31.94–41.83) 18 (4.92, 2.70–7.13) 15 (4.10, 2.07–6.13) 213 (58.20, 53.14–63.25)
Federally funded trials 2,585 746 (28.86, 27.11–30.61) 121 (4.68, 3.87–5.50) 105 (4.06, 3.30–4.82) 1,718 (66.46, 64.64–68.28)
Industry funded trials 5,286 280 (5.30, 4.69–5.90) 26 (0.49, 0.30–0.69) 20 (0.38, 0.21–0.54) 4,980 (94.21, 93.58–94.84)

The 95% confidence intervals are reported as percentages of trials that meet the cell characteristic.

Of the 14,367 trials, 366 (2.55%) had posted a study protocol. Notably, these trials with a protocol were more likely than all registered clinical trials to specify an English requirement (36.89% versus 18.98%; RR = 1.94, 95% CI 1.69–2.23; p < 0.001). Of the 366 trials, 11.8% (95% CI 8.21%–14.74%; n = 42) had information about English language requirements available only in the posted study protocol and not on the trial’s ClinicalTrials.gov webpage entry.

Differences in language requirements by funding source

The relationship between funding source and language requirement was examined. Trials that were co-funded by both federal and industry sources (e.g., industry/National Institutes of Health) were excluded from the analysis by funding source. Federally funded trials were more likely to specify an English requirement than industry-funded trials (28.86% versus 5.30%; RR = 5.45; 95% CI 4.79–6.20; p < 0.001). Federally funded trials were also more likely to specify accommodation of translation than industry-funded trials (4.68% versus 0.49%; RR = 9.52; 95% CI 6.18–14.65; p < 0.001) (Table 2).

Differences in language requirements by therapeutic area

We hypothesized that interventional studies addressing the COVID-19 pandemic may be more accommodating of other languages (either by not requiring English proficiency or specifically mentioning additional translation, such as Spanish) given the disproportionate prevalence of infection in Hispanic and Latino populations in the US [7,8]. Of the 14,367 clinical trials we reviewed across 2019 and 2020, trials related to infectious diseases (n = 1,023) of which the majority (59.7%) were related to COVID-19 (n = 611), were less likely to specify an English requirement than all registered trials (trials related to infectious diseases versus all trials: 10.07% versus 18.98%; RR = 0.53; 95% CI 0.44–0.64; p < 0.001; trials related to COVID-19 versus all trials: 8.18% versus 18.98%; RR = 0.43; 95% CI 0.33–0.56; p < 0.001).

We observed that reviewing the eligibility criteria of posted study protocols was a more accurate method of determining language requirements than reviewing the selected criteria available on a trial’s ClinicalTrials.gov webpage entry. In order to examine whether there were differences in other therapeutic areas, we reviewed registered trials with an available study protocol over a 4-year time interval, or longer if needed, to obtain at least 50 trials. There were clear differences in the requirement for English proficiency by therapeutic area: Requirements in trials related to depression (35/67; 95% CI 40.28%–64.20%) exceeded those related to diabetes (25/85; 95% CI 19.73%–39.10%), breast cancer (13/70; 95% CI 9.46%–27.68%), and prostate cancer (4/55; 95% CI 0.41%–14.14%) (Table 3). Mandating English as a requirement did not correlate with likelihood of accommodating translation, at least in this small sample.

Table 3. Language requirements in clinical trials by therapeutic area.

Therapeutic area Total Number (percent, 95% CI)
Requires English Accommodates translation Accommodates Spanish* Does not mention any language requirement
Depression 67 35 (52.24, 40.28–64.20) 5 (7.46, 1.17–13.76) 0 (NA) 27 (40.30, 28.55–52.04)
Diabetes 85 25 (29.41, 19.73–39.10) 13 (15.29, 7.64–22.95) 8 (9.41, 3.20–15.62) 47 (55.29, 44.72–65.86)
Breast cancer 70 13 (18.57, 9.46–27.68) 15 (21.43, 11.82–31.04) 5 (7.14, 1.11–13.18) 42 (60.00, 48.52–71.48)
Prostate cancer 55 4 (7.27, 0.41–14.14) 8 (14.55, 5.23–23.86) 0 (NA) 43 (78.18, 67.27–89.10)

*Number of trials specifying Spanish is a subset of the trials accommodating translation.

NA, not applicable.

Non-English-language accommodation and clinical trial enrollment

To determine whether accommodation of non-English language correlated with an increase in the ethnicities of research participants, we reviewed trials with posted results. Of the 14,367 clinical trials initiated in 2019 and 2020, only 323 had posted results on ClinicalTrials.gov. Of the 323 trials that had posted results, only 189 reported ethnicity data, and of those 189 trials, only 4 were identified as specifically mentioning accommodation of a non-English language. The small sample size of completed studies with posted results impeded any correlative analyses on non-English-language accommodation and the ethnicities of research participants.

Discussion

While many trials do not include a language requirement as an eligibility criterion, this study found that a substantial percentage of interventional clinical trials for adults in the US required that participants be native English speakers or be able to speak, read, and/or understand English. Exclusion from participation on the basis of language varied by therapeutic area and by funding source.

The specific eligibility criteria describing the English language requirements in clinical trials varied in wording; the intention and impact of that variability are uncertain. Whether “native English speaker” requirements differentiate between native English speakers and non-native English speakers who are nevertheless proficient in English, for instance, and whether research staff actually distinguish between the 2 is unclear. Similarly, the scientific reasons for clinical trial requirements for participants to be able to speak, read, or understand English, or possess the capacity for all 3, were neither explicit nor evident. Further, very few protocols included methods for determining English proficiency. How these requirements are applied by study staff, therefore, is subject to interpretation.

In comparison to the proportion of clinical trials that specified English requirements, fewer clinical trials explicitly accommodated translation into languages other than English. It is possible that this number is underestimated since trials that did not specify translation on ClinicalTrials.gov may have been registered on other clinical trial registries in non-English languages, trial protocols were not always available for review, and the computer program only searched for the terms “English” and “Spanish.” While the finding of an affirmative requirement for English proficiency, therefore, is of greater reliability, the infrequency with which translation was mentioned, even in interventional trials addressing the COVID-19 pandemic, was notable. As a related point, it should not be assumed that clinical trials categorized as “accommodates translation” are sufficiently inclusive. Any trial that accommodated at least 1 other language was categorized as “accommodates translation,” but just and appropriate inclusion of diverse participants often requires accommodation of more than a single additional language.

The information available on ClinicalTrials.gov leads to an underestimation of the language requirements of clinical trial eligibility compared to posted study protocols. The responsible party for registering the clinical trial has latitude in which eligibility criteria they choose to include on the trial’s ClinicalTrials.gov webpage entry [9]; neither the ClinicalTrials.gov Protocol Registration and Results System [10] nor Section 801 of the Food and Drug Administration Amendments Act of 2007 (FDAAA 801) [11] requires language proficiency to be an essential element of registration. The finding that the subset of clinical trials with a posted protocol were more likely to specify language requirements as part of the eligibility criteria compared to all clinical trials suggests that some responsible parties do not consider language requirements to be essential information. We recommend that responsible parties include all eligibility criteria, including language requirements, on ClinicalTrials.gov, as this information is important for clinicians, patients, families, and others to identify clinical trial opportunities. If all eligibility criteria are not included in the trial’s ClinicalTrials.gov website entry, then responsible parties should, at a minimum, upload the trial protocol containing such information to ClinicalTrials.gov.

It is surprising that federally funded trials were not only more likely to require English but also more likely to accommodate translation than industry-funded trials. Industry-funded trials are often better resourced and more multinational than federally funded trials, and therefore translation may be less likely to be mentioned. In contrast, when federally funded trials have the capacity for translation, the investigators may be more likely to make this capacity explicit.

Of the therapeutic areas analyzed, clinical trials for depression were the most likely to require English proficiency (52.24%). Whether this disproportionate requirement for English proficiency is related to the necessity of language concordance between participant and investigator, the limited availability of validated data collection and survey instruments in non-English languages, the more qualitative aspect of this research, or another reason is not clear; study protocols routinely failed to provide an explanation. It is noteworthy that trials for both breast and prostate cancer were less likely than trials for depression or diabetes to require English proficiency, a difference potentially related to cancer outcome measures being derived from imaging or laboratory studies, as well as to a focus on inclusion of diverse populations in oncology [12]. However, these results are based on small numbers and subject to variation; additional research is needed. That interventional studies addressing infectious diseases in general, and COVID-19 specifically, less often required English proficiency compared to all clinical trials is reassuring, given the prevalence and severity in Hispanic and Latino populations.

There are 2 limitations to the study methodology. One limitation is that the computer program only searched for the terms “English” and “Spanish”; if neither of these terms was identified, and a protocol was not available for manual review, language requirements that did not use the terms “English” or “Spanish” may have been overlooked. Another limitation is that the requirements to read, speak, and/or understand English and/or be a native English speaker were grouped together in the category of English language requirements.

Our findings highlight the prevalence of the routine exclusion of adult non-English-speaking individuals from interventional clinical trials with at least 1 site in the US. In some cases, exclusion may be justified for scientific or methodological reasons, such as the unavailability of a validated data collection tool in languages other than English; one would hope, however, that barriers of this nature would be temporary and other tools would be identified or developed. In some cases, language concordance between participant and investigator or research staff may be necessary, although whether that need is scientific or a matter of convenience and cost is unclear. Clinical care (including psychiatric care) is able to cross a language divide through translation, interpreters, and other means; clinical research should be held to the same expectation. In addition to depriving individuals of access to clinical trials and, most acutely, to those trials with potential therapeutic benefit, lack of appropriate representation in clinical trials limits the generalizability of the results of the research.

Funders and sponsors, institutions, investigators and their study teams, institutional review boards, and other stakeholders have the capacity to redress the problem. Funders and sponsors could include translation as an allowable cost in grant applications or contracts or provide translated documents as a matter of course. A short-form consent document summarizing the basic elements of informed consent in a non-English language is sometimes used to document that the elements of informed consent were presented orally, but such a document does not offer study-specific information. This approach may be used, but optimally should be a temporary fix. Future work should delineate the conditions under which use of a short-form consent document is adequate and the conditions under which translation of the informed consent and other study materials is necessary or expected. Institutions and investigators with access to translation services for clinical purposes could extend the services to research, and provision of interpreter and translation resources might incentivize inclusion of diverse populations. Institutional review boards could develop standardized guidance for investigators and their study teams since such guidance does not routinely exist currently [13,14]. These strategies and others, coupled with appropriate resources, will help advance more inclusive and generalizable research.

Supporting information

S1 Tables

Table A: ClinicalTrials.gov advanced search criteria. Table B: ClinicalTrials.gov funder type classification.

(DOCX)

Acknowledgments

We thank Graeme Peterson for assistance with the statistical techniques used in this paper.

Abbreviation

RR

relative risk

Data Availability

All relevant data are within or cited in the manuscript or available from ClinicalTrials.gov.

Funding Statement

The authors received no specific funding for this work.

References

  • 1.Multi-Regional Clinical Trials Center of Brigham and Women’s Hospital and Harvard. Diversity, inclusion, and equity in clinical research. Boston: Multi-Regional Clinical Trials Center of Brigham and Women’s Hospital and Harvard; 2021. [cited 2021 May 26]. https://mrctcenter.org/diversity-in-clinical-research/. [Google Scholar]
  • 2.Staples JN, Lester J, Li A, Walsh C, Cass I, Karlan BY, et al. Language as a barrier to cancer clinical trial accrual: assessing consenting team knowledge and practices for cancer clinical trial consent among low English fluency patients. Appl Cancer Res. 2018;38(1):1–7. doi: 10.1186/s41241-018-0065-9 [DOI] [Google Scholar]
  • 3.Glickman SW, Ndubuizu A, Weinfurt KP, Hamilton CD, Glickman LT, Schulman KA, et al. Perspective: the case for research justice: inclusion of patients with limited English proficiency in clinical research. Acad Med. 2011;86(3):389–93. doi: 10.1097/ACM.0b013e318208289a [DOI] [PubMed] [Google Scholar]
  • 4.Egleston BL, Pedraza O, Wong YN, Dunbrack RL, Griffin CL, Ross EA, et al. Characteristics of clinical trials that require participants to be fluent in English. Clin Trials. 2015;12(6):618–26. doi: 10.1177/1740774515592881 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Livaudais-Toman J, Burke NJ, Napoles A, Kaplan CP. Health literate organizations: are clinical trial sites equipped to recruit minority and limited health literacy patients? J Health Dispar Res Pract. 2014;7(4):1–13. [PMC free article] [PubMed] [Google Scholar]
  • 6.Betancourt JR, Renfrew MR, Green AR, Lopez L, Wasserman M. Improving patient safety systems for patients with limited English proficiency: a guide for hospitals. Rockville (MD): Agency for Healthcare Research and Quality; 2012. [cited 2021 May 26]. https://www.ahrq.gov/sites/default/files/publications/files/lepguide.pdf. [Google Scholar]
  • 7.Podewils LJ, Burket TL, Mettenbrink C, Steiner A, Seidel A, Scott K, et al. Disproportionate incidence of COVID-19 infection, hospitalizations, and deaths among persons identifying as Hispanic or Latino—Denver, Colorado March–October 2020. MMWR Morb Mortal Wkly Rep. 2020;69(48):1812–6. doi: 10.15585/mmwr.mm6948a3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Andrasfay T, Goldman N. Reductions in 2020 US life expectancy due to COVID-19 and the disproportionate impact on the Black and Latino populations. Proc Natl Acad Sci U S A. 2021;118(5):e2014746118. doi: 10.1073/pnas.2014746118 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.National Institutes of Health. Clinical trials registration and results information submission. Washington (DC): Department of Health and Human Services; 2016. [cited 2021 May 26]. https://www.federalregister.gov/documents/2016/09/21/2016-22129/clinical-trials-registration-and-results-information-submission. [Google Scholar]
  • 10.ClinicalTrials.gov. Submit studies to ClinicalTrials.gov PRS. Bethesda (MD): ClinicalTrials.gov; 2021 [cited 2021 May 26]. https://clinicaltrials.gov/ct2/manage-recs/submit-study.
  • 11.Food and Drug Administration Amendments Act of 2007. Public Law 110–85. Title VIII—Clinical Trial Databases. 2007 Sep 27 [cited 2021 May 26]. https://www.govinfo.gov/content/pkg/PLAW-110publ85/pdf/PLAW-110publ85.pdf#page=82.
  • 12.Ford JG, Howerton MW, Lai GY, et al. Barriers to recruiting underrepresented populations to cancer clinical trials: a systematic review. Cancer. 2008;112(2):228–42. doi: 10.1002/cncr.23157 [DOI] [PubMed] [Google Scholar]
  • 13.Klitzman R. How US institutional review boards decide when researchers need to translate studies. J Med Ethics. 2014;40(3):193–7. doi: 10.1136/medethics-2012-101174 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Torres R, Kovacich J. Communication methods with limited english proficiency research participants to acquire informed consent during clinical trials. The Exchange. 2014;3(1):82–96. [Google Scholar]

Decision Letter 0

Beryne Odeny

5 Apr 2021

Dear Dr Bierer,

Thank you for submitting your manuscript entitled "English Proficiency Requirements in Clinical Trial Eligibility: Common, Persistent, and Inequitable" for consideration by PLOS Medicine.

Your manuscript has now been evaluated by the PLOS Medicine editorial staff and I am writing to let you know that we would like to send your submission out for external peer review.

However, before we can send your manuscript to reviewers, we need you to complete your submission by providing the metadata that is required for full assessment. To this end, please login to Editorial Manager where you will find the paper in the 'Submissions Needing Revisions' folder on your homepage. Please click 'Revise Submission' from the Action Links and complete all additional questions in the submission questionnaire.

Please re-submit your manuscript within two working days, i.e. by .

Login to Editorial Manager here: https://www.editorialmanager.com/pmedicine

Once your full submission is complete, your paper will undergo a series of checks in preparation for peer review. Once your manuscript has passed all checks it will be sent out for review.

Feel free to email us at plosmedicine@plos.org if you have any queries relating to your submission.

Kind regards,

Beryne Odeny

Associate Editor

PLOS Medicine

Decision Letter 1

Beryne Odeny

11 May 2021

Dear Dr. Bierer,

Thank you very much for submitting your manuscript "English Proficiency Requirements in Clinical Trial Eligibility: Common, Persistent, and Inequitable" (PMEDICINE-D-21-01563R1) for consideration at PLOS Medicine.

Your paper was evaluated by a senior editor and discussed among all the editors here. It was also discussed with an academic editor with relevant expertise, and sent to independent reviewers, including a statistical reviewer. The reviews are appended at the bottom of this email and any accompanying reviewer attachments can be seen via the link below:

[LINK]

In light of these reviews, I am afraid that we will not be able to accept the manuscript for publication in the journal in its current form, but we would like to consider a revised version that addresses the reviewers' and editors' comments. Obviously we cannot make any decision about publication until we have seen the revised manuscript and your response, and we plan to seek re-review by one or more of the reviewers.

In revising the manuscript for further consideration, your revisions should address the specific points made by each reviewer and the editors. Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments, the changes you have made in the manuscript, and include either an excerpt of the revised text or the location (eg: page and line number) where each change can be found. Please submit a clean version of the paper as the main article file; a version with changes marked should be uploaded as a marked up manuscript.

In addition, we request that you upload any figures associated with your paper as individual TIF or EPS files with 300dpi resolution at resubmission; please read our figure guidelines for more information on our requirements: http://journals.plos.org/plosmedicine/s/figures. While revising your submission, please upload your figure files to the PACE digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at PLOSMedicine@plos.org.

We expect to receive your revised manuscript by Jun 01 2021 11:59PM. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

We ask every co-author listed on the manuscript to fill in a contributing author statement, making sure to declare all competing interests. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. If new competing interests are declared later in the revision process, this may also hold up the submission. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT. You can see our competing interests policy here: http://journals.plos.org/plosmedicine/s/competing-interests.

Please use the following link to submit the revised manuscript:

https://www.editorialmanager.com/pmedicine/

Your article can be found in the "Submissions Needing Revision" folder.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

We look forward to receiving your revised manuscript.

Sincerely,

Beryne Odeny,

PLOS Medicine

plosmedicine.org

-----------------------------------------------------------

Requests from the editors:

Thank you for your submission. Before we proceed, please address the following editorial and reviewer comments.

1) Please revise your title according to PLOS Medicine's style. Your title must be nondeclarative and not a question. It should begin with main concept if possible. Please place the study design (e.g. "A cross-sectional study" or “systematic review”) in the subtitle (i.e., after a colon).

2) Abstract summary - At this stage, we ask that you reformat your non-technical Author Summary. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract. The summary should be accessible to a wide audience that includes both scientists and non-scientists. Please see our author guidelines for more information: https://journals.plos.org/plosmedicine/s/revising-your-manuscript#loc-author-summary.

3) In the abstract Methods and Findings:

a) Please revise the subheading “Methods/Finding” to “Methods and Findings,”

b) Please ensure that all numbers presented in the abstract are present and identical to numbers presented in the main manuscript text.

c) Please quantify the main results with both 95% CIs and p values.

d) In the last sentence of the Abstract Methods and Findings section, please describe the main limitation(s) of the study's methodology.

4) Did your study have a prospective protocol or analysis plan? Please state this (either way) early in the Methods section.

a) If a prospective analysis plan (from your funding proposal, IRB or other ethics committee submission, study protocol, or other planning document written before analyzing the data) was used in designing the study, please include the relevant prospectively written document with your revised manuscript as a Supporting Information file to be published alongside your study, and cite it in the Methods section. A legend for this file should be included at the end of your manuscript.

b) If no such document exists, please make sure that the Methods section transparently describes when analyses were planned, and when/why any data-driven changes to analyses took place.

c) In either case, changes in the analysis-- including those made in response to peer review comments-- should be identified as such in the Methods section of the paper, with rationale.

5) In the Methods and Results section:

a) Please provide 95% CIs and p values for estimates in the main text and tables

b) When a p value is given, please specify the statistical test used to determine it.

c) Please do not report P<0.01; report as P < 0.001.

6) Figures and tables:

a) The presentation in table 3 is confusing. Please remove “n(%)” from the rows and instead write it at the top of the last 2 columns(“Requires English” and “Accommodates translation”)

b) Please define the following abbreviations in your tables 1: ICF, ASL.

c) In table 1, please replace "subject" with participant, patient, individual, or person.

7) Please look separately at Spanish in the studies since that is the second most frequently spoken language and note how often that is mentioned/accommodated

8) Please use the "Vancouver" style for reference formatting and see our website for other reference guidelines https://journals.plos.org/plosmedicine/s/submission-guidelines#loc-references. . Please ensure that weblinks are current and accessible

Comments from the reviewers:

Reviewer #1: I confine my remarks to statistical aspects of this paper.

While what was done isn't really wrong, I don't think it's ideal I am concerned that over three-quarters of the studies did not mention translation. This is a kind of missing data problem, but the usual methods of multiple imputation are likely to fail, since so much data was missing. Therefore, rather than use z-tests of two proportions, I would a) Simply look at the proportion that do accommodate translation and b) use multinomial logistic regression to examine whether the variable "translation" (with three levels) varies by any other trait you are interested in. This treats "missing" as its own category, which is, I think, a better approach than simply ignoring them.

Peter Flom

Reviewer #2: This is a well-written and interesting paper that addresses an important issue in clinical trial eligibility. Strengths of the paper include a thoughtful introduction highlighting the importance of removing barriers to research participation to increase diversity in clinical trials and the use of the use of a rigorous and reproducible search strategy to identify language requirements for clinical trials with at least one US site. The main findings include 19.0% of trials on ClinicalTrials.gov specifying an English language requirement and only 2.7% specifically mentioning accommodation of translation to another language. Compared to posted protocols, the information in ClinicalTrials.gov under-reported both English language requirements and accommodations for translation. There are a number of things that could be done to further strengthen the paper, both in terms of the results presented and the discussion.

Major Comments:

1) It would be particularly helpful, and a greater contribution to the literature, if the authors could demonstrate a relationship between accommodation for translation and actual increased diversity in clinical trials. While this would not be possible for all languages, it seems like it should be possible to search information on ClinicalTrials.gov to determine whether accommodation of translation resulted in overall greater proportions of non-white participants. Another approach would be to examine the relation between accommodation of Spanish translation and the proportion of Hispanic participants. This information would provide the missing link in the mechanism linking translation to participation.

2) Among the possible accommodations for translated documentation, it seems like a missed opportunity not to comment on the potential for using the consent short form (21 CFR 50.27).

3) An additional important point for inclusion of non-English speaking participants in trials is to modify the incentives and disincentives for sites to include these participants. These include covering not only the translation costs but the costs associated with use of interpreter services and of added staff time required for visits conducted using an interpreter.

4) There should be at least mention of the distinction between clinical trial sites potentially being able to offer staff fluent in some languages (e.g. Spanish) and clinical trial sites with truly broader language capacity (e.g. through interpreter services), which could accommodate a much wider range of languages. Stated another way, the way this analysis is presented, all trials would be 'inclusive' if they just included Spanish - which may be easier to achieve. However, true equity in access to trials will require moving beyond a single additional language.

Reviewer #3: This is a timely and important topic, and I commend the authors for submitting it to a systematic analysis. It is to be hoped that studies such as this spark a trend that will bring more equitable representation in clinical research. Below are specific comments:

Abstract: the percentages under the results section do not add up to 100, clarify that the rest of the percentage were not eligible for inclusion, did not specify language, etc.

Are the phrases "ability to read, speak, and/or understand English", "English language proficiency", "required English", "required English language", and "mandate English" interchangeable? Similarly, "who cannot read, speak, and/or understand English" and "are non-Native English speakers" can mean widely different things. Consistent terms should be used in the abstract. A definition of Limited English Proficiency (LEP) is offered in the introduction

Perhaps the larger issue is that protocols do not have a consistent language to classify proficiency and that is also something that needs to change (this point is made on page 8, but the abstract uses the terms interchangeably).

Page 2, Line 37-38 belongs in the conclusion (not introduction)

Page 2, Line 41-42 - what does "varying time points" mean?

Page 6, Line 111: The term Latinx has gained popularity in academic circles, but it is less known, widespread and accepted in non-academic communities in the United States that might be expected to identify as "Latinx" (very few people actually do). It is especially important to select terms carefully or perhaps acknowledge the complexities of using rarified academic language in an article concerned with increasing representation.

Page 11, Lines 201-208: The last paragraph outlines some general suggestions, but it would be a stronger conclusion if the authors specifically provided a set of concrete recommendations to increase language (and other kinds) of representativeness. A good place to start to look into such guidelines might be literature on community-based research and patient-centered clinical research (PCORI, a federally-funded organization does just this).

Trials that do not require English may already be exclusive "by default" - the next step would be looking at actual recruitment for those trials vs. the ones that are English-exclusive. I would imagine that the differences in the proportion of LEP subjects would be negligible and that the only significant differences would be in those trials that actively recruit non-English speakers.

It is also important to acknowledge that English proficiency requirements are only part of the story, it is equally important for clinical research to actively seek recruitment from populations that are underrepresented due to other socio-economic factors. The COVID-19 crisis underscores the importance of inclusivity in clinical research: in the US, rates of vaccination are much lower in black and Hispanic populations than in whites. In order to understand why this is the case and more urgently, begin to close those gaps, inclusive research is essential.

Finally, it may be somewhat out of the scope of this paper, but the lack of translation resources for LEP researchers in other settings (particularly low and middle-income countries) also means that research not conducted in English has few opportunities to be funded or published by major organizations. It may be worthwhile to mention that journals and funding agencies could do more to facilitate research with a global impact, which is particularly important given the global epidemic.

It is perhaps somewhat understandable that researchers simply do not have the time, training or capacity to conduct outreach efforts that include medically-underserved populations. However, without those efforts, clinical research will increase existing inequalities. Until clinical research makes a real effort to include underrepresented populations, we cannot expect medicine and healthcare to have equitable impact.

Any attachments provided with reviews can be seen via the following link:

[LINK]

Decision Letter 2

Beryne Odeny

15 Jul 2021

Dear Dr. Bierer,

Thank you very much for re-submitting your manuscript "Evaluating the frequency of English language requirements in clinical trial eligibility criteria: a systematic analysis using ClinicalTrials.gov" (PMEDICINE-D-21-01563R2) for review by PLOS Medicine.

I have discussed the paper with my colleagues and the academic editor and it was also seen again by three reviewers. I am pleased to say that provided the remaining editorial and production issues are dealt with we are planning to accept the paper for publication in the journal.

The remaining issues that need to be addressed are listed at the end of this email. Any accompanying reviewer attachments can be seen via the link below. Please take these into account before resubmitting your manuscript:

[LINK]

***Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.***

In revising the manuscript for further consideration here, please ensure you address the specific points made by each reviewer and the editors. In your rebuttal letter you should indicate your response to the reviewers' and editors' comments and the changes you have made in the manuscript. Please submit a clean version of the paper as the main article file. A version with changes marked must also be uploaded as a marked up manuscript file.

Please also check the guidelines for revised papers at http://journals.plos.org/plosmedicine/s/revising-your-manuscript for any that apply to your paper. If you haven't already, we ask that you provide a short, non-technical Author Summary of your research to make findings accessible to a wide audience that includes both scientists and non-scientists. The Author Summary should immediately follow the Abstract in your revised manuscript. This text is subject to editorial change and should be distinct from the scientific abstract.

We expect to receive your revised manuscript within 1 week. Please email us (plosmedicine@plos.org) if you have any questions or concerns.

We ask every co-author listed on the manuscript to fill in a contributing author statement. If any of the co-authors have not filled in the statement, we will remind them to do so when the paper is revised. If all statements are not completed in a timely fashion this could hold up the re-review process. Should there be a problem getting one of your co-authors to fill in a statement we will be in contact. YOU MUST NOT ADD OR REMOVE AUTHORS UNLESS YOU HAVE ALERTED THE EDITOR HANDLING THE MANUSCRIPT TO THE CHANGE AND THEY SPECIFICALLY HAVE AGREED TO IT.

Please ensure that the paper adheres to the PLOS Data Availability Policy (see http://journals.plos.org/plosmedicine/s/data-availability), which requires that all data underlying the study's findings be provided in a repository or as Supporting Information. For data residing with a third party, authors are required to provide instructions with contact information for obtaining the data. PLOS journals do not allow statements supported by "data not shown" or "unpublished results." For such statements, authors must provide supporting data or cite public sources that include it.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript.

Please note, when your manuscript is accepted, an uncorrected proof of your manuscript will be published online ahead of the final version, unless you've already opted out via the online submission form. If, for any reason, you do not want an earlier version of your manuscript published online or are unsure if you have already indicated as such, please let the journal staff know immediately at plosmedicine@plos.org.

If you have any questions in the meantime, please contact me or the journal staff on plosmedicine@plos.org.  

We look forward to receiving the revised manuscript by Jul 16 2021 11:59PM.   

Sincerely,

Beryne Odeny,

Associate Editor 

PLOS Medicine

plosmedicine.org

------------------------------------------------------------

Requests from Editors:

Thank you for revisions. Please address the following points before we proceed:

1) Please include p-values for all estimates provided in the results section - text and tables. For instance, tables 2 & 3 have 95% CIs but no p-values

2) References – please consistently include access dates for weblinks e.g., refs #1, #6, #9

Comments from Reviewers:

Reviewer #1: The authors have addressed my concerns and I now recommend publication.

Peter Flom

Reviewer #2: The revised paper provides thoughtful responses to the reviewer critiques, and has made edits and additions that strengthen the manuscript. My one additional comment on this read is that it would be helpful to know more about the characteristics of the 390 trials that specifically mentioned accommodation of translation to another language. If the reasons for affirmative accommodation of translation are related to geography, specific conditions likely to occur in non-English-speaking populations, or other identifiable correlates, this would be an important additional finding that would help readers to better understand how language requirements in clinical trials are being handled.

Reviewer #3: The authors did a good job of taking into account reviewers' comments and editing the manuscript accordingly. They likewise clearly explained their reasons when deciding not to change the manuscript.

I recommend this article for publication.

Any attachments provided with reviews can be seen via the following link:

[LINK]

Decision Letter 3

Beryne Odeny

5 Aug 2021

Dear Dr Bierer, 

On behalf of my colleagues and the Academic Editor, Dr. Margaret E Kruk, I am pleased to inform you that we have agreed to publish your manuscript "Evaluating the frequency of English language requirements in clinical trial eligibility criteria: a systematic analysis using ClinicalTrials.gov" (PMEDICINE-D-21-01563R3) in PLOS Medicine.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. Please be aware that it may take several days for you to receive this email; during this time no action is required by you. Once you have received these formatting requests, please note that your manuscript will not be scheduled for publication until you have made the required changes.

In the meantime, please log into Editorial Manager at http://www.editorialmanager.com/pmedicine/, click the "Update My Information" link at the top of the page, and update your user information to ensure an efficient production process. 

PRESS

We frequently collaborate with press offices. If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximise its impact. If the press office is planning to promote your findings, we would be grateful if they could coordinate with medicinepress@plos.org. If you have not yet opted out of the early version process, we ask that you notify us immediately of any press plans so that we may do so on your behalf.

We also ask that you take this opportunity to read our Embargo Policy regarding the discussion, promotion and media coverage of work that is yet to be published by PLOS. As your manuscript is not yet published, it is bound by the conditions of our Embargo Policy. Please be aware that this policy is in place both to ensure that any press coverage of your article is fully substantiated and to provide a direct link between such coverage and the published work. For full details of our Embargo Policy, please visit http://www.plos.org/about/media-inquiries/embargo-policy/.

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Thank you again for submitting to PLOS Medicine. We look forward to publishing your paper. 

Sincerely, 

Beryne Odeny 

Associate Editor 

PLOS Medicine

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Tables

    Table A: ClinicalTrials.gov advanced search criteria. Table B: ClinicalTrials.gov funder type classification.

    (DOCX)

    Attachment

    Submitted filename: 2021-06-15b PLOS Medicine response letter.docx

    Attachment

    Submitted filename: 2021-07-23 PLOSMed response letter.docx

    Data Availability Statement

    All relevant data are within or cited in the manuscript or available from ClinicalTrials.gov.


    Articles from PLoS Medicine are provided here courtesy of PLOS

    RESOURCES