Inclusion of Non-English speaking participants in pediatric health research: A review of 3 pediatric journals

Abstract

Importance:

The inclusion of non-English speaking (NES) participants in pediatric research is an essential step to improving health equity for these populations. While a few studies have shown lack of progress in NES research participation in the last decade, to our knowledge little work has examined NES inclusivity in pediatric research or details around the practices researchers have used to communicate with NES participants.

Objective:

We conducted a review of 3 high-impact pediatric journals to determine: 1) how frequently NES families were included in pediatric research; 2) how rates of inclusion changed over time; 3) what languages were included; and 4) methodological details around oral and written communication with NES participants.

Evidence Review:

All original investigations published in JAMA Pediatrics, Pediatrics, or The Journal of Pediatrics between 2012 and 2021 were screened. Eligible articles, which included those based in the United States and including human participants, were further reviewed to determine if they included or excluded NES participants, or if there was no specific mention of language. A second-round review was conducted on the subset of articles which included NES participants to determine methodological details (languages included, type of study, region where study was located, and oral and written communication practices with NES participants).

Findings:

Of the 8142 articles screened, 5008 (62%) met inclusion criteria; of these 469 (9%) included NES participants. The most common language was Spanish (350; 75%); 145 (31%) articles reported a “non-English” or “other” language without specification. Two hundred thirty (49%) articles reported the number of NES participants in their results section and 61 (13%) specified how they determined if participants preferred a language other than English. Less than 1/3 of articles specified how oral (101; 22%) and written (136; 29%) communication occurred with NES participants.

Conclusions and Relevance:

This review of 3 pediatric journals provides preliminary evidence suggesting exclusion of NES communities from pediatric research over the last decade and highlights an opportunity to provide more methodological detail around communication with NES participants. Best practices for improving inclusivity of NES participants are needed to guide researchers toward improved methods and more relevant results.

Non-English speaking (NES; also called limited English proficient) communities, defined as those who speak and understand English less than very well, have been rapidly growing and now account for 9% of the US population.¹ Language barriers affect healthcare access and quality, patient-physician communication, and patient safety.^2–6 Nine percent of NES individuals in the United States are ages 5 to 17 ⁷ and nearly 6 in 10 children of immigrants (≥1 parent born abroad) have a parent who identifies as NES.⁸ NES children and parents face healthcare disparities and experience poorer health overall.^9–11 Children with NES parents experience higher communication-related adverse events,^12,13 medical errors,¹⁴ and mortality.¹⁵ NES parents are also less likely to have a pediatric medical home and are more likely to report that linguistic and cultural barriers negatively affect the patient-provider releationship.¹⁶ These disparities have led to multiple calls to action to provide linguistically and culturally affirming healthcare to NES families.^17,18

An important step in improving equity for NES communities is their inclusion in pediatric research.^19–21 Inclusion of racial and ethnic minoritized individuals in research is critical to ensure scientific findings are generalizable and relevant for all communities; to improve trustworthiness of research and healthcare; and to promote health equity.^22–25 Similarly, including NES participants is necessary to address health disparities related to language, leading to calls to action by the National Academy of Medicine and National Institutes of Health to increase their representation.^26,27 Notably, in 2011, the US Department of Health and Human Services (DHHS) shared their strategic priority to be more inclusive of language in research.²⁸

Over the past decade, limited work has examined how inclusion of NES participants has changed in the wake of such calls to action. One study found that while the number of NES patients cared for at their pediatric academic medical center increased between 2011 and 2016, NES participation in research did not increase at the same rate.²⁹ Similarly, a study examining 236 emergency medicine articles from 2 separate years (2004 and 2014) found that 11% of studies excluded NES participants.³⁰ However, this work did not focus on pediatric research, only examined a small subset of articles, and did not examine what languages were included or how study teams communicated with participants. There is complexity in both oral and written communication that may lead to participants receiving inaccurate information.³¹ As an example, a study reviewing professionally translated materials found an array of errors, particularly in conceptual equivalence and omissions.³² Understanding how NES participants have been included in research is necessary to synthesize best practices and ensure that participants are able to equitably engage in the research process.

In this review, we examined how often NES participants are engaged in pediatric research by reviewing 3 high-impact pediatric journals. Our objectives were to determine: 1) how frequently NES families were included in pediatric research, 2) how rates of inclusion changed over time, 3) what languages were included, and 4) methodological details around oral and written communication with NES participants.

METHODS

Study Design

We conducted a review of original investigation articles published in 3 pediatric research journals: JAMA Pediatrics, Pediatrics, and The Journal of Pediatrics; this targeted approach is consistent with previous research.^33,34 These journals were selected due to their high impact factors, diverse readership, robust peer-review process, and broad thematic scope. Our study comprised 2 rounds of screening. In the first round, we reviewed each article to determine eligibility and whether the article included NES participants. In Round 2, we reviewed the subset of articles where NES participants were included to ascertain more details about how they were included. This study was deemed non-human subjects research by the University of Pittsburgh.

Inclusion criteria

Studies were eligible for inclusion if they were original research based in the United States published in JAMA Pediatrics, Pediatrics, or The Journal of Pediatrics from January 2012 to November 2021. We focused on US-based studies because language needs and access may be different in other countries. Multinational studies where participants were recruited from the United States were included. We limited the timeframe to the last decade to assess for change after the 2011 US DHHS call to action to improve language inclusion in research.²⁸ Studies without human participants (e.g., basic science, review of policies) were excluded, as were supplemental issues, brief reports, systematic reviews, meta-analyses, and commentaries.

Screening process

Round One

All original research articles from the 3 journals published from January 2012 to November 2021 were downloaded by the research team. We created an abstraction form in an electronic survey tool (Qualtrics XM, Provo, Utah), which included: 1) journal and article title; 2) publication date; 3) study location; 4) inclusion of human participants; 5) prospective vs retrospective recruitment (e.g., chart review, claims data); and 6) inclusion or exclusion of NES participants (or no mention of participant language). Questions 5 and 6 were completed only for articles that met inclusion criteria. To determine NES participants’ inclusion/exclusion, we searched the article for the terms: “language”, “English”, “translate”, “interpreter”, “proficient”, and “fluent” and then reviewed the text associated with those terms. As a quality control measure, for the first 20 screenings we read the entire article to ensure that we were not missing the authors’ description of NES participation using the chosen search terms; we found no article in which those terms did not identify text related to NES inclusion.

Six members of our research team screened articles. Prior to beginning this process, the screeners went through a detailed training, where each screening question was reviewed and discussed. Each screener could mark “I don’t know” for any screening question, which would automatically lead to cross-checking by AC or MR; we encouraged screeners to use this option liberally, with any hint of uncertainty. During cross-checking, all responses for flagged articles were reviewed with discrepancies noted, discussed, and adjudicated with the initial reviewer. Any discrepancy which could not be resolved was discussed with a third team member (JC). Additionally, for each screener’s first 50 articles, 5 articles were randomly selected for cross-checking. This process continued until there were no discrepancies. Figure 1 provides a flow diagram of the number of articles screened, excluded, and cross-checked.

Figure 1:

Flow diagram of study selection for the identification of articles that included non-English speaking participants published from 2012 to 2021 in 3 pediatric journals

Round Two

All articles that included NES participants underwent a second screening by 3 team members with MR available to cross-check, where the entire manuscript was reviewed. We developed an abstraction codebook survey on Qualtrics that focused on the following items: 1) type of study; 2) study location; 3) non-English languages included; 4) if and how description about oral and written communication with NES participants was included; and 5) whether the number of participants identifying as NES was included. Each question had an “I don’t know” option; selection led to an automatic cross-check of the entire abstraction, using a similar process as described for Round 1.

For study location, we classified the states from which participants were recruited into 5 US-regions: 1) West; 2) Southwest; 3) Midwest; 4) Southeast; 5) Northeast.³⁵ Studies could take place in up to all 5 regions and were classified as using a nationally representative sample only if explicitly stated in the article. The number of non-English languages included was recorded as specified in the article. Articles with participants’ language reported as “other” or “non-English” without further specification were categorized as “non-English language unspecified.”

We defined oral communication details as any reporting of how communication occurred. Studies that described these details were categorized as communicating through one or more of the following: 1) a research team member; 2) an interpreter; 3) recorded materials in a non-English language; 4) a family member; and 5) other method. We also recorded whether articles described how they assessed bilingual capabilities of research team members and how studies determined the language preference of participants. We defined written communication details in two ways: 1) what types of materials were translated; and 2) how materials were translated. The types of materials translated were categorized as: 1) surveys created by the research team; 2) existing data collection instruments; 3) consent forms/study information sheets; 4) pre-existing surveys; 5) intervention materials; 6) recruitment materials; 7) “all materials” (when authors described that all research materials were translated without further specification); and 8) other research material. We categorized method of written translation as: 1) already available in the non-English language; 2) by institutional or third-party translation services; and 3) other method.

Data analysis

In Round 1, data from the 6 screeners were merged and duplicate studies removed. We used descriptive statistics to examine frequencies of articles which included, excluded, or had no explicit mention of NES participants, across all journals. We conducted Chi-square tests to identify significant differences in the number of articles including NES participants between studies using prospective versus retrospective recruitment. We performed Cochran-Armitage trend tests to determine whether observed trends in inclusion of NES participants over time were statistically significant. We assumed a 5% significance level with no adjustments for multiplicity. Analyses were performed using SAS 9.4 (SAS Institute Inc., Cary, NC). For Round 2, we merged data from the 3 screeners and removed duplicate studies. Descriptive statistics were used to examine frequencies for each question.

RESULTS

Inclusion of NES participants

We screened 8142 original investigations in Round 1; 5008 (62%) were included in the review (Figure 1). Of the articles meeting inclusion criteria, 469 (9%) included NES participants, 412 (8%) excluded NES participants, and 4127 (82%) had no explicit mention of language (Table 1). More studies with prospective than with retrospective recruitment included NES participants (288; 13% vs 181; 7%; Chi-square=52.9; P<.001). We found a significant increase in inclusion of NES participants from 2012 to 2021 (z=3.3; P<.001; Figure 2).

Table 1.

Number of articles that included non-English speaking participants, excluded non-English speaking participants, or made no explicit mention of language among articles included in the review

	Included non-English speaking participants (n, %)	Excluded non-English speaking participants (n, %)	No explicit mention of language (n, %)
Total articles (N=5008)	469 (9)	412 (8)	4127 (82)
Articles by year
2021 (n=469)	59 (13)	41 (9)	369 (79)
2020 (n=441)	51 (12)	39 (9)	351 (80)
2019 (n=468)	56 (12)	49 (10)	363 (78)
2018 (n=531)	54 (10)	45 (8)	432 (81)
2017 (n=481)	32 (7)	37 (8)	412 (86)
2016 (n=517)	41 (8)	46 (9)	430 (83)
2015 (n=498)	36 (7)	33 (7)	429 (86)
2014 (n=524)	53 (10)	40 (8)	431 (82)
2013 (n=591)	47 (8)	43 (7)	501 (85)
2012 (n=488)	40 (8)	39 (8)	409 (84)
Articles with prospective recruitment
Yes (n=2279)	288 (13)	366 (16)	1625 (71)
No (n=2729)	181 (7)	46 (2)	2502 (92)

Figure 2:

Inclusion of non-English speaking participants in original research in 3 pediatric journals from 2012 to 2021.

Characteristics of articles including NES participants

A total of 469 articles were screened in Round 2. The most common study designs were cross-sectional (132; 28%), prospective cohort (102; 22%), and randomized clinical trials (87; 19%; Table 2). The majority occurred in the West (171; 36%) or Northeast (123; 26%) regions of the United States or used nationally representative data (92; 20%). The most common non-English language included was Spanish (350; 75%); 145 (31%) articles reported one of the languages included as “non-English” or “other” without further specification. Two hundred thirty (49%) studies reported the number of NES participants.

Table 2.

Study design, location, and languages of articles which included non-English speaking participants (N=469)

	Articles (n, %)
Study design
Cross-sectional study	132 (28)
Prospective cohort study	102 (22)
Randomized control trial (RCT)	87 (19)
Retrospective cohort study	54 (12)
Case-control study	12 (3)
Focus group/key informant interview	5 (1)
Othera	77 (16)
United States study locationb
West	171 (36)
Northeast	123 (26)
Southeast	66 (14)
Midwest	59 (13)
Southwest	41 (9)
Nationally representative sample	92 (20)
Not specified	89 (19)
Multi-national
Yes	12 (3)
No	457 (97)
Non-English languages included in studiesc
Spanish	350 (75)
Chinese (Mandarin, Cantonese)	12 (3)
Somali	10 (2)
Vietnamese	7 (2)
Korean	5 (1)
Arabic	5 (1)
Hmong	3 (0.6)
Russian	3 (0.6)
American sign language	3 (0.6)
Haitian-Creole	3 (0.6)
Italian	2 (0.4)
Portuguese	2 (0.4)
Amharic	2 (0.4)
Non-English language unspecified	145 (31)
Otherd	3 (0.6)
Language reported as a demographic in results section
Yes	230 (49)
No	239 (51)
Number of non-English languages includede
1	299 (64)
2	14 (3)
≥3	15 (3)
Non-English language unspecified	145 (31)

^aOther study designs included those that did not fall into one of our existing categories

^bPercentages sum to more than 100% because studies may have recruited participants from multiple regions

^cPercentages sum to more than 100% because more than one non-English language may have been included

^dThe “Other” category consists of when only 1 article included participants who spoke a non-English language. Languages in these articles were: French, German, Hebrew, Japanese, Oromo, Karen, Nepali.

^ePercentages sum to more than 100% because we categorized languages classified as “Other” in an article as “non-English language unspecified” for this review. For example, if non-English languages were “Other” and “Spanish” we counted this article in 2 categories: “Spanish” and “non-English language unspecified”

Oral and written communication with NES participants

Most articles (408; 87%) did not specify how they determined if participants preferred a language other than English. Oral communication with NES participants was specified in 101 (22%) articles, most often occurring through a research team member who spoke the same language (42; 42%) or an interpreter (37; 37%). For written communication, types of materials translated were described in 136 (29%) articles and how translations occurred in 59 (13%). No study reported using a family member as a way for communicating with participants. No study described how language capabilities of bilingual research team members was assessed. Types of translated materials included surveys created by the research team (43; 32%), existing data collection instruments (38; 28%), and consent forms or study information sheets (19; 14%). The majority of studies (30; 51%) that described how their materials were translated noted the information was already available in a non-English language.

DISCUSSION

This review is one of the first to describe the inclusion of NES participants in pediatric research over the last 10 years. Our results suggest that NES participants are not well-represented in pediatric health research with only 9% of articles including NES participants, although we did find an increase over time. The percentage of NES individuals in the United States has risen rapidly, now comprising around 9% of the country’s population.¹ Our results suggest a lack of inclusivity of NES participants in pediatric research, making the relevance of study findings uncertain for this growing segment of US society. Exclusion of NES participants from research may further perpetuate health disparities, as research inclusivity is associated with greater trust of healthcare and dismantling of health disparities more broadly.^22–25

Not surprisingly, we found that Spanish was the most common non-English language included in pediatric research. The intentional inclusion of Spanish speaking communities in research is critical as it is the most commonly spoken non-English language in the United States.⁷ However, the percentage of NES individuals in the US who speak a language other than Spanish is growing^1,7, highlighting the importance of equitable inclusion of multiple non-English languages in pediatric research.

We found a lack of methodological detail in articles describing studies that included NES participants. Communication with NES participants, including for recruitment, consent, and procedural instruction, etc. is a critical part of any research protocol. The methods used to assure excellent communication with NES participants at all stages of their research involvement was rarely addressed, limiting awareness and sharing of best practices. Oral communication is important to report as past work has demonstrated how lack of bilingual staff members can hinder research participation for NES participants.^36–38 Similarly, not using best practices for translation of written materials (such as back-translation and proofreading) can lead to problems with conceptual equivalence and the omission of important details.³² The heterogeneity in reporting of communication methodologies is aligned with past work showing wide variety in Institutional Review Board policies regarding consenting NES participants, including instructions for translations and qualifications for trained research staff.^39,40 Standardized guidelines for reporting methods around NES participation in research is needed.

We found that no study reported how research team members’ bilingual capabilities were assessed. The movement to professionalize language use in healthcare has led to qualification procedures for bilingual staff; ⁴¹ however, the use of such procedures to assure bilingual research staff have the capability to provide direct support to NES participants, interpret for team members, or translate written materials is not yet routine. Bilingual research staff may feel comfortable speaking a language for general purposes, but not necessarily to conduct complex consenting or describe research procedures.⁴² Standardized ways to determine professional language proficiency and engage trained interpreters in the research setting are needed. One reassuring finding was that no study reported using family members as interpreters. Language brokering, or the use of family members as interpreters or translators, is common during interactions with the healthcare system but is not acceptable due to potential for errors, negative impacts on family members, and breaches of confidentiality.^43–45

This review is subject to several limitations. We focused on US-based studies, so these results may not be generalizable to other countries. Our data present only the representation of NES participants in pediatric journals and should not be extrapolated to other specialties. We focused on three high-impact pediatric journals with broad readership and scope but recognize that we are missing articles published in other pediatric journals, particularly those focused on pediatric subspecialities. We elected not to consult supplemental materials or appendices, where further detail may have been provided. Additionally, we did not ask team members their reason for flagging an article to be cross-checked (in both Rounds 1 and 2), so cannot report how frequently cross-checking was requested for each item. Finally, for most studies, we were unable to determine if the NES participants included were parents or children, which may be important as children are less likely to identify as NES than adults.⁷ This study also focused specifically on language; however, there may be other cultural or structural factors impacting the inclusion of NES communities (80% of whom are first-generation immigrants).⁷

Implications

Intentional and multilevel changes are needed to improve inclusion of NES participants in pediatric health research, including examining institutional- and structural-level barriers and facilitators to increase representation of NES communities. More work should focus on NES communities’ experiences participating in research and ways to improve research trustworthiness. Ideally, NES participation would occur naturally through research-focused partnerships with NES families and communities, rather than treating such groups as an afterthought in protocols originally designed for English-speaking participants.

Academic institutions, funding agencies, and journals should create policies and practices that facilitate the inclusion of NES participants. Institutions should collaborate with companies to provide free or low-cost, accurate translation and interpretation services for researchers. Development of assessments to test bilingual capacity among research staff is also critical, as is workforce equity to increase the number of bilingual researchers. Funding agencies should have researchers explicitly state if they are including NES participants, and if not, scientifically justify their exclusion (similar to inclusion of women, minorities, and individuals across the lifespan requests).⁴⁶ Providing additional incentive funds for researchers conducting studies in languages other than English may help offset some of the costs. Journals must develop guidelines for authors around the level of detail needed for studies including NES participants, create expert-developed checklists for authors, and consider allowing authors additional words to provide methodological details around communication with NES participants. Fundamentally, there must be an investment in policies, processes, and resources for investigators and organizations to equitably support inclusion of NES participants in research efforts.

Conclusions

This review of 3 pediatric journals provides preliminary evidence that NES communities may be excluded in published pediatric research and highlights the importance of specifying communication methods and resources to ensure best practices are being met. This work underscores the urgency for researchers, research institutions, funders, policymakers, and journals to prioritize language inclusivity, so NES participants’ voices and experiences can be amplified through science. Our findings highlight a critical opportunity and responsibility for the pediatric health and science communities to lead efforts to meaningfully increase NES representation in research. In achieving this call to action to promote language equity, the vision of dismantling healthcare disparities experienced by NES communities may become a more immediate reality.

Table 3.

Methods reported for oral and written communication with non-English speaking participants

Article described how participant language preference determined (n=469)	Articles (n, %)
Yes	61 (13)
No	408 (87)
Article described how oral communication occurred (n=469)
Yes	101 (22)
No	368 (78)
Method of oral communication used (n=101)a
Through a research team member who speaks the same language	42 (42)
Through an interpreter	37 (37)
Recorded materials in non-English language	6 (6)
Family member	0 (0)
Other	23 (23)
Article described which materials were translated (n=469)
Yes	136 (29)
No	333 (71)
Materials translated (n=136)b
Surveys created by research team	43 (32)
Existing data collection instruments	38 (28)
Consent forms/study information sheets	19 (14)
“All materials” terminology used	14 (10)
Pre-existing surveys available to research team	13 (10)
Intervention materials	12 (9)
Recruitment materials	4 (3)
Other	21 (15)
Article described how translation of written materials occurred (n=469)
Yes	59 (13)
No	410 (87)
Method of translation (n=59)
Already available in non-English language	30 (51)
By institutional or third-party translational services	4 (7)
Other	25 (42)

^aPercentages sum to more than 100% because studies may have included more than one method of oral communication

^bPercentages sum to more than 100% because studies may have included more than one method of written communication

KEY POINTS.

Question:

How frequently are non-English speaking (NES) participants included in pediatric research and what methodological details are provided around communicating with NES participants?

Findings:

Nine percent of research articles in 3 pediatric journals included NES participants over a 10-year period. Spanish was the most commonly included language; less than a third of articles provided specific details around oral and written communication with NES participants.

Meaning:

A review of 3 pediatric journals found preliminary evidence that NES communities are underrepresented in research. Future work is needed to increase NES representation, ensure best practices in oral and written communication, and promote language and health equity.

ACCESS TO DATA AND DATA ANALYSIS

Dr. Ragavan had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.