Abstract
In this community-partnered study we conducted focus groups with non-English speaking immigrant and refugee communities of color in 4 languages to understand their perspectives on COVID-19 vaccines, barriers to accessing vaccines, and recommendations for healthcare providers. We used a mixed deductive-inductive thematic analysis approach and human centered design to guide data analysis. 66 individuals participated; 85% were vaccinated. The vaccination experience was often positive; however, participants described language inaccessibility, often relying on family members for interpretation. Community-based organizations played a role in connecting participants to vaccines. Unvaccinated participants expressed fear of side effects and belief in natural immunity. Participants shared recommendations to providers around increasing vaccine access, improving language accessibility, and building trust. Results from our study show numerous barriers immigrant and refugee communities of color faced getting their COVID-19 vaccine, but also highlights opportunities to engage with community partners. Future implications for research, policy, and practice are described.
Keywords: immigrant and refugee, language justice, structural inequities, community-partnered research, qualitative methods, COVID-19 vaccines
The COVID-19 pandemic disparately impacts marginalized communities, including people identifying as racial and ethnic minorities and those living in poverty, laying bare pre-existing inequities rooted in racism, xenophobia, and classism, among others.1-3 Non-English speaking immigrant and refugees of color (IRC) were particularly vulnerable to the health and social ramifications of the pandemic.4 IRC communities experienced higher risks of COVID-19 exposure, illness, and mortality due to employment status, lack of transportation, immigration status, and inequitable healthcare access.5,6 Therefore, developing and implementing strategies to mitigate the disproportionate impact of the COVID-19 pandemic on non-English speaking IRC communities is critical to promote health equity.
COVID-19 vaccines, which were distributed to the public starting in December 2020, are a safe and effective way to decrease COVID-19 morbidity and mortality. However, IRC communities have faced disparate access to COVID-19 vaccines as well as hesitancy to receive a vaccine among some individuals.7-9 While a substantial amount of work has examined inequities in vaccine access and trust among racial and ethnic minoritized communities, less has considered immigration or language.10-12 Studies at the beginning of the COVID-19 vaccine rollout examining perspectives of COVID-19 vaccines among non-English speaking IRC communities showed relatively high interest in getting vaccinated.13,14 However, IRC communities described several concerns about getting vaccinated, including being questioned for identification, the perception that getting vaccinated would fall under the definition of “public charge” (which certified that use of public services may be sufficient foundation for denying visa or green card applications), lack of linguistically affirming services (i.e. services in the language they prefer for medical care), and concern that getting vaccinated would lead to decreased wages or punitive measures from employers.15,16 A recent review synthesized several strategies which have been used to address these concerns and promote vaccine equity, including use of community-based organizations as trusted messengers, availability of linguistically affirming COVID-19 vaccine information, use of multilingual news-sources, among others.15
While there has been a considerable amount of research on IRC communities’ perspectives on getting vaccinated, less research has examined their actual vaccination experience.14-17 Non-English speaking IRC communities face multiple challenges accessing high-quality healthcare through the medical system, all of which are rooted in intersecting racism, xenophobia, language inaccessibility, and insurance inaccessibility.18-20 Less is known about how these barriers manifested during the unprecedented mass-vaccine distribution efforts which occurred in the face of the COVID-19 pandemic. Examining the vaccination experiences of non-English speaking IRC communities is critical to determining what challenges individuals faced while getting vaccinated and what strategies were actually used to make vaccines more accessible and trusting to these communities.
Key to the vaccination effort more broadly has been community-based organizations, who have served as trusted messengers and advocates throughout the pandemic. Community-partnered research is a unique approach in which researchers and community members engage in the scientific process together.21-24 This method dismantles traditional power structures, assumptions around scholarly and experiential knowledge, and emphasizes research reciprocity and the amplification of community priorities.25,26 Our team has worked closely together throughout the COVID-19 pandemic within the Community Vaccine Collaborative (CVC), which is a community-academic partnership focused on promoting vaccine equity.27 We are therefore well-positioned to examine the vaccination experiences of non-English speaking IRC communities.
The goal of this study was to leverage the expertise of a community-academic collaboration to explore the COVID-19 vaccination experiences of non-English speaking IRC communities. Additionally, as a secondary goal, we also wanted to hear the perspectives of non-English speaking IRC communities who had not been vaccinated as of August 2021, as many of the prior studies were done within the early rollout and may not capture their perspectives.
Methods
Overview
We conducted focus groups with non-English speaking IRC communities in Pittsburgh to understand their COVID-19 vaccination experiences. On advice from community partners, we also offered participants the opportunity to do individual interviews if they preferred. Throughout our study, we incorporated principles of human centered design (HCD) which is a strategy to translate community perspectives into workable solutions and can complement community-partnered research.28 The University of Pittsburgh Institutional Review Board deemed our study exempt.
Setting and context
This study took place in Pittsburgh, Pennsylvania (located within Allegheny County) and the surrounding area. The community partners work in Allegheny County where first-generation (born outside of the United States) immigrants account for about 6.1% of the population (compared with 13.7% nationally).29 Between 2001 and 2016, approximately 4000 refugees were resettled in the Pittsburgh area and over 3000 more relocated to Pittsburgh from other states and cities, resulting in large communities of Bhutanese and Congolese individuals in the region, among others.30-32 Additionally, Allegheny County experienced a rise in immigrants from Mexico, Central and South America, resulting in an increase in the Hispanic or Latino (hereafter referred to as “Latine”) population of 80% in the county between 2010 and 2020.30,33 The Chinese community in Pittsburgh has historical ties to the region since the 1870s, and is one of the largest immigrant groups in the region.32,34-36 Over 90,000 people in Allegheny County speak a language other than English at home, with about one third of those individuals speaking English “less than very well.”37
One community partner works in nearby Westmoreland County, where about 1.7% of the total population was born outside of the United States, although the immigrant community is rising faster than the general population.38 As of 2020, it’s estimated that over 8500 people in Westmoreland County speak a language other than English at home, with about 22.7% of those individuals speaking English “less than very well.”
Community partnership development and processes
We built our team leveraging existing partnerships through the Community Vaccine Collaborative (CVC). The CVC convened in July of 2020 and meets weekly to discuss vaccine access and trustworthiness, as well as provide reliable information to share with the communities. Background information on the formation and activities of the CVC are published elsewhere.27 Targeted subgroups within the CVC were formed to focus on topics warranting specific attention, including a group focused on COVID-19 vaccine equity for IRC communities. This subgroup co-wrote a proposal for a funding opportunity focused on vaccine disparities. After receiving funding in July 2021, the subgroup has met monthly. A core planning group consisting of a program coordinator at a community-based organization, researchers in medicine and public health, and a professor of immersive media met bi-weekly to debrief, draft agendas, and plan data collection and coding. Large group meetings began with an open discussion to address COVID-19 vaccine questions or concerns, followed by a HCD activity to support data collection and analysis (see Table 1 for an agenda of each month’s activities). Additionally, 7 bilingual community leaders recruited participants and conducted the focus groups and interviews. Community members were compensated for their time and were involved in all stages of this work, including conceptualization, participant recruitment, data collection, and data analysis.
Table 1.
June 2021 | Introduction to project, joint mission and goals |
July 2021 | Joint review and revision of focus group/interview guides in language-specific breakout rooms |
August 2021 | Practice focus groups in language-specific breakout rooms |
September 2021 | Reviewed preliminary codes from focus groups and interviews. Use an activity called “Rose, Thorn, Bud” to identify potential strengths, challenges, and opportunities emerging from the data |
October 2021 | Reviewed codes from vaccinated focus groups and interviews. Sorted codes into themes using community storymapping |
November 2021 | Reviewed codes from unvaccinated focus groups and interviews. Sorted codes into themes using community storymapping |
December 2021 | Reviewed summary of analysis from unvaccinated and vaccinated participants (except recommendations; October and November meetings) |
January 2022 | Reviewed a Miro board that outlines recommendations shared by participants. Discussed how to consolidate recommendations. Review of next steps |
February 2022 | Discussed manuscript and edits. Identified key priorities for the group to focus on |
Study team
Our study team included individuals from a variety of racial and ethnic, immigration, and linguistic backgrounds. The majority identified as people of color (Black, Latine, Asian, Indigenous, multiracial), immigrants or children of immigrants, and speak more than one language. Our team included community leaders, community-partnered researchers, healthcare providers, immersive media specialists, and representatives from different culturally-affirming organizations in Allegheny and Westmoreland Counties.
Conceptual framework
Our study was rooted in intersectionality theory, which describes how social categories (i.e. race, immigration status, language) intersect at the micro level to impact individual experiences (i.e. vaccine thoughts and experience), which reflect multiple interlocking systems of oppression and privilege at the macro level (i.e. racism, xenophobia).39-41 We also used a descriptive qualitative research approach, or research to “produce low-inference descriptions of a phenomenon,” which is an inductive approach that can be used for hypothesis generation.42,43 This framework is particularly helpful for topics with little or no past research.
Languages
All study procedures (e.g. recruitment, focus group facilitation) occurred in the languages most frequently used by participants: Spanish, Nepali, Mandarin, and French. These languages were chosen as they were the most commonly spoken by the individuals served by the community-based organizations in the community collaborative. Translations of all documents (recruitment materials, information scripts, audio recording transcriptions) occurred through a bilingual team member or a professional translation company and were proofread by another bilingual community team member.
Focus group and interview participants
Participants were included if they identified as an immigrant from one of five communities: Latine, Chinese, Taiwanese, Congolese, or Bhutanese. Participants also had to identify as non-English speaking, be age 18 or older, and live within Allegheny or Westmoreland counties. Note that the one community partner working in Westmoreland County worked with Latine communities. We defined non-English speaking as not speaking English as a primary language and having a limited ability to speak, understand, read, or write English. We included participants who both were and were not vaccinated for COVID-19.
Participant recruitment
Community leaders identified potential participants and contacted them to schedule a focus group or interview. They reached out via phone, text, or email to community members they have worked with. We used a purposive sampling technique to include immigrants with different gender identities and belonging to different age groups.43
Measures
We created two guides: one for participants who were vaccinated and another for those were not vaccinated. For vaccinated participants, the guide addressed: 1) perspectives on COVID-19 vaccine accessibility and trust; 2) the vaccination experience; and 3) recommendations for healthcare providers and systems on increasing vaccine equity and confidence. The unvaccinated guide focused primarily on participants’ reasons for not getting vaccinated. Both guides asked about ways that cultural and structural factors impacted participants’ perspectives about COVID-19 vaccines. The original guide was drafted by two authors (YS, MR) and then revised by the community partners.
Training for qualitative facilitators
The 7 bilingual facilitators attended three different trainings to prepare for data collection. First, they attended a 1-h research ethics training, which permitted them to be included on our Institutional Review Board protocol. Next, they attended a 2-h training with author MR, an experienced qualitative researcher, on how to facilitate focus groups and interviews. Finally, during one of our full team monthly meetings we practiced reviewing the consent script and conducting a focus group. The facilitators had the option of doing additional sessions through the data collection period.
Data collection
We conducted 75-min focus groups which occurred in a private in-person space or virtually through Zoom. Verbal consent was obtained prior to starting the focus group. Focus groups were conducted in Spanish, Nepali, Mandarin, or French and were audio-recorded. Audio-recordings were translated by a professional translation company and then proofread by the bilingual facilitator to ensure linguistic and conceptual equivalence. Participants received a $35 gift card. We continued conducting focus groups and interviews until data saturation was reached, meaning we found no new themes.44
Data coding and analysis
We used a mixed deductive-inductive thematic analysis approach.43-45 One of two coders individually coded each transcript using the DeDoose qualitative software package.46 We developed a preliminary codebook, which included a-priori codes developed by our research team and codes that inductively emerged from the data. We met to resolve discrepancies, with an additional team member conducting consensus coding. Data analysis occurred during our monthly community partner meetings, for which we used HCD activities including “Rose, Bud, Thorn'' (a way to understand codes across strengths, challenges, and opportunities), community story mapping (a way to sort codes and share narratives that emerged), and importance and difficulty matrix (outlining recommendations based on their difficulty and importance). Analysis occurred on Miro, which is a visual collaborative software platform on which we tagged codes based on language and vaccine status. An immersive media specialist designed the activities on Miro and guided community partners through the process.47Figure 1 shows an example of a Miro board (quotations on post-it notes, participants drag notes to different boxes in the middle and create themes).
Results
66 people participated in this study; 59 completed a focus group and 7 completed an interview. We completed 14 focus groups with between 3–5 people; 3 in Nepali, 3 in French, 5 in Spanish, and 3 in Mandarin, as well as 5 interviews in Mandarin and 2 interviews in Spanish. Vaccinated and unvaccinated focus groups/interviews took place separately. Twelve out of the 14 focus groups included participants who were vaccinated; 2/7 interview participants were vaccinated. The majority of participants identified as cisgender women (56%), ages 30–64 (56%), and were vaccinated (85%). Table 2 provides participant demographics.
Table 2.
Language of focus group or interview | n (%) |
Spanish | 21 (32) |
Nepali | 18 (27) |
French | 12 (18) |
Mandarin | 15 (23) |
Race and ethnicity1 | |
Asian or pacific Islander | 33 (50) |
Black, African, or African American | 12 (18) |
Hispanic, Latino/a/e/x | 21 (32) |
Indigenous, American Indian, or Alaskan | 0 |
Native | 5 (8) |
White or Caucasian | 0 |
Age | |
18–29 years old | 20 (30) |
30–44 years old | 22 (33) |
45–64 years old | 15 (23) |
65–80 years old | 9 (14) |
81 years or older | 0 |
Gender identity | |
Female | 37 (56) |
Male | 29 (44) |
Transgender or gender non-binary | 0 |
Birth country | |
Bhutan | 10 (15) |
China | 7 (11) |
Colombia | 6 (9) |
Democratic Republic of Congo | 11 (17) |
Guatemala | 1 (1) |
Honduras | 1 (1) |
Mexico | 9 (14) |
Nepal | 6 (9) |
Nicaragua | 1 (1) |
Taiwan | 6 (9) |
United States (Puerto Rico) | 3 (5) |
Prefer not to respond | 5 (8) |
Plan to receive COVID-19 vaccine | |
Yes, received 1st or 2nd dose | 56 (85) |
Yes, I have not received it yet but plan to | 3 (5) |
Maybe, I may. I am waiting to see | 4 (6) |
No, I am not going to get the vaccine | 2 (3) |
Prefer not to respond | 1 (1) |
1Percentages add up to more than 100 because participants could select more than one race and ethnicity
Six core themes emerged from the data: 1) COVID-19 vaccines were considered trustworthy by those who have been vaccinated, although some participants shared getting vaccinated only after it was required; 2) the vaccination experience was diverse, sometimes positive, but also with several challenges; 3) family members served as language brokers to support COVID-19 vaccinations; 4) community-based organization played a critical role to connect communities with vaccination opportunities; 5) unvaccinated individuals mistrusted the vaccines and were fearful of side effects; and 6) recommendations to better support IRC communities in improving vaccine equity.
Theme 1: COVID-19 vaccines were considered trustworthy by those who have been vaccinated, although some participants shared getting vaccinated only after it was required
In general, participants who were vaccinated expressed trust in the scientific process of vaccine development and the medical community which delivered the vaccine: “Personally, I believe in science. There are many people united in working to improve the situation so we can get out of it.” (focus group, 5, Spanish) Participants shared a belief in the importance of being vaccinated, especially as a strategy for ending the pandemic: “Before, out of fear, we didn’t go out or to the markets. After taking the vaccine, our confidence grew and we were able to go out. Now people are starting to get together for marriages, Upanayana ceremonies, and other festivals.” (focus group, 2, Nepali).
A few vaccinated participants, particularly from the French-language groups, shared a different perspective, noting they got vaccinated only after it was required for traveling: “I was hesitant as well to take the vaccine but the reason why I took the vaccine was because of the restrictions. It became too much. It is like we did not have a choice. They were forcing us to be vaccinated if we wanted to travel.” (focus group, 12, French). A participant shared a similar opinion noting they felt conflicted but eventually decided to get it: “I didn’t want us to get the vaccine, because there were some rumors going around about it being harmful. When they decided to vaccinate children at schools, I decided to get the vaccine. . . because it was also a requirement at work, and it would help us feel safer.” (focus group, 4, Spanish)
Theme 2: The vaccination experience was diverse, sometimes positive, but also with several challenges including immigration-related stressors
Participants shared they received their vaccines from a variety of locations including community-based clinics, pharmacies, employers, and mass-vaccine events. Many found the process easy to navigate: “It exceeded my expectations. I was thinking there were going to be more people, and even though there were a lot of people, it didn’t feel like it, they organized the appointments so well” (focus group, 5, Spanish). Another participant shared that their experience was positive, although they had heard of others who have faced challenges: “Others did say that they had to stand in line, were very tired, and faced other challenges. But for me, I had no issues” (focus group, 8, Nepali).
Others shared an array of barriers particularly around language: “I didn’t get anything in Spanish. . . So if someone doesn’t speak the language at all, they won’t have real access. Because all the questions, all our communication was only in English” (focus group, 3, Spanish). Another shared that while they were unable to engage bilingual services, it was less important to them: “When no one spoke Spanish in the first places I reached out to, I looked for a way to do it, and I have proof that I made my appointment, in English, by myself.” (focus group, 1, Spanish). Other barriers included long lines, lack of transportation, and cumbersome registration systems. A few participants also noted that they were asked about documentation: “Yes, they asked for my ID and I gave them one and you know I was nervous about that. But when I watched them, they just looked at the people in front of me and not copy it, so I thought it would be ok. I said I did not have health insurance card (like you told me to say) and they said ok” (focus group, 6, Mandarin). Another participant noted how concerns about documentation may keep others from getting vaccinated: “But keeping in mind how many undocumented immigrants, not just Latinos, there were many people who let themselves die of COVID-19 because of the fear of going to a hospital to get the vaccine, the fear of getting hospitalized, all for the fear that they might get deported back to their countries.” (focus group, 5, Spanish)
Some participants shared that they felt the process changed over time. One person described the different experience for them compared with their sibling: “She went to a chain pharmacy and her experience was different. They were nice to her, they were nice to us, too, but she wasn’t asked for her ID. And the information they gave her was both in English and Spanish. When my wife and I got vaccinated, when they had just started vaccinating the general public, they only spoke English and asked for those documents” (focus group, 3, Spanish). Another noted differences between getting their first and second vaccines: “I had been only here for two months so my English was not that solid. It was the first time for me to be in a clinic in the USA. The person I met did everything to make sure that I was understanding her. It was a wonderful experience but when I went back for my second dose, it was the other way around.” (focus group, 12, French)
Theme 3: Family members serve as language brokers to support COVID-19 vaccinations
Participants mentioned how family members played a critical role, often acting as language brokers: “I don’t really understand English. I can understand maybe 2 words out of 100. This is why we have our children with us who help us fill out forms and make appointments. My child played the role of interpreter there.” (focus group, 8, Nepali) Another participant noted how their family helped them download a translation service: “There was a questionnaire about my health and if I had COVID-19 before, that took a while for me to fill out. I had to keep looking up the questions in my google translator—good thing my daughter showed me how to use the camera to have instant translation” (focus group, 9, Mandarin). Some participants shared how they needed support from family for certain parts of their vaccination experience, particularly registration: “I did some online research and saw an ad about the vaccines, but it was in English, so I called my son and told him, ‘Can you understand the entire text? So, he checked it out and said there would be vaccines. And my son made the appointment for me. I got there and, since I don’t speak much English, I was worried. But when I got there, to my surprise, as soon as a nurse saw me, she came to greet me very kindly in Spanish. And I was very happy because she spoke my language.” (focus group, 4, Spanish).
Theme 4: Community-based organizations played a critical role to connect communities with vaccination opportunities
Community-based organizations were key to connecting participants to available vaccines. One participant explained “I didn’t have the vaccine; I didn’t know where to get it. I talked to a friend, and he told me that [the community organization] helped people. I looked up [the community organization], I decided to call them to ask them about it, and that’s how I got my Pfizer vaccine” (focus group, 4, Spanish). Another participant noted how challenging it was for them until they connected with a community-based organization: “When I got in touch with you [community leader], was when I started getting information, in Spanish, about the entire vaccination process. So much so, that I asked you to help me with my mom’s vaccine, if she was able to come, because there were a lot of questions, no one really knew anything. We would call and ask, in English, if someone from abroad could be vaccinated here, and they didn’t even know” (focus group, 3, Spanish). Community based organizations also worked with vaccine providers in creating culturally-affirming vaccine clinics to facilitate a more comfortable and effective vaccine experience: “The Asian clinic made it really easy! I still hear from members of [community group] that they really liked the ease of communication, good parking, and on a Sunday morning when restaurants (their work places) are not yet open” (focus group, 9, Mandarin).
Theme 5: Unvaccinated groups mistrusted the vaccines and were fearful of side effects
Unvaccinated participants held various perspectives about the COVID-19 vaccines and other vaccines, such as expressing mistrust of the vaccines and fear of side effects, trusting natural immunity or traditional medicines, and defending the personal choice to get vaccinated: “After all, the vaccine was researched and developed in the United States so research subjects were basically Caucasians, instead of all races, for the R&D. So it could be that the protection is worse for Asians than that for Caucasians.” (interview, 13, Mandarin). Fear of long-term side effects from the vaccine was also discussed “I’m thinking about myself, I haven’t had a child yet so I just don’t want that to be affected, I’m still young.” (focus group, 14, French). Participants discussed the strength of traditional medicines and natural immunity as adequate protection against COVID-19: “What we need to know and understand is the medicinal plants or ancestral treatments, which have shown evidence…Regarding this virus, there’s some research currently being conducted in Africa, which proves that there are some medicinal plants… if you take those, it’s very effective, you don’t catch Covid.” (focus group, 15, French).
Many participants expressed their belief in the importance of personal choice to get vaccinated: “Should not force people to get vaccinated, it is a decision for your own health and. . . as you take care of yourself, you are healthy, you should be good enough to not get affected or to heal from the disease if you ever get that” (focus group, 14, French). However, some participants shared that if the vaccine were mandated, they would get it: “Unless the government makes it mandatory. And not getting the vaccine becomes a violation of the law. Or they won’t let you leave your house. Then there’s nothing I could do. I would get the shots.” (interview, 11, Mandarin)
Theme 6: Recommendations to better support IRC communities in improving vaccine equity
Participants shared a variety of suggestions to improve vaccine access, such as providing the vaccine at places of work and doctor’s offices, allowing for walk-in vaccinations, creating drive through clinics, and delivering vaccines directly in homes: “vaccines [should be] easily available for anyone. Maybe bringing them [the vaccines] to places they work due to the fact that they [they employees] don’t have the time or cannot miss work, because employers won’t let them” (focus group, 4, Spanish).
Participants also conveyed the importance of language access, through interpreters or language concordant staff, to improve both vaccine access and trust. One participant discussed their ideal vaccination experience, “of course, it would've been in my first language. It is a moment of vulnerability, if you will, where you are a bit emotional. It’s not that I feel excluded, I don’t know if I’m clear, that would've been ideal for me. Getting the explanation in my language.” (focus group, 3, Spanish). Another participant shared a similar perspective: “I feel happy whenever there’s a chance to speak in Nepali. People who speak English can understand English. They don’t face many difficulties…People who don’t understand English…it makes a big difference” (focus group, 2, Nepali). Participants also touched upon the necessity of increasing information about vaccines in order to improve vaccine trust: “We should enlist more native language speaking people to encourage people to get vaccinated” (focus group, 3, Mandarin).
Discussion
To our knowledge, this is one of the first studies to use community-partnered approaches to understand the COVID-19 vaccination experiences of non-English speaking IRC communities. Consistent with research conducted with English-speaking populations in the US, a major motivator to receive the vaccine is that it may help end the pandemic and return to a sense of normalcy.48 Additionally, trust in scientific and medical communities was cited as a reason to receive (or not receive) the vaccine; this is also consistent with research conducted with English-speaking populations.49 While some themes were similar between cultural groups (e.g. language challenges), others were described more frequently by certain groups, particularly vaccine trustworthiness. Our study reaffirms the need to consider heterogeneity between and within groups and consider disaggregated data around vaccine trust.
An important theme which emerged across all language groups was how family members were needed to serve as interpreters throughout the vaccination process. Language brokering, or the use of family members as interpreters or translators, has been described in the literature as common during interactions with the healthcare system.50 However, use of family members as interpreters is deemed unacceptable for a variety of reasons, including the potential for errors in interpretation, negative impacts on the language broker, and potential breaches of confidentiality.51-53 Our study found that during this unprecedented public health emergency interpretation services were not widely available requiring many participants to rely on family members.
Discussions around the vaccine experience revealed concern in some immigrant communities around the information and documentation required to receive the vaccine. Considering historical and current day policies and practices rooted in xenophobia and racism (i.e. border wall, public charge, forced loss of language) such fear is not unfounded.54-56 COVID-19 vaccines were offered free of charge to all in the United States; however, there was considerable personal information requested (i.e. race and ethnicity). Immigrants living without documentation may hold well-founded fears about engaging with the healthcare systems, which, subsequently may decrease access to healthcare.57,58 Our study adds to this existing work by demonstrating similar concerns accessing vaccines, even outside of the traditional healthcare system.
Central to our study is the role that community-based organizations played in supporting the vaccination experience of participants, through finding vaccines, answering questions, and creating culturally-affirming clinics. Past research similarly has shown how community partners were important messengers throughout the COVID-19 pandemic.59-61 However, reliance on community organizations can also place an undue burden; in fact, many in the authorship team spent considerable time organizing vaccine clinics on top of other responsibilities. Our work highlights the importance of providing adequate resources and funding to community-based organizations during public health emergencies and natural disasters to facilitate their life-saving work.
Participants who received their vaccine in culturally affirming clinics generally reported an overall positive vaccination experience, further signaling the importance of centering culture in healthcare delivery. Culturally affirming (or culturally humble) care is a broad, multifaceted concept centered in respect of individuals' unique experiences and desire to learn from others.62 In our study, examples of culturally-affirming care included access to interpreters, partnerships with community-based organizations, and having the clinic at times that made it logistically feasible to attend. Many of these practices seek to address deeply rooted structural inequities by dismantling oppressive policies and practices rooted in xenophobia, language injustice, and transportation access. Further, the culturally affirming practices were often in response to unjust systems; as an example, having interpreters when they are not readily available. Our study highlights the importance of including culturally affirming services proactively throughout the healthcare system, rather than reactively after developing a system that is not accessible to IRC communities.
Our study is subject to multiple limitations. A strength of this study is that community leaders were involved in all parts of the research process, including recruitment, data collection, and analysis. However, we recognize that the participants in this study were likely less isolated as they were connected to a community-based organization. It will be important to also hear the perspectives of participants who are more socially isolated. Further, as the interviewers were community leaders it is possible that some of the participants may know them, which may have created social desirability bias. We conducted focus groups in multiple languages but were unable to hear the perspectives of non-English speaking participants who speak other languages. We used a combination of focus groups and interviews, based on feedback from our community-partner team that some unvaccinated individuals preferred to not speak as a group. However, we recognize that questions may be interpreted differently for interviews versus focus groups. We also recognize that in this exploratory qualitative study, we are unable to compare perspectives of different immigrant and refugee groups.
Implications
This study sets the stage for future research and clinical innovation. Larger studies on the COVID-19 vaccination experience should be conducted with non-English speaking communities in the United States to triangulate some of the findings from this study and explore regional differences. These studies should also consider comparing the COVID-19 vaccination experiences of different immigrant and refugee groups. In particular, availability of interpreter services at vaccination clinics and the use of language brokering are important to understand. Studies should also examine the impact of the COVID-19 pandemic on community-based organizations who helped support vaccination efforts; this work should both celebrate the role they played and highlight how these efforts were supported and funded. Finally, longitudinal studies, assessing people’s perspectives about COVID-19 vaccines over time would be helpful especially to understand the post-vaccination opinions of individuals who agreed to get vaccinated due to work requirements.
Several practice and policy implications also emerge from this work both for vaccination clinics currently and in the future, as well as the healthcare system more broadly. Minimizing the amount of information needed from patients at vaccine clinics and having policies where identification is not needed would be helpful in making clinics more supportive. Provider training and system-wide policies that prohibit language brokering and provide free and confidential interpreter services are critical. Further, linguistically accessibility must be considered whenever a patient is interfacing with the healthcare system (e.g. registration, making appointments, follow up), not only during the clinical visit. Healthcare providers should also be aware of anti-discrimination policies which require institutions receiving federal funding to provide quality medical care and interpretation to non-English speaking individuals.63-65 Federal guidance in the form of an organizational self-assessment and sample language access plan offers a helpful starting point for providers.66
Community-based organizations who are supporting vaccination efforts must be compensated for their time and provided adequate resources. Healthcare systems should also engage community-based organizations in their work more broadly, as they are often trusted in communities and are familiar with the needs and barriers of the communities they serve. Finally, language accessibility must be included in disaster preparedness plans, to ensure that interpreter and translation services are readily available.
Author Biographies
Yesmina Salib was the Program Coordinator for the Refugee and Immigrant Support Group program at Jewish Family and Community Services when this study took place. She is currently completing her master's degree and working as a researcher dedicated to meaningfully engaging with stakeholders and exploring social justice issues.
Joseph Amodei is an Assistant Professor of Immersive Media at Chatham University in Pittsburgh, Pennsylvania. They are a designer, researcher, activist, and educator whose interdisciplinary work operates at the intersections of art, technology, and community.
Claudia Sanchez was the Healthcare Coordinator for Casa San Jose.
Ximena Alejandra Castillo Smyntek (Ale) is the founder and director of Familia & Comunidad and Amigas & Amigos (Am+A) outreach. She works as a Community Health Deputy (CHD) for the Neighborhood Resilience Project (NRP) with Familia & Comunidad Southwestern PA outreach.
Marian Lien is an advocate and educator working to connect the needs of her Asian American Pacific Islander community to better resources and experts. She is the first Director of Equity and Inclusion at St. Edmund’s Academy, an independent school in Pittsburgh.
Sabrina Yowchyi Liu is a volunteer at the Asian Pacific American Labor Alliance Pittsburgh Chapter (APALA Pittsburgh). Her work with APALA Pittsburgh focuses on promoting workers' rights and immigrants' rights. She is an immigrant from Taiwan.
Geeta Acharya is an Epidemiology student at the University of Pittsburgh School of Public Health. She works as a research assistant at Pitt Public Health.
Benoit Kihumbu is an Data Reporting Analyst at SouthWest Behavioral Health Management Inc, Leader of the Congolese Union of Pittsburgh, Co chair in the immigrants Advisory Council through Allegheny County department of Human Services and also a Judo instructor at South Hills Judo Academy.
Pralad Mishra is a Regional Manager at Candid Homecare Inc in Pittsburgh. He is one of the founding members of Global Bhutanese Hindu Organization (GBHO) and Omkar Pariwar of Pittsburgh. He is the present General Secretary of Omkar Pariwar of Pittsburgh and the Assistant Secretary and Volunteers Enrollment and Management Director in GBHO.
Diego Chaves-Gnecco is an Associate Professor of Pediatrics at the University of Pittsburgh and founder and director of Salud Para Ninos, a bilingual and bicultural medical home for Spanish-speaking families.
Khara Timsina is the Executive Director at Bhutanese Community Association of Pittsburgh (BCAP). Khara works on community development through information, education, advocacy on racial equity and inclusion, health and civic engagement.
Jenny Diaz is the Health Promotion Specialist for Casa San Jose.
Constanza Henry is the Healthcare Coordinator for Casa San Jose, with a background in Communications, she focuses on trying to find better healthcare access for the Latinx and refugee communities as well as organize free health clinics for said communities.
Erin Mickievicz is a research assistant in the department of pediatrics at the University of Pittsburgh. She is a recent graduate with a BS in psychology and a background in adolescent health and wellness research.
Aweys Mwaliya, executive director of the Somali Bantu Community Association of Pittsburgh, former case manager of Utah Health & Human Rights, and Worldwide Flight Services Passengers Services Agent at Salt Lake International Airport. Co-founded the Somali Bantu Community Association of Utah.
Ken Ho is an assistant professor in the School of Medicine at the University of Pittsburgh. His research focus is on HIV prevention and implementation of HIV pre exposure prophylaxis. He has also been involved as investigator and coinvestigator on multiple Covid-19 vaccine trials and epidemiological studies.
Jaime Sidani is an Assistant Professor of Public Health at the University of Pittsburgh and a health educator by training. She is a mixed-methods researcher whose work focuses on substance use prevention, health misinformation on social media, and youth empowerment.
Maya Ragavan is an Assistant Professor of Pediatrics at the University of Pittsburgh and a pediatrician at UPMC Children's Hospital of Pittsburgh. She is a community-partnered, mixed-methods researcher whose work focuses on intimate partner violence prevention, immigrant and refugee health, and language equity.
Footnotes
Author contributions: Ms Salib and Dr Ragavan drafted the initial manuscript; all authors provided edits and significant revisions. All authors were part of the research team and participated in conceptualization, recruitment, data collection, and analysis.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We received funding for this study from a University of Pittsburgh Clinical and Translational Science Institute pilot grant.
Ethical approval: This study was deemed exempt by the University of Pittsburgh Institutional Review Board.
Informed consent: All participants reviewed an information script (as per our IRB) and verbally agreed to participate in the study.
Data availability: Raw data and materials are available by contacting the corresponding author.
Code availability: Codes and codebooks are available by contacting the corresponding author.
ORCID iD
Maya I Ragavan https://orcid.org/0000-0003-4443-6829
References
- 1.Khazanchi R, Evans CT, Marcelin JR. Racism, not race, drives inequity across the COVID-19 continuum. JAMA Netw Open 2020; 3(9): e2019933. [DOI] [PubMed] [Google Scholar]
- 2.Le TK, Cha L, Han HR, et al. Anti-Asian xenophobia and Asian American COVID-19 disparities. Am J Public Health 2020; 110(9): 1371–1373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Elias A, Ben J, Mansouri F, et al. Racism and nationalism during and beyond the COVID-19 pandemic. Ethnic Racial Stud 2021; 44(5): 783–793. [Google Scholar]
- 4.Clark E, Fredricks K, Woc-Colburn L, et al. Disproportionate impact of the COVID-19 pandemic on immigrant communities in the United States. PLOS Negl Trop Dis 2020; 14(7): e0008484. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Doan LN, Chong SK, Misra S, et al. Immigrant Communities and COVID-19: Strengthening the Public Health Response. Am Journal Public Health 2021; 111(S3): S224–S231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Department of Homeland Security Office of Immigration Statistics . COVID-19 Vulnerability by Immigration Status: Status-specific Risk Factors and Demographic Profiles. Department of Homeland Security, 2021. Available at: https://www.dhs.gov/sites/default/files/publications/immigration-statistics/research/reports/research/paper/covid-19/vulnerability/by/immigration/status/may/2021.pdf [Google Scholar]
- 7.Saifee J, Franco-Paredes C, Lowenstein SR. Refugee health during COVID-19 and future pandemics. Curr Trop Med Rep 2021; 8(3): 1–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Greenaway C, Hargreaves S, Barkati S, et al. COVID-19: exposing and addressing health disparities among ethnic minorities and migrants. J Travel Medicine 2020; 27(7): taaa113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Wilder JM. The Disproportionate Impact of COVID-19 on Racial and Ethnic Minorities in the United States. Clin Infect Dis 2021; 72(4): 707–709, DOI: 10.1093/cid/ciaa959 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Williams JC, Anderson N, Holloway T, et al. Reopening the United States: Black and Hispanic Workers Are Essential and Expendable Again. Am J Public Health 2020; 110(10): 1506–1508. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Thomas CM, Osterholm MT, Stauffer WM. Critical Considerations for COVID-19 Vaccination of Refugees, Immigrants, and Migrants. Am J Trop Med Hyg 2021; 104(2): 433–435. Published 2021 Jan 13. DOI: 10.4269/ajtmh.20-1614 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Hildreth JEK, Alcendor DJ. Targeting COVID-19 vaccine hesitancy in minority populations in the US: implications for herd immunity. Vaccines 2021; 9(5): 489. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Gonzalez D, Karpman M, Bernstein H. COVID-19 Vaccine Attitudes Among Adults in Immigrant Families in California: Insights and Opportunities to Promote Equitable Access. Urban Institute, 2021. Available at: https://www.urban.org/sites/default/files/publication/103973/covid-19-vaccine-attitudes-among-adults-in-immigrant-families-in-california/0/0.pdf [Google Scholar]
- 14.Zhang M, Gurung A, Anglewicz P, et al. Acceptance of COVID-19 vaccine among refugees in the United States. Public Health Rep 2021; 136(6): 774–781. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Demeke J, McFadden SM, Dada D, et al. Strategies that promote equity in COVID-19 vaccine uptake for undocumented immigrants: a review. J Community Health 2022; 47: 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Children’s Hospital of Philadelphia Policy Lab . Supporting Immigrant Communities and Those with Limited English Proficiency during the Pandemic: Stakeholder Perspectives from Pennsylvania. Philadelphia PA: Children’s Hospital of Philadelphia, 2020. Available at: https://policylab.chop.edu/sites/default/files/pdf/publications/PolicyLab-Survey-Immigrant-and-LEP-Communities-During-COVID-19-Pandemic.pdf [Google Scholar]
- 17.Ta Park VM, Dougan M, Meyer OL, et al. Vaccine willingness: Findings from the COVID-19 effects on the mental and physical health of Asian Americans & Pacific Islanders survey study (COMPASS). Prev Med Rep 2021; 23: 101480. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Al Shamsi H, Almutairi AG, Al Mashrafi S, et al. Implications of language barriers for healthcare: a systematic review. Oman Med J 2020; 35(2): e122–e122. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Omenka OI, Watson DP, Hendrie HC. Understanding the healthcare experiences and needs of African immigrants in the United States: a scoping review. BMC Public Health 2020; 20(1): 27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Burgess RA, Osborne RH, Yongabi KA, et al. The COVID-19 vaccines rush: participatory community engagement matters more than ever. The Lancet 2021; 397(10268): 8–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Ragavan MI, Ripper L, Davidson M, et al. COVID-19 information sources for Black and Latine communities: Lessons learned from a community co-created survey. Prog Community Health Partnerships. Epub ahead of print. [DOI] [PubMed]
- 22.Department of Homeland Security FEMA . FEMA Funds Community Engagement for COVID-19 Vaccinations. Department of Homeland Security, 2021. Available at: https://www.fema.gov/fact-sheet/fema-funds-community-engagement-covid-19-vaccinations [Google Scholar]
- 23.Sanchez-Youngman S, Boursaw B, Oetzel J, et al. Structural community governance: importance for community-academic research partnerships. Am Journal Community Psychology 2021; 67(3–4): 271–283. [DOI] [PubMed] [Google Scholar]
- 24.Israel BA, Eng E, Schultz AJ, et al. Methods for Community-Based Participatory Research for Health. 2rd ed.. Jossey-Bass, 2012. [Google Scholar]
- 25.Jacquez F, Vaughn LM, Wagner E. Youth as partners, participants or passive recipients: a review of children and adolescents in community-based participatory research (CBPR). Am Journal Community Psychology 2013; 51(1–2): 176–189. [DOI] [PubMed] [Google Scholar]
- 26.Vaughn LM, Jacquez F, Lindquist-Grantz R, et al. Immigrants as research partners: a review of immigrants in community-based participatory research (CBPR). J Immigrant Minor Health 2017; 19(6): 1457–1468. [DOI] [PubMed] [Google Scholar]
- 27.Scott T, Gutschow B, Ragavan MI, et al. A community partnered approach to promoting COVID-19 vaccine equity. Health Promot Pract 2021; 22(6): 758–760. [DOI] [PubMed] [Google Scholar]
- 28.Chen E, Leos C, Kowitt SD, et al. Enhancing community-based participatory research through human-centered design strategies. Health Promot Pract 2020; 21(1): 37–48. [DOI] [PubMed] [Google Scholar]
- 29.Batalova J, Hanna M, Levesque C. Frequently Requested Statistics on Immigrants and Immigration in the United States. Migration Policy Institute. 2/11/2021. Available at: https://www.migrationpolicy.org/article/frequently-requested-statistics-immigrants-and-immigration-united-states-2020#immig-now-historical (Accessed 9/16/2022). [Google Scholar]
- 30.Allegheny County . Services for Immigrants and Internationals. Pittsburgh (PA): Allegheny County. Available at: https://www.alleghenycounty.us/Human-Services/Resources/Immigrants-Refugees/Services-for-Immigrants-and-Internationals.aspx [Google Scholar]
- 31.Conway B. Pittsburgh’s Bhutanese Community Finds a Home in Carrick. NEXT Pittsburgh, 2017. Available athttps://nextpittsburgh.com/features/pittsburghs-bhutanese-community-finds-home-carrick/ [Google Scholar]
- 32.Overview of Immigrant Populations in Pittsburgh. Grantmakers of Western Pennsylvania, 2017. Available at: https://gwpa.org/sites/default/files/resources/Overview%20of%20Immigrant%20Populations%20in%20City%20of%20Pittsburgh.pdf [Google Scholar]
- 33.First Look at the 2020 Decennial Census: Pittsburgh Region. Pittsburgh PA: University of Pittsburgh Center for Social and Urban Research, 2021. Available at: https://ucsur.pitt.edu/perspectives.php?b=20210821103378 [Google Scholar]
- 34.Gillogly K. The Fight to Recognize Pittsburgh’s Lost Chinatown. Pittsburgh PA: Pittsburgh Magazine, 2019. Available at: https://www.pittsburghmagazine.com/the-fight-to-recognize-pittsburghs-lost-chinatown/ [Google Scholar]
- 35.Davis KJ. What Happened to Pittsburgh’s Chinatown? Pittsburgh PA: 90.5 WESA, 2019. Available at: https://www.wesa.fm/arts-sports-culture/2019-02-14/what-happened-to-pittsburghs-chinatown [Google Scholar]
- 36.Allegheny County PA. Data USA. Available at: https://datausa.io/profile/geo/allegheny-county-pa/ [Google Scholar]
- 37.American Community Survey: Allegheny County, Pennsylvania. United States Census Bureau. Available at: https://data.census.gov/cedsci/table?q=allegheny%20county&tid=ACSST1Y2019.S1601 [Google Scholar]
- 38.American Community Survey: Westmoreland County, Pennsylvania. United States Census Bureau. Available at: https://data.census.gov/cedsci/table?q=Westmoreland%20county&t=Native%20and%20Foreign%20Born [Google Scholar]
- 39.Viruell-Fuentes EA, Miranda PY, Abdulrahim S. More than culture: structural racism, intersectionality theory, and immigrant health. Soc Sci Med 2012; 75(12): 2099–2106. [DOI] [PubMed] [Google Scholar]
- 40.Misra S, Kwon SC, Abraído-Lanza AF, et al. Structural racism and immigrant health in the United States. Health Edu Behav 2021; 48(3): 332–341. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Bowleg L. The problem with the phrase women and minorities: intersectionality-an important theoretical framework for public health. Am J Public Health 2012; 102(7): 1267–1273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Kahlke RM. Generic qualitative approaches: pitfalls and benefits of methodological mixology. Int J Qual Methods 2014; 13(1): 37–52. [Google Scholar]
- 43.Patton MQ. Qualitative Research and Evaluation Methods. 4th ed. Thousand Oaks, CA: Sage, 2015. [Google Scholar]
- 44.Namey E, Guest G, McKenna K, et al. Evaluating Bang for the Buck. Am J Eval 2016; 37(3): 425–440. [Google Scholar]
- 45.Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006; 3(2): 77–101. [Google Scholar]
- 46.Dedoose 7.5.16. Web Application for Managing, Analyzing, and Presenting Qualitative and Mixed Methods Research. Los Angeles, CA: SocioCultural Research Consultants LLC; 2016. [Google Scholar]
- 47.Miro . Available at: https://miro.com/app/dashboard/
- 48.El-Mohandes A, White TM, Wyka K, et al. COVID-19 vaccine acceptance among adults in four major US metropolitan areas and nationwide. Scientific Rep 2021; 11(1): 21844. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Viswanath K, Bekalu M, Dhawan D, et al. Individual and social determinants of COVID-19 vaccine uptake. BMC Public Health 2021; 21(1): 818. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Diamond LC, Schenker Y, Curry L, et al. Getting by: underuse of interpreters by resident physicians. J Gen Intern Med 2009; 24(2): 256–262. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Paradise RK, Hatch M, Quessa A, et al. Reducing the use of ad hoc interpreters at a safety-net health care system. Jt Comm J Qual Patient Saf 2019; 45(6): 397–405. [DOI] [PubMed] [Google Scholar]
- 52.Juckett G, Unger K. Appropriate use of medical interpreters. Am Family Physician 2014; 90(7): 476–480. [PubMed] [Google Scholar]
- 53.Office of Minority Health . National Standards for Culturally and Linguistically Appropriate Services in Health and Health Care: A Blueprint for Advancing and Sustaining CLAS Policy and Practice. Department of Health and Human Services, 2013. Available at: https://thinkculturalhealth.hhs.gov/assets/pdfs/EnhancedCLASStandardsBlueprint.pdf [Google Scholar]
- 54.Kim SY, Wang Y, Weaver SR, et al. Measurement equivalence of the language-brokering scale for Chinese American adolescents and their parents. J Fam Psychol 2014; 28(2): 180–192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Vernice NA, Pereira NM, Wang A, et al. The adverse health effects of punitive immigrant policies in the United States: a systematic review. PLoS One 2020; 15(12): e0244054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Cholera R, Falusi OO, Linton JM. Sheltering in place in a xenophobic climate: COVID-19 and children in immigrant families. Pediatrics 2020; 146(1): e20201094. [DOI] [PubMed] [Google Scholar]
- 57.Hacker K, Anies M, Folb BL, et al. Barriers to health care for undocumented immigrants: a literature review. Risk Manag Healthc Pol 2015; 8: 175–183. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Beck TL, Le TK, Henry-Okafor Q, et al. Medical Care for Undocumented Immigrants. Prim Care Clin Off Pract 2017; 44(1): e1–e13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Fields J, Gutierrez JR, Marquez C, et al. Community-academic partnerships to address COVID-19 inequities: Lessons from the San Francisco Bay Area. NEJM Catal; 2021. [Google Scholar]
- 60.Dada D, Djiometio JN, McFadden SM, et al. Strategies that promote equity in COVID-19 vaccine uptake for Black communities: a review. J Urban Health 2022; 99(1): 15–27. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Wieland ML, Asiedu GB, Lantz K, et al. Leveraging community engaged research partnerships for crisis and emergency risk communication to vulnerable populations in the COVID-19 pandemic. J Clin Translational Sci 2020; 5(1): e6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Kibakaya EC, Oyeku SO. Cultural Humility: A critical step in achieving health equity. Pediatrics 2022; 149(2): e2021052883. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Civil Rights Act of 1964 § 7, 42 U.S.C. § 2000e et seq; 1964. [Google Scholar]
- 64.Exec. Order No. 13166, 3 C.F.R. 50121; 2000. Availavle at: https://www.govinfo.gov/content/pkg/FR-2000-08-16/pdf/00-20938.pdf [Google Scholar]
- 65.Patient Protection and Affordable Care Act of 2010, Pub. L. No. 111–148, 124 Stat. 260; 2010. [Google Scholar]
- 66.United States Department of Justice . Language Access Assessment and Planning Tool for Federally Conducted and Federally Assisted Programs, 2011. Available at: https://www.lep.gov/sites/lep/files/resources/2011/Language/Access/Assessment/and/Planning/Tool.pdf [Google Scholar]