Lessons Learned: Cultural and linguistic enhancement of surveys through community-based participatory research

Abstract

Background

Surveys are frequently implemented in community-based participatory research (CBPR), but adaptation and translation of surveys can be logistically and methodologically challenging when working with immigrant and refugee populations.

Objective

To describe a process of participatory survey adaptation and translation.

Methods

Within an established CBPR partnership, a survey about diabetes was adapted for health literacy and local relevance and then translated through a process of forward translation, group deliberation, and back translation.

Lessons Learned

The group deliberation process was the most time-intensive and important component of the process. The process enhanced community ownership of the larger project while maximizing local applicability of the product.

Conclusions

A participatory process of survey adaptation and translation resulted in significant revisions to approximate semantic, cultural, and conceptual equivalence with the original surveys. This approach is likely to enhance community acceptance of the survey instrument during the implementation phase.

Community-based participatory research (CBPR) is a means to collaboratively investigate health topics within a community whereby community members and academics partner in an equitable relationship through all phases of the research and programming process.¹ It is an intuitively appropriate approach for addressing health disparities in a sociocultural context by fostering collaborative engagement between researchers and community members to select research topics, collect data, and interpret results.^2-5 Further, CBPR has demonstrated success in targeting health issues among immigrant and refugee populations.^2-4

Surveys are commonly employed in CBPR across a variety of health topics. When combined with qualitative work and epidemiologic data, surveys add a critical step to comprehensive pre-intervention assessment of a health issue in a very local context. However, surveys are often difficult to implement in CBPR with immigrant and refugee populations. Survey instruments with acceptable validity to answer questions that are important for partnerships are frequently not available for diverse languages and ethnicities. Beyond the challenges of semantic equivalence, simple translation of existing English-language surveys will often lose conceptual equivalence with the original survey; likewise, translated surveys run the risk of ignoring differences in cultural norms that may impact participant reflection on a health topic.⁶ Accordingly, there are recommended approaches described in the literature aimed at mitigating these pitfalls.^7-9

In CBPR, this time and resource-intensive process of survey translation is daunting. However, a participatory process of survey translation using one of these frameworks has the potential to not only improve the survey product, but also to strengthen existing partnerships and eventual survey implementation.^10,11 It is with this shared expectation of benefit that our established CBPR partnership in Rochester, Minnesota, conducted participatory translation of a multi-instrument diabetes survey from English to three languages: Cambodian, Spanish, and Somali. In this paper we use our experience with the Somali translation to frame lessons learned from the process.

The diabetes epidemic results in significant societal and health care costs, affecting 8.3% of the United States (US) population with an economic impact estimated to be $174 billion annually.¹² As a whole, immigrants and refugees arrive to the US with a lower prevalence of diabetes than the general population, but rates begin to rise shortly after arrival.^13-15 Once diagnosed with diabetes, immigrants and refugees are less likely to adhere with diabetes care recommendations than US-born patients with diabetes, thereby placing them at higher risk for complications.¹⁶ Further, we have recently documented health disparities among Somali patients with diabetes in our community.¹⁷ Therefore, as one piece of a needs assessment to inform future intervention work aimed at reducing these disparities, we intend to conduct a survey among members of the Somali community (among other immigrant and refugee communities) in Rochester who have diabetes, to understand their disease-related knowledge, attitudes, and behaviors. Here, we describe lessons learned from the participatory process of survey translation from English to Somali.

METHODS

Partnership description

In 2004, a community-academic partnership developed between Mayo Clinic and Hawthorne Education Center, an adult education center that serves approximately 2000 immigrants and refugees per year through coursework, connection to community resources, and a health clinic. Between 2005 and 2007, this partnership matured by formalizing operating norms, adapting CBPR principles, and adding many dedicated community and academic partners to form the Rochester Healthy Community Partnership (RHCP). The mission of RHCP is to promote health and well-being among the Rochester community through CBPR, education, and civic engagement to achieve health equity.⁵ Since its inception, RHCP has matured, developed a community-based research infrastructure, and become productive and experienced at deploying data-driven assessments and interventions with immigrant and refugee populations.^3,4 Community and academic partners have conducted all phases of research together including joint dissemination of research results at community forums and academic meetings.¹⁸

RHCP community leaders have prioritized diabetes as one of the most important health threats to their communities. Further, disparities in diabetes care have been documented among Somali patients in our community.¹⁷ RHCP community and academic partners have developed a long-term intervention framework to address this disparity. As part of a needs assessment to inform this future intervention work, a survey was written to understand diabetes-related knowledge, attitudes and behaviors among community members with the disease across several different ethnic groups.

RHCP community and academic partners agreed that a participatory process of survey adaptation (from existing surveys) and translation was the most effective means of overcoming the challenges of survey development for socio-linguistically heterogeneous community members. In this paper, we describe lessons learned from this participatory process of survey adaptation and translation from Somali to English. RHCP community leaders for these activities in the Somali community were from the Somali Community Resettlement Services. The mission of this organization is to promote and advance the social well-being and welfare of its members by providing community and resettlement services, with the ultimate goal of building a vibrant community, whose members can successfully settle and effectively integrate into the wider American Community. Health promotion is a key pillar of their work, and they have been a long-term RHCP member.

Diabetes survey instruments

Based on literature review and group discussion between RHCP community and academic partners, several diabetes survey instruments were chosen for adaptation to assess diabetes-related knowledge, attitudes, and behaviors. Survey instruments were adapted with permission from the authors including the Diabetes History Survey¹⁹, Diabetes Care Profile²⁰, the Summary of Diabetes Self-Care Activities Measure²¹, and the Diabetes Knowledge Questionnaire²². There were a total of 135 survey items from the original instruments. Survey implementation was approved by the Mayo Clinic Institutional Review Board.

Survey adaptation for cultural and linguistic relevance

While all surveys described above carry adequate validity, the application of these instruments to our culturally and linguistically heterogeneous community was methodologically problematic. We attempted to minimize this limitation through two processes: 1) editing of original (English language) survey items for meaning and local applicability, and 2) editing of each survey item by forward translation, group deliberation, and backward translation by a core group of community leaders following the World Health Organization's (WHO) translation process.⁷

Through a series of three meetings (approximately 90 minutes per meeting), a working group of two academic partners and six bilingual community partners from the Somali, Hispanic, and Cambodian communities adapted the original survey items for target populations with low health literacy. The content of some items were deemed inappropriate on the basis of low cultural or experiential relevance. These items were either revised or eliminated. A process of group deliberation, debate, and consensus was used to finalize each revision to the English language survey.

The WHO translation process for the Somali translation started with forward translation conducted by a professional Somali translator who was not affiliated with the partnership. The panel discussion/group deliberation was conducted through 12 hours of meetings between three Somali RHCP community partners (A.A, A.O, A.H.) and one academic partner (C.F.). Two of the community partners had also participated in the group-based adaptation of the English language survey described above. During the panel discussion meetings, the translated survey was scrutinized for meaning. Group deliberation between Somali partners and the academic partner culminated in a consensus Somali translation. The same process of group deliberation, debate and consensus was used to finalize each change to the translated version of the survey. The purpose of these deliberations was to ensure a translation that mirrored the meaning of the original survey items with added cultural and local applicability to ensure that participants respond in the manner intended. The consensus Somali translation was then back-translated to English by a second professional interpreter (neither interpreter was involved in the meetings). The RHCP academic partner examined the back translation to ensure that the meaning of original English-language survey items was maintained. Items with discordance between the original survey and the back translation were brought back to the Somali community panel for further revision. These steps culminated in a final Somali-language diabetes survey.

Notes were taken by the academic partners through all of the meetings. These notes were discussed in real time with community partners at the group meetings to reflect on and summarize the ways in which the survey was changed as a result of the process. Notes were further inductively organized and analyzed for meaning to elucidate generalizable themes highlighting benefits and limitations of the participatory meetings on the final survey instrument and its eventual implementation. These resultant “lessons learned” were shared and revised by the community collaborators to reflect the most important lessons that arose from the process.

RESULTS

The Somali participatory survey adaption process resulted in significant changes to the final product that enhanced cultural and linguistic applicability. First, editing of the original English-language survey resulted in a reduction in total items from 135 to 110. Questions were most commonly eliminated across an entire domain (rather than individual questions within a domain) so that scoring of domains could be consistent with the original survey intent. Domains were eliminated by the group if they were deemed low priority by the partnership, or if the concepts contained in the question stem were too abstract. Less commonly, individual questions were eliminated within a domain if it did not threaten the scoring/interpretation scheme of the items. Individual questions were most commonly eliminated if they were written from an ethnocentric vantage point that could not be altered without losing the intent of the item. Almost all of the remaining 110 items underwent minor changes to either the question stem or response scales to enhance relevance and meaning for participants with low health literacy. For example, many phrases were modified by the group to use simpler language and to eliminate colloquialism.

The panel discussion also resulted in significant changes to the Somali translation of the survey. These discussions prevented flaws that were not identified or recognized in the initial process of translation. The back translation confirmed that the modified translation did not compromise meaning of the original survey items.

Issues of literacy resulted in changes to the survey. First, the panel concluded that participants from their community have very little experience filling out surveys and the lack of familiarity with Likert scales may be problematic. This was mitigated through development of graphics to depict the meaning of scale extremes (e.g., faces) and through use of percentages when appropriate (e.g., “somewhat agree” response option augmented with “50% agree”). Second, community partners highlighted the fact that the written Somali language is relatively new (1974). As a result, many Somali refugees who resettled in Rochester have not learned to read or write the Somali language. Therefore, the survey will be delivered in an interview format with a Somali-speaking facilitator. Finally, many individual phrases were changed from the literal translation to a phrase that more accurately mirrored the intent of the original survey. For example, Somali panelists stated that the literal translation of the phrase “healthy food” is often interpreted as all food eaten in a normal diet, including food that would be considered unhealthy. Therefore, the Somali translation was augmented to reflect the correct concept, which was back-translated as “special diet”.

Cultural and religious practices had a major impact on survey translation. First, panel members described a stigma about diabetes and illness in general (not unique to the Somali population) that may lead participants to answer in ways that minimize the severity of their disease. This prompted subtle changes to question stems or approaching the question from a different direction so that participants feel more comfortable stating the true nature of their illness. Second, Somali panel members stated that religious and cultural norms would especially impact items that address quality of life. Because health and disease may be seen as determined by Allah, survey participants may be likely to report maximal quality of life and low physical or emotional disease burden. Therefore, question stems and response choices were carefully re-worded to facilitate a true range of permissible response options. Panel members recommended changing these items to be more concrete because general questions about well-being would generate overly positive responses. For example, when assessing physical well-being, participants changed the translated language to include examples of physical strength and activity. Likewise, questions related to health-related emotional well-being were augmented to ask more directly how much participants worried about their health.

Finally, survey implementation will be significantly impacted by our process. As stated above, we intend to adopt an interview format based on literacy concerns. Based on cultural norms of gender-based interaction, there should be an attempt to ensure gender concordance between the facilitator and survey participants. Further, community partners noted that the comfort level of participants should be maximized by delivering the survey in a space familiar to Somali families. Finally, distrust of researcher motives should be addressed by demonstrating community engagement through active participation in survey implementation by RHCP Somali partners and by engaging in dialogue prior to survey implementation that frames the purpose of the survey in broader terms, including future intervention work. These recommendations will all be operationalized when the survey is implemented.

LESSONS LEARNED

There were several lesson learned about the participatory process of survey adaptation and translation:

Participatory survey adaptation and translation enhances community ownership of the instrument and process

Through leadership in every phase of survey development, community partners are invested in leading the survey implementation. Their leadership in implementation will be critical to its success. Further, this “ownership” of the process helps develop a shared vision of the eventual impact of this work on persons with diabetes in their community. All of these factors increase the likelihood of results impacting policy and practice in our community.

Participatory survey adaptation and translation improves local applicability and relevance of the instrument

Even within specific immigrant groups (e.g., Somali-Americans), there is complex cultural heterogeneity stemming from geographic, social, and economic variability in their home country and from reactions to differing regional externalities in their new country (e.g., rural versus urban). Therefore, the participatory process of survey development results in a survey instrument that is intensely local, thereby maximizing cultural equivalency with the original survey.

Group deliberation is the most important component of survey translation in CBPR

The most significant changes to the survey resulted from the group deliberation process. In the WHO survey translation process, this step approximates the “expert panel” discussion of translated material. Participatory engagement adds to this step by recognizing the essential role of community expertise. By pairing community experts with a content expert for group deliberation, a survey is revised to reflect social, cultural, and local norms while maintaining accuracy of the content.

We were surprised by the amount of time necessary to get this step right. Discussions were lively and engaged. Group members should be composed of representative community partners who are comfortable with each other and who are all comfortable having their voices heard. Social power imbalances within the group should be avoided. Time and resources should be prioritized to maximize the experience of this group. Accordingly, the most significant limitation of this approach is the large amount of time needed to pay sufficient attention to this step. We found that forward and back translation does not necessarily benefit from a participatory process; these steps could be completed outside the partnership (e.g., contracted services) if that is more logistically feasible.

Participatory survey adaptation and translation informs survey implementation, not just survey development

Surprisingly, issues surrounding survey implementation were equally, if not more important than survey development, to the entire process. Panel members were invaluable in addressing key components of survey setting, delivery, and personnel. Further, panel members themselves are likely to take a leadership role in survey implementation to ensure that their recommendations are carried out. While this survey has not yet been implemented, recommendations derived in the group deliberation will be fully incorporated. Importantly, the need for these changes to implementation would likely not have been fully recognized in pilot testing of the survey, where the implementation focus is typically on timing issues rather than on how social nuance of the setting and delivery could impact answers provided by participants.

CONCLUSIONS

Our experience with participatory survey adaptation and translation demonstrates that this approach is feasible for use with immigrant and refugee groups engaged in a CBPR partnership. Our expectation is that this participatory process of survey adaptation and translation will preserve validity and reliability constructs of the original survey instruments. Based on the extensive revisions completed in our group deliberation process, simple translation of the survey would almost certainly not achieve psychometric benchmarks. Face validity (how well questions address a construct) of a survey is clearly enhanced by community expertise during adaptation and group deliberation. Likewise, content validity (how well questions represent all facets of the construct) should be preserved through the final step of comparing content of the forward translation with the original English-language surveys, for which content validity was incorporated. During data analysis, we will evaluate internal consistency (Cronbach α) of each survey domain as a measure of reliability, which we anticipate will be preserved as a result of the participatory process.

In summary, a process of participatory survey adaptation and translation with CBPR partners from immigrant and refugee communities was feasible. The survey underwent significant cultural and linguistic revisions to approximate semantic, cultural, and conceptual equivalence with the original surveys. This approach is likely to enhance community acceptance of the survey instrument during the survey implementation phase. Finally, the participatory process of this work is likely to beget an equally deep level of community participation in survey implementation, data analysis, dissemination, and subsequent intervention development.