Translation Cost, Quality and Adequacy

Abstract

Purpose

Although the inclusion of non-native-speaking participants in nursing research is important in every country where nursing research takes place, the literature contains little on the method of achieving quality translation while simultaneously addressing cost containment. We describe a process for evaluating translation adequacy and demonstrate its use in comparing procedures for translating data from non-native-speaking interviews.

Organizing Construct

This work demonstrates a process for establishing, evaluating, and achieving translation adequacy when conducting qualitative research for cross-cultural comparisons.

Method

In an ethnographic investigation of disability in Mexican American women, we describe a process for obtaining translation adequacy, defined here as the methodological goal whereby the quality of the translated text meets the needs of the specified study. Using a subset of responses transcribed from Spanish audiotapes into Spanish text, text was subjected to two separate translation processes, which were compared for adequacy based on error rates and accuracy of meaning as well as cost.

Findings

The process for discriminating translation adequacy was sensitive to differences in certified versus noncertified translators. While the non-certified translation initially appeared to be seven times less expensive than the certified process, auditing and correcting errors in noncertified translations substantially increased cost. No errors were found with the certified translations.

Conclusions

The level of translation adequacy needed for any qualitative study should be considered before beginning the study itself. Based on a predetermined level, translation choices can be assessed using specified methods, which can also lead to greater transparency in the research process.

Clinical Relevance

An ongoing process to verify translation outcomes including cost, a component minimally discussed in the current literature, is relevant to nurses worldwide. Awareness of expense and quality issues makes greater methodological transparency possible in the design of translation projects and research studies.

Regardless of national context, nursing research with non-native-speaking (NNS) populations presents specific challenges. The researcher must, for example, establish trust within minority communities, communicate with participants in their own language(s) either first-hand or through an interpreter, and accurately translate their responses. Today especially, this research must be done within funding and budgetary constraints: In some countries there is no or limited government funding for nursing research, and in the U.S., budgetary increases are minimal (for the year 2013, a 0.01% increase for the NINR over 2012; see AACN, 2012). Not only does the literature contain little information on the process of achieving quality translation and cost containment (Cantor et al., 2005), but at face value, these costs can appear to vary greatly depending on the process. Here, we describe the evaluation of our method for achieving translation quality, comparing accuracy and costs for translating non-native-language interviews into native language text. In the U.S. as in many countries, the NNS segments of society may bear the burden of health care inequality (Pottie et al., 2011). If we are to decrease health care disparities in NNS populations, we must clarify “best practices” in achieving translation quality while simultaneously addressing translation cost.

Translation Cost, Quality, and Adequacy

Cross-cultural translation research comprises many aspects of methodological decision- making surrounding research communication with NNS populations (Harkness, 2011; Poss & Jezewski, 2002). This may include decisions related to achieving cost containment, high quality and translation adequacy within quantitative, qualitative, as well as mixed-methods. Within the scholarly literature on translation issues specifically in qualitative research (e.g., Regmi, Naidoo, & Pilkington, 2010; Squires, 2009), information on the comparative analysis of qualitative data obtained from NNS respondents is limited (Lopez, Figueroa, Connor, & Maliski, 2008; Nikander, 2008). With respect to cost, little has been written about the economic component of translation in cross-cultural qualitative research, although Cantor and colleagues (2005) have provided a useful cost analysis of translating an English-language survey instrument to Spanish. With respect to translation quality in NNS research, as Harkness and Schoua-Glusberg (1998) have said, it is best “to think of assessment in terms of appropriateness or adequacy for a given task” (p. 94). According to Even-Zohar (1975), whose research included translations in 11 languages, “An adequate translation is a translation which realizes in the target language the textual relationships of a source text …” (cited in Toury, 1995, p. 56). Ideally, adequacy must include regional and national colloquialisms. Spanish, for example, has significant country-of- origin differences that must be taken into account (Medrano et al., 2010).

The evaluation process reported here was used to obtain cost containment and translation adequacy, e.g. the ability to meet the needs of the given research tasks within the most economically prudent manner, between Spanish and English text derived from a larger study of disability in Mexican American women as compared with non-Hispanic White (NHW) women. We chose Mexican American women for the study because Spanish-speaking people represent the fastest growing NNS population in the U.S., with 66% of those who are of Hispanic origin being of Mexican descent (U.S. Census Bureau, 2010).

To clarify best practices in achieving cost containment and translation adequacy, the following questions were addressed: (a) What types of errors occur in the translation of Spanish to English interviews? (b) What is the turnaround time for transcription and translation using certified versus noncertified translators? (c) How does the translation process differ between working with certified translators and working with noncertified translators? The certification of translators in the U.S. is conducted through the American Translator’s Association (2009), the only official certification agency. Recognized worldwide, this agency has strict candidate prerequisites for testing and provides experts in the regional nuances of language. Noncertified translation, done without regulated policies surrounding the training of personnel, is often less expensive, more readily available, and less time consuming. Indeed, noncertified translators may even offer “certified translation” simply by going to a notary to internally “certify” that they have done a translation in the past

Ultimately, we posit that costs and results will vary among translators regardless of certification. Due to variation in skill and training, we do not recommend the use of either certified or noncertified translators as a rule. Instead, we recommend the use of an ongoing process for verification of translation results so that best methodological procedures can be followed given the abilities of personnel, the funding available, and the quantity of data.

Background Literature

Internationally, compelling health initiatives and evidence-based practices are leading to dissemination of research methods and findings beyond the English-speaking world. Researchers are calling for high quality translations but there has been little documentation of what it will cost to achieve the adequacy of translation demanded. For instance, the Cochrane Collaboration has published their methodology for translating their review abstracts into Chinese, using noncertified translators. Aiming for translation reliability and readability, the effort required for acceptable translation was evidently high, but there was no mention of their cost (Shen, Lo, Chiu, Chen, & Kuo, 2009). Further, Medrano et al. (2010) reviewed translation methodologies for the translation of Spanish interviews to English reported from 1992 to 2002 in the U.S. publication Hispanic Journal of Behavioral Sciences. Without providing specific data, Medrano et al. noted that as the quality of the translation increased so did the cost.

The uncertainty related to translation cost might deter qualitative researchers who depend on large quantities of text from attempting cross-language studies. Qualitative researchers may prefer a slightly different approach than what is done by quantitative researchers who translate survey items. Chen and Boore (2010) recommend the following process when the qualitative researcher is not fluent in the participant’s language: (a) verbatim transcription in the original language; (b) two bilingual translators to transcribe concepts that emerge; (c) back translation; and (d) an expert committee for final agreement. The use of Brislin’s (1970) back-translation, however, is not a clearly beneficial process when working with the translation of large data sets of qualitative data. The goal of Brislin’s (1970) method of back translation was to “translate their experimental instructions or questionnaires from one language to another in order to test hypotheses about human behavior” (p. 185). The strict application of Brislin’s methods for quantitative surveys, however, is not routine in qualitative work, for good reason: the use of the translations is different.

The goal of the qualitative researcher is multifaceted tridirectional communication. Translation must be accurately conveyed to the participant, with the true meaning obtained from the participant and then accurately conveyed to a larger audience in the final research dissemination language(s). The onus on the qualitative researcher is to obtain in-depth responses in order to best understand an experience or phenomenon such as disability. Hence, recommendations surrounding methods of translation with qualitative data should be made based upon empirical data supported by the study design.

In the present study, a comparative process was used to decide between two methods of translation. Our goals were to choose the translation method that provided cost containment and translation adequacy. First, an adaptation of Brislin’s translation method created for use with qualitative research (Lopez et al., 2008; see Figure 1) was used. At the same time, a certified translator/transcriptionist was employed to translate the Spanish text to English text (Figure 2). The findings are reported here.

Figure 1.

Noncertified Spanish Translation Process

Figure 2.

Certified Translation Process

Methods

Participants in our cross-cultural comparative study of health disparities in disability outcomes in NHW and Mexican American women were 55 to 75 years of age and living in Texas. The total sample for the larger study currently includes 62 NHW women and 60 Mexican American women; 13 of the Mexican American women chose to participate using the Spanish language. Foundational work preceded the interviews including meaningful translation of the interview questions with probes, and verification of language adequacy. Topical biographical interviews were conducted and audiotaped with each participant in a place of their choosing, most frequently their homes. The questions were translated from English to Spanish by the first author, and reviewed by other bilingual research team members. Audio-taped interviews were (a) transcribed verbatim in the language in which they were conducted; (b) then checked (audited) for accuracy by two bilingual research staff members; (c) back translated; and then (d) two bilingual research team members came to a final agreement on translation, comparing the texts with the audio tapes.

Interviews from 8 of the 13 Mexican American women who chose to participate in Spanish were used for the present study; each of the 8 participants was interviewed four times, for a total of 32 interviews. Interviews from the remaining 5 participants in the larger study were not included in this analysis in order to compare the processes and decide on the most appropriate translation method before proceding with translation of the remaining interviews. The education of these 8 women ranged from no formal schooling, to one with a college degree obtained decades before in Mexico. All were low-income to below the poverty level.

For the present study, a professional service that did not use certified translators first transcribed and translated the 32 Spanish interviews. Rather than using certified translators, this service applied their own criteria for testing and evaluating the proficiency of their employees. This was followed by data auditing for transcription errors by our research staff and co-authors. Next, the translations were checked against the Spanish audiotapes by a bilingual research team member whose first language was Spanish. Any errors were noted and then corrected. Transcripts were passed to a second bilingual team member who was a native Texan Mexican American, familiar with the local culture and regional language. Errors were noted, corrected again, and then re-categorized. Time spent checking and correcting translation was noted. The checked transcripts of the 32 Spanish interviews were then sent to a certified translator from the American Translator’s Association. She had lived in both Texas and Mexico, was familiar with both cultures and their colloquialisms, and was proficient in Spanish and English. The certified translations were checked for accuracy once. There were no errors found. Finally, we compared the results for the noncertified and certified Spanish translator services. For a subset consisting of the first 8 interviews (44,648 words), we compared data from the certified and noncertified processes for the time, cost, and quality of transcription and translation.

Time was calculated based upon turnaround time and time spent auditing and correcting. Turnaround time was measured as the total time from sending the audiotapes to the translation service to the return of the translation. The interviews were transferred to and from the services, both certified and noncertified, via a secure server. Research assistants recorded the amount of time spent checking each translation for errors. The total amount of time spent on the translations was calculated. Cost was measured based on the total cost for translating and checking for errors. The amount billed for each interview was documented. Further, the cost to have each interview audited was calculated by multiplying the time spent on each translation per research assistant by his or her wage. Billed translation costs and calculated wages were summed for a total cost. Quality was measured by calculating the number of errors remaining after translation and auditing for errors were complete. Finally, types of errors were measured by performing a content analysis of interview errors for noncertified translations after auditing for accuracy.

Results

The results are organized by cost, quality, time, and procedural issues in Table 1. The results are organized below according to the research questions.

Table 1.

Comparison of Noncertified and Certified Spanish Translation Services, Time, and Cost Following Transcription of a Subset of 8 Interviews (44,648 words)

Transcription: Spanish Audio to Spanish Text
Transcript Turn- around Time	Time Auditing and Correcting	Cost of Service	Cost of Auditing and Correcting	Total Time and Cost
12 days per interview on average	16 hr per interview on average	$591.75	$2,416.19	110 days and $3,007.94
Noncertified Translation: Spanish Text to English Text
29 days per interview on average	31 hr per interview on average	$879.00	$4,651.44	264 days to final product at cost of $5,530.44
Certified Translation: Spanish Audio to English Text
7.5 days per interview on average	0	$6,069.70	0	60 days total to final product at cost of $6,069.70

1. What types of errors are found in the translation of Spanish to English interviews?

Data obtained from noncertified translations were classified into two error categories: significance or non-significance for meaning. After auditing and correction by two separate research assistants, one noncertified translation alone yielded 21 errors related to inaccuracy. Significant errors were often due to omitted words or words rephrased into alternative meanings, such as “very sick” instead of “I had diabetes and osteoporosis,” or “Manhattan” instead of “Mexico.” Nonsignificant errors were recorded as poor wording. A poor wording choice did not alter the meaning. Instead, these were considered errors because the sentence did not provide the most parsimonious communication. For instance, ‘she walked to the store’ is parsimonious. ‘It was that she ventured out towards the place where food items could be purchased’ is not parsimonious and was considered a non-significant error.

2. What is the turnaround time for transcription and translation using certified versus noncertified translators?

Using data from 32 interviews, turnaround time from Spanish audio to Spanish text ranged from 4 to 29 days and averaged 12 days. “Days” were recorded from when the audiotape was submitted to the day it was returned. Time spent checking and correcting the transcribed audio to text data from 32 interviews was 436.95 hrs total, for an average of 6.42 hrs spent auditing and correcting each interview following transcription from audio to text. The first research assistant spent an average of 7.78 hrs on each interview and the second research assistant spent an additional average of 4.86 hrs on each interview in ensuring that the data were transcribed correctly. Hence, once a transcription was received from the service, at least 12 hrs were spent by two separate people together in checking the transcription for errors and making corrections. Using data from 32 completed interviews, the noncertified translators spent from 8 to 56 days to complete the translations, with an average of 25.7 days. This does not include the time required to audit and correct the data. Using data from same 32 interviews, the certified translator took from 3 to 23 days to complete translations, with an average time of 19.5 days.

3. How does the translation process differ between working with certified and working with noncertified translators?

The key difference was in who translated the transcribed Spanish text into English. The noncertified translation process required multiple checks due to inconsistent accuracy, whereas the certified translation was accurate, as verified during the coding process. The total cost of translation for 8 identical interviews or 44,648 words was $5,530.00 for the noncertified service (who charged by the word) with data checking; it was $6,070.00 for the certified service. This includes direct billing from services plus salary costs based on time spent correcting errors. When contracting with the certified translator we knew the individual’s professional and cultural background, compared to not knowing how many different translators were working on translations submitted to the noncertified agency, nor the translators’ backgrounds.

Discussion

Rationales for selecting and evaluating translation methods need to be explicit in publications, especially given the range of procedures reported worldwide. In the present study, tracking procedural issues impacted cost, quality, and time. While at face value the non-certified translation was seven times less expensive than the certified process ($879 compared to $6, 069), the expense of having research personnel audit and correct errors, added $4,651. (Table 1). Although cost was slightly higher for the certified translations, the noncertified translations were laden with errors and took staff away from necessary activities. This is why, following analysis of 32 interviews done with 8 of the 13 participants who chose to be interviewed in Spanish for the larger study, we used only the certified translation service for the remaining 5 participants.

Our process for deciding upon the best method for translation may be beneficial to others designing and conducting research with NNS groups. If our research team had only used and not evaluated the non-certified method described, the study might have been lengthened by as much as 6 months and added significantly to the cost. At the same time, we cannot generalize our findings to all certified and non-certified translators. Our outcomes have limitations grounded in the unique personnel and procedures of our study. For instance, although translators certified through the American Translator’s Association (2009) have standards for credentialing the certified translators, the non-certified translators have varied abilities and expertise. It is possible that a researcher might employ a highly qualified non-certified translator. Another consideration is the design goal of the project. We chose to discuss translation adequacy as an outcome along with cost containment because we understand that different qualitative projects have different text related purposes. Some projects are evaluating text based upon micro-level analyses that consider length of pauses as well as the words. Other researchers are considering overall meaning. This requires different measures of translation adequacy. Further, the difficulty of the audio and written text may vary from study to study requiring different translator skill sets. Unique dialects and speech patterns may be heard best by individuals with matching skill sets. Finally, we allowed the certified and non-certified translators to decide how they preferred to translate. The certified chose to translate from the Spanish text and the non-certified agency preferred to translate from the audio files. This may have impacted translation quality.

Translation is a highly demanding process. Not only does it cost to have staff check translations for accuracy, it also costs to remove them from other important activities. Although we measured the cost of the former, we did not try to calculate the cost of the latter. Even though we worked with reputable services, our requirements for qualitative research had to be communicated so that transcriptions and translations were done verbatim and not corrected to satisfy rules of grammar or other unspecified inclinations. Further, the processes that agencies used to test their employed noncertified translators varied. National organizations offered the best verification of translator abilities, providing trustworthy translators who had references.

As discussed, the procedures for ensuring high-quality translations of qualitative data are not the same as those for quantitative tools, although conceptual understanding (Frederickson, Rivas, Acuña, Whetsell, & Tallier, 2005) and recognition of colloquial language (Li, Liu, Zhang, Wang, & Chen, 2011) are essential to both. Further methods research is needed to understand how to best translate research for meaning in qualitative studies. Creating outcome evaluation goals would be beneficial to those conducting ethnographic as well as other methodologies including phenomenological and narrative research.

In conclusion, researchers must be fully aware of potential cost-to-quality translation discrepancies. They must also include a design process for ongoing evaluation of translation methods chosen; a plan should be in place to correct any difficulties that might occur. Our comparison of the use of noncertified translators with that of certified translators showed that the noncertified service was less accurate with similar costs over time due to the high level of inaccuracies. Having a process in place for ongoing evaluation made it possible to maintain the quality of translation adequacy.

Table 2.

Measures to Consider for Translation Adequacy

Time	Cost	Quality
Transcription turnaround Translation turnaround Hrs spent checking and correcting transcribed text Hrs spent checking and correcting translated text	Transcription services Time spent checking and correcting transcription Noncertified translation Auditing and correcting translation Certified translation	Accuracy of regional and national colloquialisms Type/frequency of errors in transcription Type/frequency of errors in translation