Accuracy of Computer-Generated, Spanish-Language Medicine Labels

Abstract

OBJECTIVE

We evaluated the accuracy of translated, Spanish-language medicine labels among pharmacies in a borough with a large Spanish-speaking population.

METHODS

A cross-sectional, telephone survey of all pharmacies in the Bronx, New York, was performed. Selected pharmacies were visited to learn about the computer software being used to generate Spanish medicine labels. Outcomes included the proportion of pharmacies providing Spanish medicine labels, frequency of computerized translation, and description of Spanish medicine labels produced.

RESULTS

Of 316 pharmacies, 286 (91%) participated. Overall, 209 (73%) provided medicine labels in Spanish. Independent pharmacies were significantly more likely to provide Spanish labels than were hospital or chain pharmacies (88% vs 57% vs 32%; P < .0001). Pharmacies that provided Spanish labels mostly commonly (86%) used computer programs to do so; 11% used lay staff members, and 3% used a professional interpreter. We identified 14 different computer programs used to generate Spanish labels, with 70% of pharmacies using 1 of 3 major programs. We evaluated 76 medicine labels generated by 13 different computer programs. Overall, 32 Spanish labels (43%) included incomplete translations (a mixture of English and Spanish), and 6 additional labels contained misspellings or grammar errors, which resulted in an overall error rate of 50%.

CONCLUSIONS

Although pharmacies were likely to provide medicine labels translated into Spanish, the quality of the translations was inconsistent and potentially hazardous. Unless regulations and funding support the technological advances needed to ensure the safety of such labeling, we risk perpetuating health disparities for populations with limited English proficiency.

In August 2000, President Bill Clinton signed Executive Order 13166, Improving Access to Services for Persons with Limited English Proficiency (LEP), to enforce Title VI of the Civil Rights Act of 1964, which specifies that public funds cannot be used in ways that discriminate against persons according to racial or national origin. Executive Order 13166 specifies that agencies receiving federal funding (for example, Medicaid and Medicare) must strive to “develop and implement a system to provide services so that LEP persons can have meaningful access to them.”¹

Several studies in the past few years have documented gaps in access to appropriate language services for individuals with LEP.^2–5 In New York City, 38.6% of 200 randomly selected pharmacies that served clients with LEP on a daily basis translated prescription labels every day.² In a study of 764 pharmacies in Colorado, Georgia, North Carolina, and Texas, 34.9% of pharmacies reported being unable to provide any translated instructions for medicines.⁴

One major mechanism that pharmacies use to provide translation of medicine labels is computerized translation software. In a 2004 survey of all pharmacies (99.4% participation) in the Bronx, New York, a county in which 44% of residents are Spanish-speaking, we documented that 69% of pharmacies could provide medicine labels in Spanish, most commonly (86%) by using computerized translation software.³ Anecdotal remarks made by pharmacists in that study raised questions about the quality of computer-translated medicine labels.³ In 2006, a survey led by the New York Academy of Medicine revealed that 75% of pharmacies in New York City reported that they could use computerized translation software to provide medicine labels in ≥1 language.² When pharmacists were questioned regarding barriers to providing translated medicine labels, 5% reported concern about the accuracy of the translated information. No studies have evaluated systematically the accuracy of computer-generated, non-English medicine labels.

New York City, where 47.6% of the population speaks a language other than English at home,⁶ is at the center of the struggles to meet Executive Order 13166. In 2007, a complaint was filed with the New York State Attorney General, charging pharmacies with bias for not providing adequate translation services to customers with LEP.⁷ After conducting its own investigations, the Office of the Attorney General secured agreements with major pharmacies in New York to provide counseling to customers with LEP in their own languages and to provide written translations of prescriptions in the predominant languages in New York.^8,9 On September 3, 2009, Mayor Michael Bloomberg signed legislation requiring chain pharmacies in New York City to provide language assistance services to clients, including written translations of medicine labels for customers speaking 1 of the top 7 foreign languages in New York City (Spanish, Chinese/Cantonese/Mandarin, Russian, Korean, Italian, French Creole, and Bengali).^10,11 In this article, we report on the availability of Spanish-language prescription labels in the Bronx, New York, during autumn 2007. We also report on the accuracy of computer-generated, Spanish medicine labels and discuss the findings in the context of current legal mandates.

METHODS

Study Design

We conducted a cross-sectional survey of all pharmacies in the Bronx, New York, in September 2007. First, we used the New York State Education Department Office of the Professions Web site to retrieve a list of all pharmacies in the county. Then, we used Yellow-Pages.com, Verizon Superpages.com, and Mapquest.com to obtain a telephone number for each pharmacy on the list. A trained research assistant called each pharmacy and asked to speak with the pharmacist. If the pharmacist was not available, then the interview was conducted with the pharmacy technician, store manager, or clerk. No personal identifying information was collected from any individual interviewed. The study was reviewed and classified as exempt research by the institutional review board of Montefiore Medical Center.

The survey consisted of 7 questions. The first question asked whether the pharmacy was able to provide prescription labels in Spanish. If the pharmacist answered no, then the survey was finished. If the answer was yes, then we asked how long the pharmacy had been able to provide Spanish prescription labels, the methods used to provide the labels, whether the pharmacist spoke Spanish, how frequently prescription labels were translated into Spanish (every day, at least once per week, less than once per week, or never), how frequently the pharmacy served Spanish-speaking customers (every day, at least once per week, less than once per week, or never), and how frequently the pharmacy encountered a request for a Spanish prescription label (every day, at least once per week, less than once per week, or never). For pharmacies that used a computer program to translate prescriptions, we also asked whether translations were checked by any Spanish-speaking personnel. Finally, we recorded the name of the computer software used and the pharmacy’s address.

After surveying all of the pharmacies, the research assistant visited a convenience sample of representative pharmacies to learn more about the computer software being used to translate prescriptions. Pharmacies to be visited were selected so that the research assistant could visit multiple pharmacies in a geographic area that used different software. For each computerized translation program identified in the survey, 2 or 3 pharmacies that used that program were visited. At each pharmacy, the pharmacist was presented with 4 different prescriptions written for a fictional patient (Table 1). These prescriptions were generated by using the computerized physician order entry software of Montefiore Hospital (HP IDX Carecast [Hewlett-Packard Development Company, LP, Palo Alto, CA]). Pharmacists were told that they were participating in a study to understand the accuracy of translated medicine labels and were asked to produce the medicine labels in Spanish as they normally would for a customer. Pharmacists were shown a copy of the institutional review board letter classifying the study as exempt research and were told that we were studying the quality of the computerized translation program and not the services of the pharmacy itself.

TABLE 1.

Examples of Errors in Computer-Generated Spanish Medicine Labels (N = 76) for Prescriptions Presented to 22 Pharmacies, Compared With English Back-translations

Spanish Medicine Label	Back-translation to English
Prescription 1: ferrous sulfate (15 mg/0.6 mL), 0.6 mL administered orally twice per day; give with juice
Toma 0.6 mL dos veces al dia por la poca con jugo	Taking 0.6 mL 2 times to the day by the little with juice
Tome 0.6 mL por la boca dos veces al dia (give con juice)	Tome 0.6 milliliter by the mouth twice to the day (give with juice)
Tome 0.6 milliliters por la boca dos veces al dia with juice	Tome 0.6 milliliters by the mouth twice to the day with juice
Give por la boca 0.6 mL dos veces al dia with juice	Give by the mouth 0.6 milliliter twice to the day with juice
0.6 mL boca dos veses aldia	0.6 mL mouth two kiss aldia
Prescription 2: ferrous sulfate (15 mg/0.6 mL), 1.2 mL administered orally each day; give 2 dropperfuls with juice once per day
Take 1.2 aldia give dropperfuls with juice once a day	Take 1.2 aldia give dropperfuls with juice eleven to day
Give mL [dos (2) dropperfuls, 0.6 mL each] with juice diariamente for 30 days	Give mL [dos (2) dropperfuls 0.6 mL each] with juice diariamente for 30 days
Usar 2 goteros entero con, jugo una ves al, dia por 30 dias	To use 2 goteros whole number with juice one you see, the day by 30 days
Give 1.2 mL tome por la boca al dia; give 2 dropperfuls with juice once a day	Give 1.2 mL tome by the mouth to the day, give 2 dropperfuls with juice eleven to day
Tome 1.2 mL por la boca una veces al dia (give 2 dropperful con juice once a day)	Tome 1.2 mL by mouth times to the day (give 2 dropperful with juice eleven to day)
Give tome 1.2 mL por la boca once daily dos dropperfuls with juice once a day	Give tome 1.2 milliliter by two eleven mouth daily dropperfuls with juice eleven to day
Prescription 3: Bactroban cream (2%), 1 topical application twice per day; take for 7 d
Apply topically 2 veces al dia por 7 days	Apply topically 2 times to the day by 7 days
Apply to affected areas dos veces al dia como indicado	Apply to affected area twice to the indicated day like
Aplique to affected areas dos veces al dia for 7 days; for external use only	Apply to affected area twice to for day 7 days; for external uses only
Prescription 4: amoxicillin (400 mg/5 mL), 7.5 mL administered orally twice per day; take for 10 d
Give 1–1/2 cucharadita (S) dos veces al dia for 10 halfteas	Give 1–1/2 teaspoon (S) twice to for day 10 days
Tome one y a half teaspoons por la boca twice a dia para 10 days	Tome one and to half teaspoons by the mouth twice to day for 10 days

Not all pharmacies agreed to generate all 4 prescription labels.

Statistical Considerations

Survey data were analyzed by using descriptive statistics. We coded each pharmacy type (chain, hospital, or independent), and χ² analysis was used to compare the provision of Spanish medicine labels according to type of pharmacy. Translated medicine labels obtained from the pharmacy visits were evaluated objectively by a bilingual physician investigator (Dr Sharif) for the presence of English words in the Spanish medicine labels. The proportion of medicine labels that contained English words was calculated. Then we used the Microsoft Office (Microsoft, Redmond, WA) translation program, which uses the WordLingo language translation service (WordLingo, Las Vegas, NV), to back-translate the Spanish medicine labels into English. The back-translated medicine labels were evaluated by the physician researcher (Dr Sharif) for adherence to the initial prescription instructions.

RESULTS

Availability of Spanish Medicine Labels

Of the 316 pharmacies identified in the database, 286 (91%) participated in the telephone survey. Of those, 207 were independent pharmacies, 71 were chain stores, and 8 were hospital pharmacies. Respondents included pharmacists (86%), pharmacy technicians (10%), managers (3%), and other staff members (1%). Independent pharmacies were more likely than chain or hospital pharmacies to encounter Spanish-speaking customers on a daily basis (88% vs 64% vs 55%; P < 0001).

Overall, 209 pharmacies (73%) reported that they could provide medicine labels in Spanish. Independent pharmacies were significantly more likely to provide Spanish labels than were hospital or chain pharmacies (88% vs 57% vs 32%; P < .0001). Of the pharmacies that translated labels into Spanish, 72% reported that they did so daily, whereas 96% reported that they had Spanish-speaking customers on a daily basis. Of the pharmacies that produced Spanish medicine labels, 43% reported that they received requests for Spanish labels many times; the rest received such requests only occasionally (40%), rarely (15%), or never (2%).

Methods Used to Produce Spanish Medicine Labels

Of the 209 pharmacies that provided Spanish medicine labels, most (86%) used a computer program to translate prescription instructions into a Spanish medicine label; 11% used lay staff members, and 3% used professional interpreters. We identified a total of 14 different computer programs that were used to translate prescriptions into Spanish medicine labels; however, 70% of pharmacies used 1 of 3 programs, namely, Micro Merchant (Micro Merchant Systems, Jericho, NY), OPUS-ISM (OPUS-ISM, LLC, New York, NY), and BestRx (Best Computer Systems, Bloomingdale, IL). All 4 hospital pharmacies that provided Spanish labels used computerized translation software. There was a trend toward independent pharmacies being more likely than chain stores to use computer software to produce Spanish labels (87% vs 74%; P =.10).

Of the pharmacies that used computer programs to translate prescriptions, 170 (95%) reported that they had someone check the printout for accuracy. Persons who checked translations included lay staff members (56%), pharmacy technicians (34%), pharmacists (8%), and medical interpreters (1%). Only 5 of the 14 pharmacists who checked computer-generated labels for accuracy reported being fluent in Spanish.

Quality of Computer-Generated, Spanish Medicine Labels

All 22 pharmacies that we visited agreed to produce translated medicine labels, although some pharmacies did not want to produce all 4 labels, citing time pressures. We collected 76 medicine labels generated by 13 different computer programs. Table 1 describes the set of computer-generated, Spanish medicine labels obtained from the 22 pharmacies we visited. Overall, 32 Spanish medicine labels (42%) included incomplete translations (a mixture of English and Spanish). An additional 6 labels contained significant spelling/grammar errors, which resulted in an overall error rate of 50%. Phrases that were not translated included “dropperfuls,” “apply topically,” “for 7 days,” “for 30 days,” “apply to affected areas,” “with juice,” “take with food,” and “once a day.” Table 1 also presents the back-translation into English of the Spanish medicine labels; important deviations from the intended instructions are evident. Deviations included errors such as “by the little” for the translation of the misspelled Spanish word poca (for boca, meaning “mouth”), “two kiss” for the translation of the misspelled Spanish phrase dos veses (for dos veces, meaning “two times”), and “eleven times a day” for the translation of the phrase “once a day.”

DISCUSSION

Although pharmacies in this study were likely to provide medicine labels translated into Spanish, the quality of the translations was inconsistent and potentially hazardous. Similar to previous reports,^2–5 the majority of pharmacies that provided translated medicine labels used computer programs to do so. There are no previous reports of the quality of translated labels with which to compare our findings.

This study had several limitations. First, because the pharmacists who participated in the survey were aware that they were participating in a research study led by a prominent academic institution in their neighborhoods, they might have tried harder to provide Spanish medicine labels. Conversely, because they knew that the request for the label was for research purposes and not for a real patient, they might not have tried hard enough. Therefore, the labels we evaluated in this study might be of either higher or lower quality than the ones usually presented to pharmacy clients. Second, the prescription labels that we presented to the pharmacies might have been unclear to begin with, which resulted in poorer ability to translate. However, we did use a standard computerized order entry system, which was used by the one academic medical center in the borough, to generate the prescriptions. One study of pharmacy interpretation of English prescriptions documented variability in the production of English medicine labels.¹² This issue brings up the potential for compounded errors in the interpretation of a prescription and then translation into Spanish.

We also considered that our use of the Microsoft Office translation program to back-translate Spanish medicine labels might be a limitation, because we have shown in this study that translation software is inadequate for translating medication instructions for patients. However, the inadequacies we found were attributable not to the translation per se but to incomplete translation resulting from missing terms in the software database, which caused lay personnel (who often were not fluent in Spanish) to enter information manually into the database. In response to suggestions from reviewers of an earlier version of this manuscript, we obtained back-translations from 4 native Spanish-speaking, bilingual individuals (data available from the authors on request). In every case, the translators provided interpretations of the medicine labels by using their knowledge of English to come up with correct or nearly correct instructions. Therefore, we think that using bilingual interpreters to back-translate the labels would be misleading for the purposes of this study. Our interest in back-translation of the medicine labels was to demonstrate how a monolingual Spanish-speaking person would interpret the labels. For the purposes of the current study, we think that use of a computerized translation program was the best method to reflect how the labels would appear to the population for which they were intended. Ultimately, the most important information would be gained by presenting the labels to a large, representative sample of monolingual, Spanish-speaking, pharmacy clients, to determine how they would interpret the labels; such a survey would be an important follow-up study. Regardless of the validity of the back-translations, the key finding in our report is that 42% of labels included incomplete translations (a mixture of English and Spanish).

Because of the high response rate in our survey, we are confident that we have represented adequately the current availability of Spanish language medicine labels in the Bronx, New York. By targeting the major computer programs used in this area, we also have a good objective evaluation of the current capabilities of computerized translation programs used in practice. Visits to pharmacies in the course of this study provided unique insights that could not have been achieved through written or telephone surveys alone. In observing the process of obtaining a translated medicine label, we noted several key points. Many of the computer programs used to produce translated labels work in similar ways and produced common errors. For example, each computer program contains a database of Spanish words that can be used to translate instructions. However, we learned through our interactions that the databases often are incomplete or even may contain errors in spelling or grammar. Translations for certain words used commonly in English medication instructions (eg, “dropperful”) were not present in any database and had to be entered manually by the pharmacy technician or pharmacist.

The potential consequences of leaving untranslated English words and phrases in medicine labels are immediately apparent and frightening to any physician. The mixture of English and Spanish words in the “translated” labels poses obvious opportunities for harm. For example, the translations for the second prescription, in which English and Spanish words were mixed, could be misunderstood so that iron would be dosed 11 times per day, rather than 1 time (once) per day (Table 1). An error in interpretation of this kind could be life-threatening, as has been demonstrated in at least one real-life anecdote, in which a patient took 11 pills each of his β-adrenergic receptor blocker and diuretic daily.¹³ The impact of misspelled Spanish words is evident, for example, when we look at the word vez, which in Spanish means “time.” If this is misspelled as ves (which is phonetically correct for a native English speaker), then the word in Spanish means “you see.” The plural form of vez is veces; if this is misspelled as veses, then the word means “you kiss.”

Pressures to improve the availability of Spanish medicine labels, such as Mayor Bloomberg’s signing of the Language Access in Pharmacies Act, serve as critical steps in reducing ethnic and racial disparities in health. However, the findings from this study stress the importance of a multifactorial response to the identified need to serve populations with LEP. Regulations stipulating that written medicine labels in a variety of languages must be made available must be evaluated in the context of the technological capabilities of the pharmacy industry. Unless regulations and funding support the technological advances needed to ensure the safety of such labeling, we risk perpetuating health disparities for populations with LEP.

We found that independent pharmacies were most likely to serve clients with LEP on a regular basis, were most likely to provide Spanish medicine labels, and were more likely than chain pharmacies to use computerized translation software for this purpose. We speculate that independent pharmacies also are more likely to have limited resources for professional interpretation services to supplement the computerized translation software.

Research that evaluates the impact of initiatives to improve communication with individuals with LEP is important as we progress along this path. A recent study evaluating the new labeling initiatives at Target pharmacies demonstrated no impact on client health service use.¹⁴ Such studies are important as policymakers make decisions regarding health care spending and investment in technology and resources.

In this study, we evaluated only Spanish medicine labels; however, the law just signed in New York provides for medicine labels to be produced in 6 other, commonly spoken languages. Because Spanish is probably the easiest to translate in the health care industry, it is worrisome to consider what the status of translation for other languages might be. Studies that evaluate formally the quality of those translations will need to be performed.

We suggest that the use of professional interpreters, at least for the most-commonly encountered local languages, in pharmacy settings is vital to ensuring that pharmacies are able to meet not only the letter of the law but also its spirit. All translated computer labels should be reviewed by a professional interpreter before being given to a customer; telephone interpreters should be used when live interpreters are not available. We recommend that pharmacies that use any form of translation put quality-assurance programs in place to evaluate the quality of their translated medicine labels. To be effective, initiatives to reduce disparities in health could benefit greatly from collaboration between the technology industry, researchers, physicians, pharmacists, and policymakers, as well as direct representation of the communities served.

WHAT’S KNOWN ON THIS SUBJECT.

Pharmacies use computerized translation software to generate Spanish medicine labels to meet the language needs of their clients.

WHAT THIS STUDY ADDS.

The quality of computer-generated, Spanish medicine labels is inconsistent and potentially hazardous.

ABBREVIATION

LEP

limited English proficiency