Can a Web-Based Curriculum Improve Students' Knowledge of, and Attitudes About, the Interpreted Medical Interview?

Abstract

Objectives

To develop and evaluate a web-based curriculum to introduce first year medical students to the knowledge and attitudes necessary for working with limited English proficient (LEP) patients through interpreters.

Method

Six hundred and forty first year medical students over 4 consecutive years took this curriculum as part of their Patient Physician and Society course. They viewed 6 patient-physician-interpreter video vignettes, gave open text analyses of each vignette, and compared their responses to those generated by experts, thereby receiving immediate formative feedback. They listened to video commentaries by a cultural expert, lawyer, and ethicist about working with LEP patients, completed pre- and postmodule questionnaires, which tested relevant knowledge and attitudes, and were provided a summative assessment at the end of the module. Students completed an optional survey assessing the educational value of, and providing open text commentary about, the module.

Results

Seventy-one percent (n=456) of first year students who completed the module consented to have their data included in this evaluation. Mean knowledge (19 items) scores improved (46% pre- to 62% postmodule, P<.001), reflecting improvements in knowledge about best interpreter practices and immigration demographics and legal issues. Mean scores on 4 of 5 attitude items improved, reflecting attitudes more consistent with culturally sensitive care of LEP patients. Mean satisfaction with the educational value of the module for 155 students who completed the postmodule survey was 2.9 on a scale of 1 to 4.

Conclusion

Our web-curriculum resulted in short-term improvement in the knowledge and attitudes necessary to interact with LEP patients and interpreters. The interactive format allowed students to receive immediate formative feedback and be cognizant of the challenges and effective strategies in language discordant medical encounters. This is important because studies suggest that the use of these skills in patient encounters leads to greater patient and provider satisfaction and improved health outcomes.

All graduating medical students should be able to deliver high-quality, culturally appropriate care to the more than 21 million people in the United States who report limited English proficiency (LEP).^1,2 An unfunded mandate of Title VI of the Civil Rights Act of 1964 requires that health care organizations that receive federal funding provide LEP patients with interpretation and, therefore, access to medical services equal to that received by English speakers.³ Enforcement of this mandate, along with the rapid increase in the number of LEP patients, has created a huge demand for training health care providers in the use of interpreters. Physicians with prior training report increased use of professional interpreters and greater satisfaction with the medical care provided to LEP patients, when compared with physicians with no prior training.⁴ For these reasons, medical schools must provide students with curricula that emphasize appropriate skills and attitudes to work with lay and trained interpreters, and to effectively conduct an interpreted medical interview.⁵

There is evidence that most U.S. and Canadian medical schools do not provide adequate instruction about cross cultural aspects of patient care, despite the rapid diversification of the populations.⁶ In New York City, 22% of the population speaks limited English. The proportion presenting for care at Bellevue Hospital Center, the public teaching hospital of New York University School of Medicine (NYU SOM), is even higher. Strategies to communicate successfully with LEP patients are not only a prerequisite to conducting culturally sensitive care, but are essential to students' clinical learning. Yet, before 2001, there was little formal curriculum at NYU to prepare medical students to interact with LEP patients. An efficient instructional design was needed to train a large number of preclinical students for this challenging clinical learning environment.

A recent review of the literature found web-based education to be equivalent to other instructional methods in terms of gains in, and satisfaction with, learning,⁷ as long as the best educational principles are applied.⁸ Studies comparing multimedia and traditional educational approaches suggest an improvement in students' performances using multimedia.^9,10 Web-based instruction has the advantages of allowing students to work at their own pace and to participate in interactive learning with immediate feedback and self-assessment.¹¹ Additional advantages include being able to deliver stimulating material to large groups, even when faculty and face time in the curriculum are limited.¹²

There is a small and growing literature documenting that training health care providers in the effective use of medical interpreters will improve the accuracy of communication with, and the clinical care and health outcomes of, their LEP patients.¹³ Drawing on this, as well as on our extensive experience with cross language clinical practice and with interpreter training, we developed, implemented, and evaluated a web-based module on working with interpreters. This module introduces first year medical students to the knowledge and attitudes needed to prepare them to care for, and learn from, patients across language barriers.

METHODS

The Physician Patient and Society (PPS) course at NYU is a 2-year course that introduces students to clinical medicine. Starting in 2001, all first year NYU medical students were required to complete this web-based module as part of the Culture and Diversity unit of PPS. The unit also consists of a lecture on racial disparities in health and health care, and 2 seminars and a writing assignment on culture and health beliefs. The web-based module is the only place in the course where working with an interpreter is addressed.

Curriculum and Module Development

The Interpreter Module Development Team included an Associate Director of the Physician Patient and Society Course (A.L.K.), NYU Project Assistant for the Macy Initiative on Physician Patient Communication (Macy Initiative) (A.L.), the Director and Assistant Director of the Center for Immigrant Health (F.G. and J.C.), and the Director of New York University School of Medicine Advanced Educational Systems (AES) (M.N.). The Macy Initiative has implemented successful clinical clerkship communication skills curricula at 3 medical schools.¹⁴ Since 1990, the Center for Immigrant Health (CIH) has conducted programs on, and studies of, the health care needs of immigrant groups in New York City, including the training of medical interpreters and health care providers.¹⁵ The AES engages in information technology research and development to support the educational mission of the medical school. We developed the content and technical design in parallel, and used a systematic curriculum development and evaluation process.¹⁴

The entire module development process took approximately 1 year. First, a targeted needs assessment was conducted. It included a literature review, and focus groups with, and interviews of, clerkship students. The results generated the goals and objectives for the interpreter module (Table 1). Based on these, clinical scenarios were scripted and filmed with professional actors by a video producer. The scenarios presented common communication challenges faced by primary care physicians working in discordant language encounters (i.e., obtaining an accurate history in the Emergency Department from a patient in pain, discussing symptoms and treatments with a patient who uses herbal remedies, and a physician relying on his own limited proficiency in the patient's language to deliver important information).

Table 1.

Goal and Objectives of the Curriculum

Goal: To introduce first year medical students to the knowledge and attitudes needed to care for patients across language barriers.

Learning Objectives: (cognitive and affective)

After completing this module students will be able to:

• Identify pitfalls of using untrained interpreters like family members, hospital staff and bilingual medical students

• List effective strategies for working with Limited English Proficiency (LEP) patients and interpreters

• Define legal rights of LEP patients

• Recall basic facts about immigration in the United States

• Recognize unique aspects of caring for immigrants that include:

Access to care

Special situations: i.e., victims of domestic violence or refugee status.

• Identify their attitudes and beliefs regarding limited English proficient patients and language discordant medical interviews involving interpreters

The team then developed a storyboard of the module and designed the interactive and assessment elements. Construction of the module required approximately 40 hours of time from an experienced programmer and 30 hours of time from a video and audio editor, including the time to create English subtitles for the video segments. Grant funding was obtained to cover the costs of video production ($20,000). Programming and final implementation and maintenance were supported by the Dean's Office. Faculty time was donated. A number of small pilots of the module were conducted to test for technical “bugs” (α testing) and a second set to assess for user acceptability (β testing). The module resides on a medical school server. Access is authenticated using unique NYU user IDs. Students are notified by e-mail of the module requirement, given the URL link, and continue to receive reminder e-mails until they complete the module. Evaluation of the project was approved by the NYU SOM Institutional Review Board.

Educational Strategy

After responding to an online consent, students begin the module by reviewing the goals and objectives, while listening to a brief audio “voiceover” from a bilingual NYU SOM clerkship student. He describes his experience being involved as an interpreter with an LEP family of a critically ill patient in the surgical intensive care unit. Students then complete the online pretest, consisting of 19 multiple choice question (MCQ) knowledge and 5 Likert-type (5-point scale) attitudes items. Following this, they view a short film of New York City scenes, with a voiceover review of the demographic facts highlighting the magnitude of the language barrier challenge for health care providers. These three elements are designed to create motivation to learn the content of the module by providing relevance and context, and by identifying knowledge deficits.

To achieve cognitive objectives, students first view a brief lecture about effective interpreting practices. They then analyze 6 patient-physician-interpreter video vignettes that highlight common pitfalls and effective strategies for interpreting. Designed to reflect the diverse patient population in NYC, the vignettes present the cases of a Russian speaking middle-aged woman with acute chest pain, a Chinese speaking man with back pain using herbal medication, and a Haitian woman with an abnormal mammogram. In the first 2 cases, an untrained lay interpreter (e.g., family member, medical student, hospital worker) initially attempts to interpret for the patient and physician. Each ad hoc interpretation is then followed by the same interaction with a trained interpreter.

For each vignette, students respond to open-ended questions and submit an open-text analysis of the pitfalls or the effectiveness of the interpretation strategies used. Immediate formative feedback is given by comparing student responses alongside those generated by the authors who are physicians with expertise in this area (F.G., A.L.K., and J.C.) (Fig. 1: Top). This highly interactive element of the module is designed to (a) teach students to critically analyze doctor-patient encounters for the impact of poor interpreter practice on data gathering and rapport building and (b) provide students with a practical model for effective interpreter practice. In addition, this section is aimed at supporting a sense of self-efficacy and at promoting positive attitudes and beliefs towards working with LEP patients.

FIGURE 1.

Demonstration of a comparison of a student's feedback to one generated by an expert.

At any point in the module, students have the option to consult a glossary and list of website links or view brief video commentaries on the cases by well-known experts including a federal civil rights attorney, a medical ethicist, and a social scientist who studies cross cultural communication.

Students conclude the module by taking a posttest consisting of the same knowledge and attitudes items as the pretest. Summative feedback is immediately provided by comparing the students' overall pre- and posttest scores, and by providing correct answers to questions they answered incorrectly on the posttest. On average, it took students 42 minutes to complete the module (range, 31 to 55 minutes).

Finally, the students complete an optional online survey to provide us with feedback about them and about the module, including student country-of-origin, language ability, experience functioning as an interpreter, and what their “take home points” were from the module.

Implementation and Maintenance

Our students are required to own computers with adequate processing speed, broadband connections, and enough memory space to accommodate this technology. Once the module was developed, the cost of maintenance and annual administration became minimal.

Student feedback about the technical aspects of the curriculum, solicited by e-mail and in postcourse debriefing, has been used to make changes in the module each year. The majority of this feedback has related to technical difficulty with video streaming, addressed through an upgrade of the School library's computers. Minor editing of content has been made as well.

Evaluation

To assess achievement of the goals of the module the pre- and posttest were comprised of MCQs measuring knowledge of best interpreter practices and immigration demographics and related legal issues, and attitude items (scored on 5-point strength of agreement scale) originally selected from a larger pool of questions and items developed and refined over many years by the CIH. To better assess the impact of the module on student attitudes we added 3 new attitude questions to the original 2 for the class completing the module in the year 2004.

We used an immediately pretest postmodule design to evaluate our curriculum and answer the following questions:

1. Was there an increase in the scores on an MCQ test assessing knowledge of best interpreter practices, immigration demographics, and legal issues?

2. Was there an improvement in students' attitudes and beliefs regarding LEP patients and working with interpreters?

We also used a postmodule survey to gather students' opinions on the utility and ease of use of the module.

Data Collection

To complete the module, each student was required to register online using a unique identifier. If the identifier matched an entry in an Oracle database (Copyright © 1999–2005, Oracle) table of allowable identifiers, the sign-on was validated. In the database table, the identifier was marked as used and could no longer be a valid identifier for future use. Students submitted responses that were sent to a separate database, which did not have the identifier information, thereby ensuring anonymity.

Data Analysis

For MCQs we calculated a mean of percent correct scores for the pre- and the posttest, and tested statistical significance using a paired Student's t-test. For the 5 Immigration Demographics and Legal issues items, the data were constrained and skewed, prompting us to also conduct a Wilcoxon Signed Ranks Test of the pre–post comparison. We found a similar P value of .014. We, therefore, report the results of the t-test for ease of interpretation. For attitude questions, the responses were on a Likert scale of 1 (strongly disagree) to 5 (strongly agree). We calculated mean pre- and postscores for each attitude question (ordinal scaled and paired), used Wilcoxon Signed Ranks test to compare the ranks of the Likert responses, and tested for significance. Text-based data were initially categorized independently by 3 authors (A.K., S.E.S., and A.L.), and consensus on categories was reached and then summarized.

RESULTS

Subjects

Seventy-one percent (456/640) of first year students in 4 successive years (2001 to 2004) consented to have their data analyzed for this study; 24% (155/640) also took the optional postmodule online survey on the module. Of the first year medical students who took the curriculum, 17% were born outside the United States, 25% spoke a language other than English at home, and 21% have acted as interpreters in a medical setting.

Knowledge and Attitude/Belief scores

Knowledge questions tested demonstrate a significant pre- and postmodule score improvement (42% to 64% correct; P<.001). Questions and scores for item subsets are available online. Table 2 shows results for the 2 attitude items that were measured across all 4 years and the additional 3 items added in the 4th year of this study. The first two items, concerning the health care provider and immigrants living in the United States, were scored such that desirable answers would lead to higher scores. There was a statistically significant improvement in 4 out of 5 attitude and belief questions from pre- to posttest.

Table 2.

Pre- and Postmodule Attitude and Belief Scores (1=Strongly Agree to 5=Strongly Disagree Scale)

Questions	Pre	Post	P value
N=456
Health care provider does not have to be concerned with cultural differences, if a foreign-born patient has lived in the U.S. for a long time	3.60	3.77	<.001
N=456
Immigrants who have lived in the U.S. for >5 y should know how to speak English	2.84	3.10	<.001
N=94
If a patient refuses to see me because of my sex, I would feel slighted or annoyed	2.78	2.94	.01
N=94
When patients nod or say yes even when they don't understand, it makes me feel at a loss, hopeless, stuck	1.70	1.59	.05
N=94
Working with patients who don't speak English is worth the hassle of finding an interpreter because it is so interesting	2.26	2.15	.15

At the end of the module, 155 students (24% of the total) indicated their views of the educational value of the module. The mean satisfaction score with the educational value of the module on a scale of 1 to 4 was 2.9. Eighty-nine students agreed that the module inspired them to explore areas of cultural awareness in medicine. The most common source of dissatisfaction as reported through e-mail and in open-ended comments was the slow video downloading time for students using a dial-up modem in the first year. This was addressed and improved in the subsequent years.

The open-ended comments and feedback were largely positive and reflected an eagerness to master the material. Students specified that the module made them wary of using family members as interpreters, helped them appreciate the impact of immigration status on access to care, and expressed interest in learning more about the various interpreter modalities and hospital policies regarding interpretation.

DISCUSSION

Learning how to work with interpreters is an important component of cultural competency and systems-based practice training that is not typically included in the medical school curriculum.^6,16 A recent report describes a small program at one institution to train bilingual medical students to function as interpreters,¹⁷ and there have been some interventions aimed at improving student and resident competency with cross cultural aspects of patient care.^18–23 However, after an extensive search of the literature, we have not found reports of curricula designed to teach the interpreted medical interview to medical students. Our web module is the first of its kind to prepare students to effectively conduct language discordant medical interviews. It enabled us to provide the curriculum to a large number of students, with minimal resources, before they began clinical training.

We performed a needs assessment, determined our technical resources, secured a commitment from all participating parties, and used appropriate interactive multimedia. Active learning was encouraged through self-assessment, reflection, critique, and immediate formative and summative feedback. Student motivation was ensured by providing an engaging format, and by integrating the web module into the culture and diversity unit of the medical humanities course. The module was evaluated by the students, and the content and links were updated to ensure easier navigation and minimize technical glitches, thus adhering to the best practices of web-based learning design.⁸

Delivering basic material through the web has a number of potential advantages if planned and executed effectively. It is likely to be increasingly used in the future.²⁴ Others have found that web-based learning leads to student satisfaction equivalent to using traditional methods.²⁵ If students are familiar with the material before patient encounters and bedside rounds, there will be more efficient learning. The time saved can be used for further skills and communication training during actual patient encounters.

The module resulted in improved scores on the students' knowledge and attitudes around interacting with LEP patients and interpreters. We feel that this is educationally meaningful, although we cannot determine the clinical significance of this because the curriculum and the assessment were not designed to measure clinical impact. We believe that the knowledge of immigration demographics improved marginally because the educational material was presented as a voiceover film of New York City, instead of engaging in video vignettes, precluding active interaction. According to the students' comments on the postmodule survey, some students were inspired to further explore the legal, ethical, and cultural content provided in the module. They demonstrated that learning on the posttest.

Our expectation is that this module will help students better communicate with patients during their clinical clerkships. Evaluating these future student-patient encounters for the use of the proper techniques would lend predictive validity and further solidify the success of the curriculum. With increasing waves of immigration to the United States, residents and physicians are increasingly caring for LEP patients having received little or no prior training. With little change in the fundamental structure of the module, our curriculum can be disseminated to students in other institutions, and to residents and practicing physicians to address this gap. We believe that the perceived educational value of this curriculum will be much higher for clinical students and residents who frequently conduct language discordant patient interviews.

Our study has certain limitations. Without a control group, we were unable to study the influence of confounding variables, such as a preference for online learning or the cultural diversity of our students on their performance. For motivated students, the delivery format itself, whether a learning module delivered web-based or through a “live” instructor, has little effect on student achievement, as long as the delivery method is appropriate to the content.⁷ Research suggests that good distance learning-teaching techniques are fundamentally identical to good traditional teaching. We believe that the online format was the most appropriate to maximize interactivity and reinforce learning. The cultural diversity of our medical students, some of whom have interpreted in the medical setting, may have influenced student learning and enthusiasm, although it is difficult to predict its impact. The generalizability of our results is limited by the fact that they are based on only the 71% of our students who gave informed consent. In addition, while our knowledge and attitude items have face validity, their concurrent and predictive validity remain to be tested.

The need to train physicians to effectively work with interpreters has been fuelled by the rapid increase in the LEP patients and a federal mandate in the year 2000 to enforce the civil rights act of 1964. This web-based curriculum resulted in short-term improvement in knowledge and attitudes, a prerequisite for optimal language discordant patient-physician communication, a crucial component of quality health care.²⁶