Abstract
Background
Cross-cultural communication in primary care is often difficult, leading to unsatisfactory, substandard care. Supportive evidence-based guidelines and training initiatives (G/TIs) exist to enhance cross cultural communication but their use in practice is sporadic. The objective of this paper is to elucidate how migrants and other stakeholders can adapt, introduce and evaluate such G/TIs in daily clinical practice.
Methods
We undertook linked qualitative case studies to implement G/TIs focused on enhancing cross cultural communication in primary care, in five European countries. We combined Normalisation Process Theory (NPT) as an analytical framework, with Participatory Learning and Action (PLA) as the research method to engage migrants, primary healthcare providers and other stakeholders. Across all five sites, 66 stakeholders participated in 62 PLA-style focus groups over a 19 month period, and took part in activities to adapt, introduce, and evaluate the G/TIs. Data, including transcripts of group meetings and researchers’ fieldwork reports, were coded and thematically analysed by each team using NPT.
Results
In all settings, engaging migrants and other stakeholders was challenging but feasible. Stakeholders made significant adaptations to the G/TIs to fit their local context, for example, changing the focus of a G/TI from palliative care to mental health; or altering the target audience from General Practitioners (GPs) to the wider multidisciplinary team. They also progressed plans to deliver them in routine practice, for example liaising with GP practices regarding timing and location of training sessions and to evaluate their impact. All stakeholders reported benefits of the implemented G/TIs in daily practice. Training primary care teams (clinicians and administrators) resulted in a more tolerant attitude and more effective communication, with better focus on migrants’ needs. Implementation of interpreter services was difficult mainly because of financial and other resource constraints. However, when used, migrants were more likely to trust the GP’s diagnoses and GPs reported a clearer understanding of migrants’ symptoms.
Conclusions
Migrants, primary care providers and other key stakeholders can work effectively together to adapt and implement G/TIs to improve communication in cross-cultural consultations, and enhance understanding and trust between GPs and migrant patients.
Keywords: Primary Health Care, Transients and Migrants, General Practice, Community-Based Participatory Research, Cross-cultural communication, Equity
Background
Effective communication is at the core of person-centred care. In situations where patients and doctors do not share language or culture even basic communication can become problematic, with detrimental effects on access, outcomes and safety [1–3]. With migration on the rise globally, problems with cross-cultural communication are increasingly common in primary care, leaving migrants vulnerable to sub-standard care [4–6] and healthcare providers unsatisfied with the quality of care they deliver [7–9]. Everyone has a fundamental right to health and to access health care, legally enshrined in both international and European instruments, such as the European Charter of Fundamental Rights [10]. Therefore, this form of health inequity urgently requires attention.
Primary care interventions can improve healthcare access, quality, health outcome and equity in health [11]. There are also guidelines and training initiatives (G/TIs) available that are designed to support communication in primary care cross cultural consultations, for example as developed in the EU funded MEMP-TI project (http://www.mem-tp.org/.] However, like many other clinical guidelines [12, 13] their implementation in routine practice remains patchy [14, 15]. One of the reasons being the fact that these G/TIs are often not well tailored to daily practice, and stakeholders (patient service users as well as primary care providers are often not involved in their development. Consequently, there is a reliance on informal strategies rather than the use of professional trained interpreters by appropriately trained healthcare providers: this knowledge-practice gap is under researched [15–18]. Raising the voices of marginalised communities is recognised as a key lever for effective intervention design, implementation and evaluation in primary care [19, 20]. Many national health policies are explicit about the value of community participation for practice improvement in primary care [21–23]. However, the involvement of migrants is rare as they are considered ‘hard-to-reach’ on the basis of inaccessibility, language discordance and cultural difference [24] and there are specific challenges involving undocumented migrants in health research [25]. Recent research provides evidence of the value of participatory methodologies to engage migrants and indicates the positive impact of their involvement on the development of guidelines to improve communication in cross-cultural consultations [14, 24]. Taken together, this shows that it is imperative to involve migrants in the implementation and evaluation of available G/TIs in primary care to improve communication in cross-cultural consultations. The need to address this has become a pressing issue following the vulnerabilities of the recent large influx into Europe of those who have been forcibly displaced [26].
The objective of this paper is to elucidate how migrants and other stakeholders can adapt, introduce and evaluate their selected G/TIs in daily clinical practice.
Methods
Study Design and setting
The RESTORE project was a qualitative, comparative case-study [27] that prospectively investigated and supported the implementation of G/TIs to improve communication for migrants in five primary care settings (Austria, England, Greece, Ireland and the Netherlands), using a novel combination of Normalisation Process Theory (NPT) and Participatory Learning and Action (PLA) methodology [16]. Policy analysis work was undertaken in a sixth country – Scotland.
A detailed description of the study protocol is available [28].
NPT is a theoretical framework concerned with the work that individuals and organisations have to carry out in order to embed and normalise new, complex ways of working into routine practice [29]. It has been widely used to guide the implementation of a variety of system improvements in primary care practice, [29] and alerts researchers and implementers to the realities of implementation in real time and the interactions that do, or do not, occur between the individuals and groups charged with that implementation, by focussing attention on four principal constructs (Table 1).
Table 1.
Construct | What it addresses |
---|---|
Sense-making | Can those involved in the implementation make sense of it? |
Cognitive Participation | Do relevant stakeholders ‘buy into’ the implementation work? Can those involved maintain their involvement and get others involved and engaged? |
Enacting | What has to be done to make the intervention being implemented work in routine practice? |
Appraisal work | How can the intervention be monitored and evaluated? Can it be re-designed to sustain its use? |
PLA creates a participatory space where stakeholders can work together democratically [25] on an implementation project. Our rationale for combining NPT and PLA is available elsewhere [30].
A comprehensive mapping process [14] in the six RESTORE countries identified twenty G/TIs that were designed to support communication in cross-cultural consultations. Research teams in five countries (Austria, England, Ireland, the Netherlands, and Greece) used NPT to appraise these G/TIs to create a smaller set of four or five G/TIs, selected for their likelihood of successful implementation in primary care in their countries [14]. Migrants and other key stakeholders in each setting were then invited to examine the set of G/TIs relevant to their setting and they successfully selected one as an implementation project for a primary care site in their region [31]. This paper focuses on the next stage of RESTORE: stakeholders’ participation in adaptation, delivery and evaluation of the selected G/TIs.
Sampling and recruitment
Using purposive and network sampling [32], we recruited 66 stakeholders across five study sites to participate in adaptation, delivery and evaluation of the selected G/TIs, including migrant representatives, general practitioners (GPs), practice nurses, receptionists, practice assistants, practice managers, academics, interpreters, health service planners and policy makers (See Table 2). In each group diversity was reached regarding age and gender; migrants came from a large variety of different countries. Of these, 45 had been involved in the prior G/TI selection. All stakeholders had a good command of the national language at each fieldwork site.
Table 2.
Country | |||||
---|---|---|---|---|---|
Austria1 | England | Greece | Ireland | Netherlands | |
Migrants/migrant representatives | 3 | 5 | 2 | 5 | 3 |
General practitioners | 4 | 2 | 4 | 2 | 2 |
Primary care nurses | 0 | 0 | 5 | 0 | 3 |
Primary care administrators | 0 | 1 | 1 | 2 | 2 |
Interpreters/cultural mediators | 0 | 0 | 0 | 3 | 1 |
Health service planners/policy makers/academics | 1 | 2 | 7 | 1 | 1 |
Trainers | 2 | 2 | 0 | 0 | 0 |
TOTAL | 10 | 12 | 19 | 13 | 12 |
1In Austria, there were also three focus groups with migrants from Philippines, Turkey, and sub-Saharan Africa (n = 30) after the selection of a G TI was made
A commitment to engage multiple stakeholders in a meaningful participatory dialogue was at the core of the RESTORE project methodology. Overall this was successful as evidenced by the details of our sample. However there were challenges with various stakeholders in each setting which had to be addressed (see Table 3). Interestingly, dealing with these challenges was greatly helped by stakeholder engagement because stakeholders could draw on their knowledge of their own backgrounds (healthcare, community etc.) to help identify potential solutions to overcome challenges.
Table 3.
Engaging migrant communities: |
• Austria: GPs work in single handed practices without migrant patient representative groups. The research team fostered dialogue between academically-oriented stakeholders from primary care and migrant representatives. |
• Greece & the Netherlands: GPs engaged with migrants at project meetings in general but expressed discomfort about involving migrant representatives directly in discussions about the practice. Research teams addressed this by engaging in parallel dialogues about these issues with migrant representatives and shared the information across the groups. |
Engaging primary care staff: |
• England: Restructuring of primary care made it difficult to involve a GP practice in the early stages of fieldwork. GP members of the stakeholder group offered their perspectives until a primary care team agreed to participate. |
• Greece & the Netherlands: Healthcare staff found it difficult to attend long PLA focus groups. In the Netherlands research teams introduced shorter sessions; in Greece they met individually with practice staff. |
• Ireland: Poor engagement of some GPs and administrators in the participating practice was offset by sustained commitment by the principal GP and practice manager. |
Engaging interpreters: |
• Greece: No formal primary care interpreting service existed. The research team made innovative arrangements with an NGO and a certified interpreter to negotiate telephone-based interpretation services for primary care patients. |
• Ireland: There was no national interpreting service with trained interpreters. This was resolved by exploring and drawing on expertise of trained community interpreters within the stakeholder group. |
Engaging policy makers and health service planners: |
• England: Restructuring of primary care due to policy changes meant that a key policymaker stakeholder was moved to a different job. The stakeholder group brought in new policymakers/health service planners at a later stage of the project. |
Data generation and analysis
Data were generated and analysed throughout the process of implementation, which lasted between 15 and 19 months after stakeholders had selected a G/TI for their local setting. We used specific PLA techniques for data-generation in PLA informed focus groups. These focus groups consisted each time of the same stakeholders, as described in Table 2 (with minor variations depending on availability), to establish continuity in the adaptation, implementation and evaluation process. PLA techniques used included flexible brainstorming, direct ranking, card sort, seasonal calendar and speed evaluation (see Table 4). These have been used in primary care research previously, with the order they were used in designed to facilitate discussion and decision-making [24, 33].
Table 4.
Flexible Brainstorming | Fast and creative approach of using materials, such as pictures or objects, to generate information and ideas about the topic. |
Direct Ranking | A transparent and democratic process that enables a group of stakeholders to indicate priorities or preferences. |
Card sort | An interactive method for facilitating and recording brain storming around topics. |
Seasonal calendar | Seasonal Calendar is a grid-based diagram used for co-operative planning and democratic decision-making. A flexible adaptive tool, it can be used as a ‘running record’ of stakeholder’s planning over time. |
Speed evaluation | Speed evaluations are short verbal or written evaluations, often used at the end of a PLA session to indicate (to stakeholders and researchers alike) what key positive, negative and/or neutral experiences have occurred. |
The data generated were then used to document details of stakeholders’ adaptation of their G/TIs, their planning to deliver them in daily practice and to evaluate their impact.
All researchers had prior experience of qualitative research and were trained in the use of PLA techniques [34]. A total of sixty two PLA style group meetings were held, with an average of twelve in each setting. Each meeting was audio-recorded and transcribed verbatim.
These data were subjected to a deductive qualitative analysis using an iteratively developed NPT coding frame (Table 5). At all sites at least 10% of all data were double coded with active attention for data that fell outside NPT. This paper focusses principally on two constructs of NPT: enacting (delivery in practice) and appraisal (evaluation) work and describes the results, challenges and solutions, according to the different stages of work: the engaging of stakeholders; the adaptation of the G/TIs to the local context; the delivering of the G/TIs in practice; and the evaluation of the implementation.
Table 5.
NPT construct | Sense-making | Cognitive Participation | Enacting | Appraisal work |
---|---|---|---|---|
Descriptor | Stakeholders making sense of the G TIs presented to them Which ones do they think have value and benefit for their local setting? What are the positive and negative features of the G/TIs they examined? Were there shared views or not during this phase of thinking about the G/TIs? |
Stakeholders’ engagement with the implementation project, process of direct ranking and selecting one G/TI to Process of selecting a G/TI reveals data about their motivation, willingness and perceived capacity to implement the G or TI, plus their feelings that it is right to be involved in the implementation work. Stakeholders thinking about other relevant stakeholders to enrol e.g., migrants, other practice staff members, trainers is part of the work of driving it forward Active enrolment work e.g., meeting someone to recruit them to the project |
Stakeholders activities to introduce G Tis into clinical settings Planning imminent activities e.g. discussing divisions of labour about who makes a poster Information on what resources are available in their context to them to support these activities Stakeholders’ skills and participation in training to develop their skills Stakeholders fine tune their selected GTI to improve its quality or relevance for them Stakeholders implement knowledge from their selected G Tis into practice Divisions of labour to progress the implementation project e.g. who does what to enrol other stakeholders, to ensure smooth running of an interpreted consultation Descriptions of trying out new ways of working/communication with migrants and how it felt to apply the new way of working in e.g., consultations – did stakeholders have confidence in it? Data about the length of consultations and whether the new ways of working had positive or negative effects on consultations in practice |
Stakeholders’ appraising the impact of the new way of working preferably after a period of use Stakeholders’ discussions about planning formal appraisals e.g. setting up exit interviews or patient questionnaires to monitor impact; collecting data on number of interpreted consultations Stakeholders’ reflections on findings from these appraisals, informal appraisals also relevant Can the new way of working be sustained in practice and what changes do stakeholders think are necessary to the service to ensure this? |
In addition to the transcript data, each research team completed 5 fieldwork reports with standardised headings in English, that were created to monitor fieldwork and to facilitate the comparative analysis. These reports contained rich narrative descriptions of each of the implementation processes. The content of the reports was derived from a variety of sources, including PLA style focus groups, transcripts and data displays, team meeting minutes and fieldwork debriefings. The monitoring reports were discussed at RESTORE team meetings, encouraging a process of iterative reflection and cross-country comparison and exchange. Issues around the coding procedure and coding frame were discussed across the RESTORE teams, both face to face and by teleconference. These combined activities were important for contextualising and synthesising knowledge of the transcript data from across settings.
Results
Adapting G/TI to local context
The process of adaptation was more substantial and far-reaching than we anticipated, especially in Austria (TI) and England (TI). Table 6 charts the changes that stakeholders made to the selected G/TIs in each setting.
Table 6.
Original G/TI | Adapted G/TI |
---|---|
Austria | |
Training Initiative: New European migrants and the NHS: Learning from each other, Manual for Trainers, First Edition February 2009, NHS Lothian, Dermot Gorman, Scotland | |
• Aimed at community health professionals, GPs and clinical support staff • Content specific to Eastern European migrants in Scotland • Material on a broad range of healthcare issues including pregnancy and midwifery • E-learning module |
• Aimed at GPs • Content adapted to Turkish, African and Arabic migrants in Austria • Material focused on healthcare issues relevant to GPs • Lectures, quality circles and e-learning module |
England | |
Training Initiative: Ears of Babel: Culturally sensitive primary healthcare, Pharos, Netherlands | |
• One training session (4 hours) • Aimed at GPs only • Delivered by GP trainer & Migrant trainer • Focus on palliative care • Presentation, role play, group discussion |
• Two training sessions (1½ hour, 2½ hours) • Aimed at multidisciplinary practice team • Delivered by professional drama based training company • Focus on mental health • Actor performed scenarios & adapted role play, group discussion |
Greece | |
Guideline: Guidance for communication in cross-cultural general practice consultations: Developed using a participatory research approach, Discipline of General Practice, Centre for Participatory Strategies, Health Services Executive & The Health Research Board, Ireland | |
• Developed in setting with established face-to-face interpretation services | • Introduced in setting without face-to-face interpretation services • Setting up telephone interpretation service |
Ireland | |
Guideline and Training Initiative: Working with an interpreter is easy: Self-directed training package for health professionals, SPIRASI, Ireland | |
• Aimed at health professionals only • Apparent acceptance of the use of informal interpreters in certain circumstances • Lack of detailed information about the dynamics of culture |
• Aimed at inter-stakeholder multi-cultural multi-disciplinary group • Agreement on need to use formal interpreters • Additional training session on the dynamics of culture • Complemented by PLA style ‘Walk-Through’ to allow stakeholders to practise the application of knowledge from training into practice |
The Netherlands | |
Training Initiative: “Did I explain it clearly?” How to communicate with migrants with lower education and less command of the Dutch language, Pharos, The Netherlands | |
• One training session (4 hours) • Aimed mainly at medical practice assistants • Use of formal interpreters • Focus on migrants with limited education and command of the Dutch language |
• Two training sessions (4 hours, 3 hours) • Aimed at entire practice team (including GPs and practice nurses) • Use of formal and informal interpreters • Focus on migrants and natives with limited education and command of the Dutch language • Developing ‘improvement plans’ with GP practice • Regular evaluations of the impact of the training |
The flexible and democratic approach of the PLA methodology allowed us to work with stakeholders to adapt the selected G/TIs to the local context. Stakeholders built shared knowledge and confidence in each other’s expertise which aided their decision-making about the adaptations.
Adaptation was not limited to the settings where a G/TI was selected from another country (as happened in Austria (TI), England (TI) and Greece (G)) but was also required in the settings where a national G/TI had been selected (Ireland (TI&G) and the Netherlands (TI)). The nature of the adaptations related to the target group, content, mode of delivery and trainers for the G/TIs.
Adapting target group
Across most settings stakeholders chose to expand the intervention from a focus on GPs alone to all general practice staff: the stakeholder group discussions made it clear that cross-cultural communication is a collective, and not an individual responsibility, which requires joint efforts for the practice to speak with the same voice. In Austria, where GPs work single-handed practices, this was impractical.
Adapting content
In England the stakeholder group chose to change emphasis of the TI from palliative care to mental health as they felt this would increase its relevance locally. In Ireland the original training on working with a professional interpreter had to be extended as a GP stakeholder did not feel fully equipped after the initial training to deliver interpreted consultations. Stakeholders worked together to design a role-played ‘walk-through’ consultation with a professional interpreter member of the stakeholder group. After consultation, the GP’s response was:
“I’ve clearly never had it [the services of a professional interpreter], it’s brilliant! [….] The thoughts of doing that [consultation] with someone with broken English, with or without a friend, a non-professional interpreter, would be a nightmare. It would take so long to deal with one problem and so many people in the waiting room! (Ire, GP, SH1)
Adaptation work of the training in the Netherlands was fundamentally driven by a decision of the Dutch government to withdraw funding for formal interpreters in primary care. As a result Dutch GPs wanted to know how they could work with informal interpreters and this was included in the training.
Adapting mode of delivery
In both the Netherlands and England stakeholder groups decided to split the 4 h training session of the original TI into two shorter sessions to leave scope for an iterative cycle of reflection, evaluation and improvement and also to fit in with GP practice preferences for shorter training sessions, due to time pressures in their surgeries.
Adapting trainers
Where the original TI was meant to be delivered by a GP and migrant trainer team, a migrant service representative (SH02) in the English site proposed inviting a local professional training company to deliver the training. After discussion the group agreed to involve the training company.
Introducing G/TIs in practice
Overall, after the adaptation work, all four training courses were delivered in the RESTORE sites Austria, England, Ireland and the Netherlands. Following the guideline they had chosen to implement, new interpreting services were offered in two sites but recruitment challenges meant they were infrequently used, in Ireland, or not used at all in Greece (discussed below).
The introduction of the G/TIs required additional work:
Concerning the chosen Training Initiatives:
Logistics of delivering training
In Austria, England and the Netherlands stakeholders identified and liaised with suitable trainers developing training materials collaboratively and liaising extensively with GP practices regarding timing, duration and location of training sessions, necessary materials and refreshments.
Concerning the implementation of a guideline for interpretation services.
Resources for interpreted consultations
Both in Greece and in Ireland part of the implementation of the chosen guideline was the implementation of an interpretation service itself, as this was previously non-existent in the clinical settings involved. The stakeholders were involved in all aspects of setting up interpretation services from scratch, including identifying existing interpreting services, resources for using services and training the practice teams in using these interpreting services.
Advertising and recruiting for interpretation services
In Ireland the stakeholders realised there was limited ability to advertise services in the research site because, due to funding models for GP in Ireland other practices might perceive this as an attempt to ‘poach’ their patients which would reduce the income of other practices.
In Greece stakeholders discussed a potential problem with migrant worker’s employers attending consultations, and came to an agreed solution:
Migrant: “I notice that many migrants come in with their boss, primarily the migrants that work in the fields and usually the boss tries to interpret. What will the GP do then? Will the service still be offered?” (Gr, MSU, SH14)
GP: “I think that we should ask the boss (…) to wait in the waiting area and we should use the phone line service only if okay with the patient. Does everyone agree with this? (..)(Gr, GP, SH6)
[All nod and say yes]
Evaluation
Stakeholders began planning the evaluation of their selected G/TI early in the implementation journey. They considered a wide range of potential strategies to formally evaluate the effectiveness of the intervention in practice and realised plans to gather data in four of the sites, using interviews, evaluation forms and e-mail communication with service users, healthcare providers and practice staff (see Table 7).
Table 7.
COUNTRY➔ Evaluation activity ↓ Number of: |
Ireland Implementation of G &TI | England Implementation of TI | Austria Implementation of TI | The Netherlands Implementation of TI |
---|---|---|---|---|
Interviews to appraise impact on practice of the implemented G/TI | 4 MSU 1 GP 1 I |
1 PS | 3 MSU | |
Evaluation Forms to appraise training and impact on practice | 1 (M, GP, PS, I, H, T) |
7 (GP) | 15 (GP, PN, PS) | |
E-mails to appraise impact on practice | 6 (GP, PN, PS) |
MSU = migrant service users
GP = General Practitioner
PN = primary care nurses, receptionists and practice assistants
PS = primary care administration/management staff
I = member of interpreting community
H = Health service planning and/or policy personnel
After the G/TI had been implemented, stakeholders examined data from formal evaluations in PLA style focus groups devoted to evaluation and discussed their own perspectives on the worth of the G/TIs.
These sessions were highly valued by stakeholders, especially the contribution of the migrants: they flagged up issues the practice staff would not have considered, e.g. that the posters and leaflets in the practice were too difficult to read, and at the same time confirmed the positive effect of the selected G/TI.
Most stakeholders, in the evaluation stage including migrant service users as well as practice members who had not been members of the original stakeholder group, reported benefits of the implemented G/TI in daily practice. Focusing on the data from the formal evaluations, the training in cultural competencies and communication skills in the Netherlands and England led to positive effects in consultations, for example more effective communication in consultations between healthcare professionals and migrants with low literacy (GPs and practice nurses The Netherlands):
“We are more aware of low literacy: we ask patients about it, take more time and arrange more support of social workers (..) we register it now in the patient record, and discuss consequences with the other practice members (The Netherlands, GP, SH4)
Also in Austria participants experienced the training to be helpful.
“I can transfer the lessons learned in the training into daily practice, especially in addressing mental health problems (Austria, GP, SH5)
Training also led to a more tolerant and positive attitude towards migrant service users amongst receptionist staff:
“I am now reacting much calmer than I did before, with more patience, when a migrant who doesn’t speak Dutch stands before me at the desk”. (The Netherlands, PN, SH5)
“The receptionists were all talking about the training which was good, because you know from that the other girls [receptionists] were saying positive things.” (England, PN, SH15)
Even in Greece, despite the lack of uptake of the interpretation service, this positive shift towards migrant service users was noted:
“RESTORE helped me open my eyes to my migrant patients and their needs, where in the past I just scanned over them.” (Greece,/GP/SH6)
There were examples of increased flexibility in accommodating migrants’ appointments amongst all staff (England), and adaptation of the practice to the needs of low literate migrants (the Netherlands). Stakeholders in England and the Netherlands considered that these practice level changes were due to the fact that all clinical, managerial and reception practice members had been involved collectively in the training and thus shared the responsibility to implement the G/TI in the practice:
“We could also go to a training low literacy together, and then go home, and then you have heard the information and that’s it. But we really have worked with each other, and therefore it is more relevant…” (The Netherlands, PN, SH5)
The data about training in the use of interpreters in Ireland indicate improvements in consultations for the GP, interpreters and migrants involved.
The GP described advantages in terms of having a clearer picture of symptoms and more confidence in devising treatment plans:
“So (..) I got a much truer picture of the type of symptoms she was having, and therefore [knew] which treatment to give her (….)[The last day] I gave her a treatment, without an interpreted consultation (..)(…).that wasn’t at all appropriate. So today we revised that, I told her to get rid of that prescription (…). (Ire, GP, SH1)
The trained interpreter involved remarked on this positive feature of the consultation as well:
“I think after the first few sentences she [the patient] actually realised that the communication is flowing and the fact that she’s speaking only Polish means she can focus on what she wanted to say. Not on how to translate it herself. And I think I could see that the difficulty she had was with medical terminology. There was a name of a medical condition that she had and that was interpreted - I think this is where I gained the trust.” (Ireland, interpreter, SH7)
She also commented on the benefits of working with a GP and the fact that they had trained together.
Migrants’ comments on having better confidence in the GP’s diagnosis and treatment and a reported ease about having a ‘stranger’ i.e. the interpreter in the consultation.
“In my case, it’s easy to trust when the interpreter is present, because I knew that she would be able to convey everything that I meant and that I would be understood. I did not feel any discomfort about it [the presence of the interpreter].” (Ireland MSU, SH2)
Exploration of the reasons for lack of uptake of the newly established interpreter service as recommended by the Greek stakeholders’ selected guideline, provided valuable insights. Primary health care providers were keen to try this innovative new service and had overcome resource challenges to actually provide the service, but migrants identified barriers in implementation not previously foreseen: fear among migrants that it would cost them money or that they would have to use their own cellular phone for the telephone interpreter. Furthermore - despite the above mentioned intention of GPs to ask them to wait outside - migrants’ employers often attended their consultations to act as their interpreter. While migrants would have preferred a formal interpreter they felt obstructed to express this view.
Disadvantages of interpreted consultations mentioned in Ireland and Greece relate to practice level challenges – the lack of structural resources to provide interpreters, and logistic challenges organising triadic consultations and difficulties accessing trained interpreters.
The dominant challenges in each setting to sustaining the new ways of working recommended by the G/TIs in daily practice were time constraints that would undermine good intentions to continue new ways of working, or simply forgetting to keep a new practice going. While stakeholders could consider possible strategies for reconfiguration for these challenges, it was more difficult when the challenges related to lack of resources for on-going funding beyond the lifetime of RESTORE:
The lack of structural support, specifically finances for interpreter services, was a major barrier in Ireland and Greece. Another challenge identified during the evaluation in Greece was that migrants were unaware of their rights when it comes to their health and they themselves stated that ‘health care advocacy’ is essential in their community:
“If the services available were clear to us migrants, on what rights we have and this was posted at the health centres, in many cases I wouldn’t bring someone with me. I am lost when I enter the health centre.” (Gr, MSU, SH3)
Stakeholders were aware that these issues were related to local or national policy and anticipated that data from RESTORE could bring about changes to improve the organisational and contextual conditions to facilitate the sustained use of the G/TIs in practice.
Discussion
Summary of findings
The process of local adaptation of G/TIs was complex, time consuming but productive ensuring a tailored intervention for delivery and enhancing buy-in amongst stakeholders. Introduction of adapted G/TIs in practice settings involved intensive planning and problem solving about logistics, resources and advertising new services. Formal evaluations had to be carefully planned as well as ‘reflection space’ for stakeholders to collaboratively consider the impact of introducing G/TIs. Training resulted in a more tolerant attitude and more effective communication, with better focus on migrants’ needs. Implementation of interpreter services, when possible, led to more trust of migrants in doctors’ diagnoses and GPs reporting a clearer picture of migrants’ symptoms.
Comparison with existing literature
This paper extends the existing literature [33, 35–37] by detailing the work required to implement G/TIs to improve communication between migrants and their primary care providers. It focuses attention on the need for adaptation of G/TIs to local context and on methods to achieve this. Our emphasis on the importance of adaptation sits well with the knowledge-to-action cycle proposed in the so-called ADAPTE initiative (www.adapt.org/). [38], although we consider that our commitment to comprehensive stakeholder involvement and our focus on collective problem-solving represent substantive advances over clinician-led adaptation processes. The involvement of migrants and other key stakeholders in evaluation is rare and our study demonstrates a positive impact in the field of migrant healthcare.
Like other interventions, the introduction of the selected G/TIs was challenging work that required attention to resources, skills and training and people’s confidence and trust in the new ways of working (29). Stakeholders overcame a range of issues to try out new knowledge in practice and to see if it would help them achieve their goals in primary care.
In relation to evaluation, our analysis resonates with the findings of earlier research that cultural competency training improves knowledge, attitudes and skills of GPs and practice nurses [39–41] and results in more patient satisfaction [40, 42, 43]. The involvement of all practice members, including reception staff and practice assistants, made it possible to clarify long-term agreements about improvements in their local settings for daily practice. These are novel findings in relation to implementation of G/TIs in this field and resonate with previous research that inter-professional collaborative agreements are needed to effectively change practice long-term [34, 44, 45].
By focusing on these different forms of implementation work, we have drawn attention to the ways in which some problems could be resolved by stakeholders themselves (e.g. enhancing their skill sets by designing additional training) while others could not (e.g. changes in national policy funding models for primary care and interpreting services as barriers to implementation).[46]
Strengths and limitations
The case study design facilitated careful comparative analysis across national settings. PLA was an effective methodology for engaging migrants and other stakeholders from primary care in implementation research. However, while the sample had representation from the desired range of stakeholder groups it did not include migrants who currently experience language problems, which is a clear limitation. Involving clinicians and primary care practice staff in research was challenging, as is often the case [47]). The time consuming nature of PLA, as well as the engagement of different stakeholders in the PLA dialogue was sometimes challenging, as reflected in the fact that GPs in Greece and The Netherlands did not want to include migrants in meetings about practice organisation.
We had a rich data set comprised of fieldwork transcripts and researchers’ reports and we which enhanced our ability to understand the specifics of each national setting. We prospectively applied the theoretical framework NPT for structuring our analysis, and employed standard techniques enhancing the quality and rigour of the analysis.
The generalizability of our findings could be considered limited, given their qualitative nature. However, the aforementioned strengths of our comparative case study and the range of participants across settings has identified multiple transferrable points.
Implications for research and practice
We recommend involving migrants and other key stakeholders when adapting, introducing and evaluating interventions designed to improve cross-cultural communication in primary care settings. The involvement of all clinical and administrative practice members in G/TIs training on cultural competence, and consideration of the evaluation of training, appears to enhance buy-in to future practice changes.
Further research is needed into the effects of these interventions on practitioner knowledge and behaviour as well as migrants’ health outcomes. We also need to know more about the best methods of including migrants who currently experience language problems within their host country [48, 49] and into the quality and appropriate form of interpretation services [50, 51]. Finally, more intersectoral collaborative work is needed to identify solutions to problems relating to policy, funding and other related macro level factors.
Conclusions
Successful implementation of G/TIs to improve communication in cross-cultural general practice consultations benefits from mutual engagement between migrants and other key stakeholders [52], flexible adaptation of G/TIs to meet local needs before their introduction; and collaborative evaluation after a period of time in use.
Acknowledgments
We are grateful for the contributions of the GP practices, migrants and all others involved at any stage of the implementation journeys. We specifically thank Ciaran Clissmann, Lucy Cooper, Orla McGarry and Christine Prinz for their support in implementing the RESTORE project.
Funding
The RESTORE project was funded by the EU Seventh Framework Programme (FP7/2007–2013) under Grant Agreement No. 257258. RESTORE: REsearch into Implementation STrategies to support patients of different ORigins and language background in a variety of European primary care settings
Availability of data and materials
The dataset(s) supporting the conclusions of this article are available at the local research sites.
Authors’ contributions
All authors for this article helped substantially to conceptualize ideas, interpret findings, and review drafts of the manuscript. All authors read and approved and take responsibility for the accuracy of the final manuscript.
Competing interests
COD, FM, CD and AMacF are members of the international study group on NPT; MORdeB and TdeB are Co-Founders and Co-Directors of the Centre for Participatory Strategies, Co. Galway, Ireland.
Consent for publication
All participants have given consent to publish.
Ethics approval and consent to participate
We have ethical approval from our respective national committees in the five fieldwork settings: The Irish College of General Practitioners (reference number 100456), Liverpool Local Research Ethics Committee (reference number UoL0000671), Ethics Committee of Medical University of Vienna (reference number 1081/2012), Research Ethics Committee Radboud University Nijmegen Medical Centre (reference number 2010/436); Ethical Committee ar the University of Heraklion, Crete and National Drug Organization (EOF) (reference number 8297/19-7-2010).
Abbreviations
- G/TIs
Guidelines and training initiatives
- GP
General practitioner
- NPT
Normalisation process theory
- PLA
Participatory learning and action
- RESTORE
REsearch into implementation STrategies to support patients of different ORigins and language background in a variety of European primary care settings
Contributor Information
E. Teunissen, Email: Erik.Teunissen@radboudumc.nl
K. Gravenhorst, Email: K.Gravenhorst@liverpool.ac.uk
C. Dowrick, Email: cfd@liverpool.ac.uk
E. Van Weel-Baumgarten, Email: Evelyn.vanWeel-Baumgarten@radboudumc.nl
F. Van den Driessen Mareeuw, Email: francinevandendriessenmareeuw@hotmail.com
T. de Brún, Email: tomas.debrun@nuigalway.ie
N. Burns, Email: n.burns1@lancaster.ac.uk
C. Lionis, Email: lionis@galinos.med.uoc.gr
F. S. Mair, Email: Frances.Mair@glasgow.ac.uk
C. O’Donnell, Email: Donnell@glasgow.ac.uk
M. O’Reilly-de Brún, Email: mary.oreillydebrun@nuigalway.ie
M. Papadakaki, Email: mpapadakaki@yahoo.gr
A. Saridaki, Email: asaridaki@med.uoc.gr
W. Spiegel, Email: wolfgang.spiegel@meduniwien.ac.at
C. Van Weel, Email: Chris.vanWeel@radboudumc.nl
M. Van den Muijsenbergh, Email: Maria.vandenMuijsenbergh@radboudumc.nl
A. MacFarlane, Email: Anne.MacFarlane@ul.ie
References
- 1.Regenstein M, Andres E, Wynia MK. Appropriate use of non-English-language skills in clinical care. JAMA. 2013;309:145–46. doi: 10.1001/jama.2012.116984. [DOI] [PubMed] [Google Scholar]
- 2.Flores G. The impact of medical interpreter services on the quality of health care: a systematic review. Med Care Res Rev. 2005;62:255–99. doi: 10.1177/1077558705275416. [DOI] [PubMed] [Google Scholar]
- 3.Karliner LS, Jacobs EA, Chen AH, Mutha S. Do professional interpreters improve clinical care for patients with limited English proficiency? A systematic review of the literature. Health Serv Res. 2007;42:727–54. doi: 10.1111/j.1475-6773.2006.00629.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Cheng IH, Drillich A, Schattner P. Refugee experiences of general practice in countries of resettlement: a literature review. Br J Gen Pract. 2015;65:e171–76. doi: 10.3399/bjgp15X683977. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Shannon P, O’Dougherty M, Mehta E. Refugees’ perspectives on barriers to communication about trauma histories in primary care. Ment Health Fam Med. 2012;9:47–55. [PMC free article] [PubMed] [Google Scholar]
- 6.Joshi C, Russell G, Cheng IH, Kay M, Pottie K, Alston M, et al. A narrative synthesis of the impact of primary health care delivery models for refugees in resettlement countries on access, quality and coordination. Int J Equity Health. 2013;12:88. doi: 10.1186/1475-9276-12-88. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Parsons JA, Baker NA, Smith-Gorvie T, Hudak PL. To ‘Get by’ or ‘get help’? a qualitative study of physicians’ challenges and dilemmas when patients have limited English proficiency. BMJ Open. 2014;4:e004613. doi: 10.1136/bmjopen-2013-004613. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Tripp-Reimer T, Choi E, Skemp Kelley L, Enslein JC. Cultural barriers to care: inverting the problem. Diabetes Spectr. 2001;14(1):13-22. https://doi.org/10.2337/diaspect.14.1.13.
- 9.Papic O, Malak Z, Rosenberg E. Survey of family physicians’ perspectives on management of immigrant patients: attitudes, barriers, strategies, and training needs. Patient Educ Couns. 2012;86:205–209. doi: 10.1016/j.pec.2011.05.015. [DOI] [PubMed] [Google Scholar]
- 10.Rechel B, Mladovsky P, Ingleby D, Mackenbach JP, McKee M. Migration and health in an increasingly diverse Europe. Lancet. 2013;381:1235–45. doi: 10.1016/S0140-6736(12)62086-8. [DOI] [PubMed] [Google Scholar]
- 11.Furler J, Ruano AL, Shi L (editors). Interventions in primary health care to improve outcome and equity in health. 2016. http://www.equityhealthj.com/series/PHCandEquity. Accessed 31 May 2016.
- 12.Cochrane LJ, Olson CA, Murray S, Dupuis M, Tooman T, Hayes S. Gaps between knowing and doing: understanding and assessing the barriers to optimal health care. J Contin Educ Health Prof. 2007;27(2):94–102. doi: 10.1002/chp.106. [DOI] [PubMed] [Google Scholar]
- 13.Larisch A, Oertel WH, Eggert K. Attitudes and barriers to clinical practice guidelines in general and to the guideline on Parkinson’s disease. A national survey of German neurologists in private practice. J Neurol. 2009;256(10):1681–8. doi: 10.1007/s00415-009-5178-3. [DOI] [PubMed] [Google Scholar]
- 14.De Brún T, O’Reilly -de Brún M, van Weel-Baumgarten E, Van Weel C, Dowrick C, Lionis CA O’Donnell C, Burns N, Mair FS, Saridaki A, Papadakaki M, Princz C, van den Muijsenbergh M, MacFarlane A. Guidelines and training initiatives that support communication in cross-cultural primary-care settings: appraising their implementability using normalization process theory. Fam Pract. 2015;32:420–25. doi: 10.1093/fampra/cmv022. [DOI] [PubMed] [Google Scholar]
- 15.Van den Muijsenbergh M, van Weel-Baumgarten E, Burns N, O’Donnell C, Mair F, Spiegel W, et al. Communication in cross-cultural consultations in primary care in Europe: the case for improvement. The rationale for the RESTORE FP 7 project. Prim Health Care Res Dev. 2014;15:122–33. doi: 10.1017/S1463423613000157. [DOI] [PubMed] [Google Scholar]
- 16.Priebe S, Sandhu S, Dias S, Gaddini A, Greacen T, Ioannidis E, Kluge U, Krasnik A, Lamkaddem M, Lorant V, Riera RP, Sarvary A, Soares J, Stankunas M, Strabmayr C, Wahlbeck K, Welbel M, Bogic M. Good practice in health care for migrants: views and experiences of care professionals in 16 European countries. BMC Public Health. 2011;11:187. doi: 10.1186/1471-2458-11-187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.O’Reilly-de Brún M, de Brún T, Okonkwo E, Bonsenge-Bokanga JS, De Almeida Silva MM, Ogbebor F, Mierzejewska A, Nnadi L, van Weel-Baumgarten E, van Weel C, van den Muijsenbergh M, MacFarlane A. Using participatory learning & action research to access and engage with ‘hard to reach’ migrants in primary healthcare research. BMC Health Serv Res. 2016;16:25. doi: 10.1186/s12913-015-1247-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Hadziabdic E, Hjelm K. Arabic-speaking migrants’ experiences of the use of interpreters in healthcare: a qualitative explorative study. Int J Equity Health. 2014;13:49. doi: 10.1186/1475-9276-13-49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Friedman E, Hill P, Ruano A. Health, equity and the post-2015 agenda: raising the voices of marginalized communities. http://www.biomedcentral.com/collections/Go4Health Accessed 31 May 2016. [DOI] [PMC free article] [PubMed]
- 20.Van den Muijsenbergh M, Teunissen E, van Weel-Baumgarten E, van Weel C. Giving voice to the voiceless: how to involve vulnerable migrants in healthcare research. Br J Gen Pract. 2016;66:284–85. doi: 10.3399/bjgp16X685321. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Department of Health and Children. Primary Care: A New Direction. Quality and Fairness - A Health System for You. Health Strategy. 2001. http://health.gov.ie/wp-content/uploads/2014/03/primcare-report.pdf. Accessed 31 May 2016.
- 22.The Netherlands Organisation for Health Research and Development. Patiëntenparticipatie in onderzoek, kwaliteit en beleid. 2016. http://www.zonmw.nl/nl/programmas/programma-detail/patientenparticipatie-in-onderzoek-kwaliteit-en-beleid/algemeen/. Accessed 31 May 2016.
- 23.Dowrick C, Lovell K, Lamb J, Aseem A, Beatty S, Bower P, et al. Increasing equity of access to high quality mental health services in primary care: a mixed-methods study. Programme Grants Applied Res. 2013;1:1–184. doi: 10.3310/pgfar01020. [DOI] [PubMed] [Google Scholar]
- 24.O’Reilly-de Brun M, de Brun T, Okonkwo E, Bonsenge-Bokanga JS, De Almeida Silva MM, Ogbebor F, et al. Using participatory learning & action research to access and engage with ‘hard to reach’ migrants in primary healthcare research. BMC Health Serv Res. 2016;16:25. doi: 10.1186/s12913-015-1247-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Teunissen E, van den Bosch L, van Bavel E, van den Driessen MF, van den Muijsenbergh M, van Weel-Baumgarten E, et al. Mental health problems in undocumented and documented migrants: a survey study. Fam Pract. 2014;31:571–577. doi: 10.1093/fampra/cmu038. [DOI] [PubMed] [Google Scholar]
- 26.World Health Organization. How health systems can address health inequities linked to migration and ethnicity. 2010. http://www.euro.who.int/__data/assets/pdf_file/0005/127526/e94497.pdf. Accessed 6 Jun 2016
- 27.Stake RE. The Art of case study research. Thousand Oaks: Sage; 1995. [Google Scholar]
- 28.MacFarlane A, O’Donnell C, Mair F, et al. REsearch into implementation Strategies to support patients of different Origins and language background in a variety of European primary care settings (RESTORE): study protocol. Implement Sci. 2012;7:111. doi: 10.1186/1748-5908-7-111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.May CR, Mair F, Finch T, MacFarlane A, Dowrick C, Treweek S, et al. Development of a theory of implementation and integration: normalization process theory. Implement Sci. 2009;4:29. doi: 10.1186/1748-5908-4-29. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.de Brún T, O’Reilly-de Brún M, O’Donnell CA, MacFarlane A. Learning from doing: the case for combining normalisation process theory and participatory learning and action research methodology for primary healthcare implementation research. BMC Health Serv Res. 2016;16(1):346. doi: 10.1186/s12913-016-1587-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Lionis C, Saridaki A, Dowrick C, O’Donnell C, Mair FS, van den Muijsenbergh M, et al. Engaging migrants and other stakeholders to improve communication in cross-cultural consultation in primary care: a theoretically informed participatory study. BMJ Open 2016, in press. BMJ Open, in press: bmjopen-2015-010822.R1. [DOI] [PMC free article] [PubMed]
- 32.Mays N, Pope C. Qualitative research in health care. Assessing quality in qualitative research. BMJ. 2000;320:50–52. doi: 10.1136/bmj.320.7226.50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Mc Menamin R, Tierney E, Mac FA. Who decides what criteria are important to consider in exploring the outcomes of conversation approaches? A participatory health research study. Aphasiology. 2015;29(8):914–38. doi: 10.1080/02687038.2015.1006564. [DOI] [Google Scholar]
- 34.O’Reilly-de Brun M, de Brun T. The use of Participatory Learning and Action Research in intercultural health: some examples and questions. Migr Soc Change. 2010;6(1).
- 35.Hadziabdic E, Heikkila K, Albin B, Hjelm K. Problems and consequences in the use of professional interpreters: qualitative analysis of incidents from primary healthcare. Nurs Inq. 2011;18:253–61. doi: 10.1111/j.1440-1800.2011.00542.x. [DOI] [PubMed] [Google Scholar]
- 36.Greenhalgh T, Voisey C, Robb N. Interpreted consultations as ‘business as usual’? An analysis of organisational routines in general practices. Sociol Health Illn. 2007;29:931–54. doi: 10.1111/j.1467-9566.2007.01047.x. [DOI] [PubMed] [Google Scholar]
- 37.MacFarlane A, O’Reilly-de Brún M. An evaluation of uptake and experience of a pilot interpreting service in general practice in the HSE Eastern Region. 2009. http://hdl.handle.net/10147/212690 Accessed 31 May 2016.
- 38.Harrison MB, Legare F, Graham ID, Fervers B. Adapting clinical practice guidelines to local context and assessing barriers to their use. CMAJ. 2010;182:E78–84. doi: 10.1503/cmaj.081232. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Beach MC, Gary TL, Price EG, Robinson K, Gozu A, Palacio A, et al. Improving health care quality for racial/ethnic minorities: a systematic review of the best evidence regarding provider and organization interventions. BMC Public Health. 2006;6:104. doi: 10.1186/1471-2458-6-104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Chipps JA, Simpson B, Brysiewicz P. The effectiveness of cultural-competence training for health professionals in community-based rehabilitation: a systematic review of literature. Worldviews Evid Based Nurs. 2008;5:85–94. doi: 10.1111/j.1741-6787.2008.00117.x. [DOI] [PubMed] [Google Scholar]
- 41.Beach MC, Price EG, Gary TL, Robinson KA, Gozu A, Palacio A, et al. Cultural competence: a systematic review of health care provider educational interventions. Med Care. 2005;43:356–73. doi: 10.1097/01.mlr.0000156861.58905.96. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Harmsen H, Bernsen R, Meeuwesen L, Thomas S, Dorrenboom G, Pinto D, Bruinzeels M. The effect of educational intervention on intercultural communication: results of a randomised controlled trial. Br J Gen Pract. 2005;55(514):343–50. [PMC free article] [PubMed] [Google Scholar]
- 43.Lie DA, Lee-Rey E, Gomez A, Bereknyei S, Braddock CH. Does cultural competency training of health professionals improve patient outcomes? A systematic review and proposed algorithm for future research. J Gen Intern Med. 2011;26:317–25. doi: 10.1007/s11606-010-1529-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Grumbach K, Bodenheimer T. Can health care teams improve primary care practice? JAMA. 2004;291:1246–51. doi: 10.1001/jama.291.10.1246. [DOI] [PubMed] [Google Scholar]
- 45.Ho MJ, Yao G, Lee KL, Hwang TJ, Beach MC. Long-term effectiveness of patient-centered training in cultural competence: what is retained? what is lost? Acad Med. 2010;85:660–64. doi: 10.1097/ACM.0b013e3181d296b0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Dahal G, Qayyum A, Ferreyra M, Kassim H, Pottie K. Immigrant community leaders identify four dimensions of trust for culturally appropriate diabetes education and care. J Immigr Minor Health. 2014;16:978–84. doi: 10.1007/s10903-013-9805-9. [DOI] [PubMed] [Google Scholar]
- 47.Salmon P, Peters S, Rogers A, Gask L, Clifford R, Iredale W, Dowrick C, Morriss R. Peering through the barriers in GPs’ explanations for declining to participate in research: the role of professional autonomy and the economy of time. Fam Pract. 2007;24(3):269–75. doi: 10.1093/fampra/cmm015. [DOI] [PubMed] [Google Scholar]
- 48.Lamb J, Dowrick C, Burroughs H, Beatty S, Edwards S, Bristow K, et al. Community engagement in a complex intervention to improve access to primary mental health care for hard-to-reach groups. Health Expect. 2015;18:2865–79. doi: 10.1111/hex.12272. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Ismail MM, Gerrish K, Naisby A, Salway S, Chowbey P. Engaging minorities in researching sensitive health topics by using a participatory approach. Nurse Res. 2014;22:44–8. doi: 10.7748/nr.22.2.44.e1268. [DOI] [PubMed] [Google Scholar]
- 50.Greenhalgh T, Robb N, Scambler G. Communicative and strategic action in interpreted consultations in primary health care: a Habermasian perspective. Soc Sci Med. 2006;63:1170–87. doi: 10.1016/j.socscimed.2006.03.033. [DOI] [PubMed] [Google Scholar]
- 51.Hasbun Avalos O, Pennington K, Osterberg L. Revolutionizing volunteer interpreter services: an evaluation of an innovative medical interpreter education program. J Gen Intern Med. 2013;28:1589–1595. doi: 10.1007/s11606-013-2502-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.World Health Organization. Health of Migrants − the way forward. Report of a global consultation. 2010. http://www.who.int/hac/events/consultation_report_health_migrants_colour_web.pdf Accessed 6 Jun 2016
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The dataset(s) supporting the conclusions of this article are available at the local research sites.