Systematic review of Indigenous cultural safety training interventions for healthcare professionals in Australia, Canada, New Zealand and the United States

Abstract

Objective

To synthesise and appraise the design and impact of peer-reviewed evaluations of Indigenous cultural safety training programmes and workshops for healthcare workers in Australia, Canada, New Zealand and/or the United States.

Design

Systematic review.

Data sources

Ovid Medline, Embase, PsycINFO, CINAHL, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, Bibliography of Indigenous Peoples in North America, Applied Social Sciences Index & Abstracts, ERIC (Education Resources Information Center), International Bibliography of the Social Sciences, ProQuest Dissertations & Theses Global, Sociological Abstracts, and Web of Science’s Social Sciences Citation Index and Science Citation Index from 1 January 2006 to 12 May 2022.

Eligibility criteria

Studies that evaluated the outcomes of educational interventions for selecting studies: designed to improve cultural safety, cultural competency and/or cultural awareness for non-Indigenous adult healthcare professionals in Canada, Australia, New Zealand or the United States.

Data extraction and synthesis

Our team of Indigenous and allied scientists tailored existing data extraction and quality appraisal tools with input from Indigenous health service partners. We synthesised the results using an iterative narrative approach.

Results

2442 unique titles and abstracts met screening criteria. 13 full texts met full inclusion and quality appraisal criteria. Study designs, intervention characteristics and outcome measures were heterogeneous. Nine studies used mixed methods, two used qualitative methods and two used quantitative methods. Training participants included nurses, family practice residents, specialised practitioners and providers serving specific subpopulations. Theoretical frameworks and pedagogical approaches varied across programmes, which contained overlapping course content. Study outcomes were primarily learner oriented and focused on self-reported changes in knowledge, awareness, beliefs, attitudes and/or the confidence and skills to provide care for Indigenous peoples. The involvement of local Indigenous communities in the development, implementation and evaluation of the interventions was limited.

Conclusion

There is limited evidence regarding the effectiveness of specific content and approaches to cultural safety training on improving non-Indigenous health professionals’ knowledge of and skills to deliver quality, non-discriminatory care to Indigenous patients. Future research is needed that advances the methodological rigour of training evaluations, is focused on observed clinical outcomes, and is better aligned to local, regional,and/or national Indigenous priorities and needs.

STRENGTHS AND LIMITATIONS OF THIS STUDY.

• Our systematic review built on existing tailored Indigenous systematic review methodologies to implement a method aimed at optimising relevance for Indigenous peoples by ensuring that their expertise and knowledge was centred throughout the project.

• Our systematic review applied data extraction and appraisal tools that were designed and implemented in partnership with Indigenous community partners.

• The review is limited to Indigenous cultural safety programmes with evaluations that have been published in the peer-review and grey literature and as such, may not have captured the true breadth of existing Indigenous cultural safety training programmes and related evaluations.

• The review is limited to interventions directed towards healthcare providers.

Introduction

Colonisation has long been recognised by Indigenous peoples from around the world as a cross-cutting and foundational determinant of Indigenous/non-Indigenous health disparities.¹ More recently, a series of apologies by world leaders has enhanced general societal awareness of anti-Indigenous colonial injustices, abuses and harms.^2–5 Simultaneously, a rapidly growing body of academic scholarship clearly demonstrates ongoing, widespread and harmful anti-Indigenous colonial policies and practices that are rooted in racist ideologies of white supremacy.^6–12

Common manifestations of persistent colonialism include the emergence of deeply rooted negative anti-Indigenous stereotyping and assumptions in microlevel social interactions, organisational design and social architecture.^{10 13 14} In healthcare contexts, this includes: racist contamination of the healthcare provider–Indigenous patient interface; organisational level barriers to equitable Indigenous health services access and Indigenous/settler imbalances in the distribution of health and social resources.^{10 13 15} Social media and linked public reporting have begun to expose the life-threatening severity of explicit attitudinal anti-Indigenous racism but there can be resistance to acknowledging the underlying challenges of ongoing implicit and system-level failures. For example, Joyce Echequan was able to record the anti-Indigenous racist disparagement she experienced from healthcare staff when seeking treatment for a life-threatening illness at the Lanaudiere hospital in Joliette, Quebec immediately prior to her death.¹⁶ The behaviours of the individual providers were widely regarded as grossly unacceptable following media reporting. However, the Premier of Quebec refused to acknowledge the role of systemic racism in Joyce’s death.¹⁷

Multiple studies have demonstrated that implicit race preference bias is common among healthcare providers,¹⁸ even when they explicitly express antiracist values and attitudes.¹⁹ Further, implicit race preference bias has been linked to differential application of clinical practice guidelines, with non-adherence disproportionately impacting socially excluded racialised and ethnic patient populations.²⁰

Not surprisingly, given the broad scope and injurious impacts of anti-Indigenous racism, its interruption in healthcare contexts has emerged as a priority for Indigenous and allied policy-makers, practitioners and researchers. Of the Truth and Reconciliation Commission of Canada’s seven Calls to Action in the domain of health, two address the need to provide ‘cultural competency’ training for healthcare providers.²¹ These policy recommendations have been accompanied by a rapid growth of interventions designed to interrupt anti-Indigenous racism, primarily through educational interventions for healthcare providers and trainees.^{22 23} On engagement with this literature,²² it became apparent to our team that the approach, content and evaluations of existing cultural competency trainings vary widely. It was unclear which training approaches and strategies were most effective, especially with respect to improving disparities in clinical outcomes.

In order to address these knowledge gaps, we conducted a systematic literature review focused on the design and impacts of existing Indigenous cultural safety and competency training interventions. The primary aim of this review was to identify, appraise and synthesise the design and impacts of these educational interventions on non-Indigenous healthcare professionals’ knowledge, attitudes and practices. The secondary aim was to investigate whether specific training approaches, strategies, formats or educational content were more successful, and if yes, for whom and in what ways. To help manage heterogeneity, we restricted this review to Indigenous-specific educational interventions in Australia, Canada, New Zealand and the United States. These globally affluent countries share both relatively well-resourced health and social service systems and a history of European colonisation that continues to negatively impact the health and well-being of First Peoples, including equitable access to these service systems.

Methods

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 statement was used to guide our literature review and reporting.²⁴ Online supplemental figure 1 documents the process of article screening for inclusion in our analysis. Tables 1 and 2 summarise key aspects of the included studies: intervention content; participants; evaluation methods and study outcomes.

Table 1.

Summary of interventions

Author(s)	Year	Country	Intervention	Content delivery	Setting	Core curriculum topics	Participants(s)
Barajas37	2021	United States	10 min online PowerPoint presentation and YouTube video	Online module(s)	Online	Cultural knowledge, spirituality and beliefs; professional practice issues; interpersonal communication skills	Emergency department healthcare providers and staff (n=6)
Barnabe et al 38	2021	Canada	Phase I: half-day workshop and phase II: full day workshop (6 months later)	Online module(s); interactive group discussions, reflections and experiential exercises	Clinical	Determinants of Indigenous health; oppressive and racist policies, colonisation and white racial privilege; specific health focus	Rheumatologists (n=34)
Brewer et al 39	2020	New Zealand	2 self-paced online modules	Online module(s); self-learning tools; personal reflections	Online	Family structures, kinship and responsibilities; cultural knowledge, spirituality and beliefs; past policies and practices; determinants of Indigenous health; health disparities; professional practice issues; oppressive and racist policies, colonisation and white racial privilege; interpersonal communication skills; specific health focus	Speech Language Therapists (n=11)
Chapman et al 40	2014	Australia	3×2 hours workshops over 6 weeks	Didactic lecture; interactive group discussions, reflections and experiential exercises; personal reflections	Clinical	Cultural knowledge and ideology	Emergency Department: nursing, clinical and allied health staff (n=48)
Crowshoe et al 41	2018	Canada	Full day (8 hours) workshop	Interactive group discussions, reflections and experiential exercises	Clinical	Determinants of Indigenous health; professional practice issues; oppressive and racist policies, colonisation and white racial privilege; interpersonal communication skills	Family physicians and allied health professionals (n=32)
Hinton et al 44	2014	Australia	3 full-day workshops over 2 months	Didactic lecture; interactive group discussions, reflections and experiential exercises; self-learning tools	Clinical	Specific health focus	Clinical and Allied Health Staff (n=21)
Hulko et al 45	2021	Canada	8–10 hours of online training over 8–10 weeks, and a full day Storytelling Session and Talking Circle with an Elder	Online module(s); story telling and talking circles; knowledge quiz; personal reflections	Online and classroom	Indigenous diversity; family structures, kinship and responsibilities; cultural knowledge, spirituality, and beliefs; past policies and practices; determinants of Indigenous health; health disparities; professional practice issues; oppressive and racist policies, colonisation and white racial privilege; specific health focus	Nurses (n=38)
Kerrigan et al 46	2020	Australia	Full day (7 hours) workshop	Didactic lecture; interactive group discussions, reflections and experiential exercises	Clinical	Cultural knowledge, spirituality and beliefs; past policies and practices; professional practice issues; oppressive and racist policies, colonisation and white racial privilege; interpersonal communication skills	Hospital staff (n=621)
Kerrigan et al 47	2022	Australia	7×18–20 min podcasts (1/week)	Online podcasts; diary entries	Online	Counterstories; interpersonal communication skills; social justice	Physicians (n=16)
Liaw et al 48	2015	Australia	Half day workshop, case study toolkit and cultural mentors	Workshop; cultural mentor; self-learning tools	Clinical	Interpersonal communication skills; cultural respect	Clinical practice - solo physician/groups (n=10)
Liaw and Wade49	2019	Australia	Half day workshop, case study toolkit and cultural mentor	Workshop; cultural mentor; self-learning tools	Clinical	Interpersonal communication skills; cultural respect	General practice clinics (n=56); general practitioner physicians (n=334); practice managers (n=56); practice nurses (n=93)
Sauvé et al 51	2022	Canada	Half-day in-person simulation workshop	Simulation training	Clinical	Determinants of Indigenous health; professional practice issues; oppressive and racist policies, colonisation and white racial privilege	Physicians (family medicine residents) (n=29)
Wheeler et al 52	2021	Australia	1.5-hour online module and a full day in-person workshop (2–3 weeks later)	Online module(s); interactive group discussions, reflections and experiential exercises; personal reflections	Online and classroom	Health disparities; professional practice issues; interpersonal communication skills	Pharmacists (n=39)

Table 2.

Summary of evaluation and outcomes

Citation	Study design	Method	Tool(s)	Reported outcome(s)
Barajas, 202137	Mixed methods, quality improvement	Postsurvey	7 dichotomous (yes/no); 2 open-ended questions	Positive impact on insights, knowledge and anticipated behaviour change.
Barnabe et al 202138	Mixed methods	Presurvey (1 week preintervention) and postsurvey (3 months postintervention). Satisfaction survey (1 week postintervention)	Social Cultural Confidence in Care Scale; free-text questions; Experience survey	Significant change in knowledge, skills, and approach to social and cultural factors. Intervention was reported as being relevant and meeting expectations.
Brewer et al, 202039	Qualitative longitudinal	Postsurvey. Follow-up interview (6 months postintervention)	Course feedback; structured interviews	Major themes of ‘putting it into practice’ and ‘keeping it at the forefront’.
Chapman et al, 201440	Quantitative	Presurvey and postsurvey	Area human resources development/population health survey of participation in Aboriginal awareness training workshop	Some change of perceptions towards Aboriginal and Torres Strait Islander peoples. Small effect on familiarity. No effect on attitudes.
Crowshoe et al, 201841	Mixed methods	Presurvey (1-week preintervention) and postsurvey (3 months postintervention). Participant observations. Intervention satisfaction survey	Onsite satisfaction evaluation; observations of participant engagement with content on day; online survey	Significant improvement in knowledge, skills, awareness, confidence and approach to patient care. Strong agreement that the workshop met objectives and expectations.
Hinton et al, 201444	Mixed methods, action-oriented	File audit	2009 vs 2011 audit of inpatient files	Some improvements to the quality of recovery-oriented care, as shown through an increase in recording client social history, family issues and cultural factors.
Hulko et al, 202145	Mixed methods, community-based	Presurveys and postsurveys, knowledge quizzes, and case study care planning. Talking circles.	Approaches to Dementia Questionnaire; Indigenous Cultural Competency Knowledge Quiz; care plans for ‘Alice’; Talking Circle transcripts	Improvement in the knowledge, skills and values of the nurse participants. Storytelling sessions were reported as being effective at building capacity.
Kerrigan et al, 202046	Mixed methods	Postsurvey	Likert-scale questions on Quality of Training; free-text questions	Provided good to excellent information provided on all topics. Participants wanted further and more specific cultural education opportunities.
Kerrigan et al, 202247	Qualitative, participatory action	Qualitative journal entries. Postintervention interviews	Weekly reflections; feedback interviews	Raised the critical consciousness of participants leading to self-reported attitudinal and behaviour change.
Liaw et al, 201548	Mixed methods, pragmatic	Presurveys and post-surveys and patient file audits (6 months postintervention). Postintervention interviews	Cultural Quotient questionnaire; file audit of health checks and clinical risk factors managed; follow-up interviews with staff, cultural mentors and patients	Clinical practices improved their readiness to provide culturally appropriate care. Individual clinic staff improved their cultural strategic thinking.
Liaw and Wade, 201949	Mixed methods, cluster RCT	File audit. Presurvey and postsurvey (12 months postintervention)	Cultural Quotient questionnaire; audit of rates of healthcare claims and chronic disease risk factors.	No significant change in Indigenous health check rates or cultural quotient scores.
Sauvé et al, 202251	Quantitative	Presurvey and postsurvey	Abridged Scale of Ethnocultural Empathy	Significant increase in empathy, knowledge of Indigenous social determinants of health and motivation to engage with Indigenous patients in a culturally safe manner.
Wheeler et al, 202152	Mixed methods	Presurvey and postsurvey. Training acceptability survey	Cultural Capability Measurement Tool; additional adapted questions; acceptability survey	Significant improvement in cultural capability, confidence, and skills. Significant change in motivation to improve health outcomes for Indigenous patients and reduce barriers. Acceptability of the intervention and perceived value-add to participant practice.

Search strategy

Consistent with the search methods outlined in the Cochrane Handbook for systematic reviews,²⁵ an Information Specialist (CZ) conducted database searches in Ovid Medline, Embase, PsycINFO, CINAHL, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, Bibliography of Indigenous People in North America, Applied Social Sciences Index & Abstracts, ERIC (Education Resources Information Center), International Bibliography of the Social Sciences, ProQuest Dissertations & Theses Global, Sociological Abstracts, and Web of Science’s Social Sciences Citation Index and Science Citation Index. Search strategies were adapted for each database and used a comprehensive combination of subject headings and keywords for the concepts of Indigenous peoples, cultural competence and health professionals’ education. Databases were searched for English language records from 2006 to 12 May 2022 (based on the emergence of literature describing and evaluating Indigenous cultural safety interventions) and uploaded into Colandr.²⁶ The reference lists of seminal texts and review articles were then reviewed for additional records. An additional three articles were identified for study inclusion. For the detailed search strategies see online supplemental figure 2.

Study screening

Two independent reviewers screened all title and abstracts for full-text review using the following inclusion criteria:

1. Study specific to Indigenous contexts in what is now known as Australia, Canada, New Zealand and/or the United States.

2. Study describes educational interventions (workshops, training, coursework, community visits, etc) designed/implemented to improve cultural safety, cultural competency and/or cultural awareness.

3. Educational intervention focused on a majority of non-Indigenous adult participants healthcare professionals who provide services (eg, health or social services) to Indigenous peoples.

Full texts were obtained for all studies that passed this title and abstract screening stage and in the event that there was not enough information in the abstract to determine inclusion according to these three criteria.

Three researchers collaborated on full-text screening and further eliminated articles that on full reading, did not meet the primary inclusion criteria and two secondary inclusion criteria: (1) detailed information about the educational intervention’s design and implementation; (2) defined evaluation outcomes. As per our inclusion criteria, we excluded studies in which the majority of the learners were Indigenous and/or the focus of the intervention was at the organisational versus healthcare provider level. We additionally excluded train-the-trainer interventions in which the participants were not directly providing health services. Our two-phased screening protocol is available as online supplemental file 1.

Data abstraction and quality appraisal

Three researchers collaborated on data abstraction across the following categories: study methods (design, evaluation methods and tools, participants, sampling/recruitment), study population, sampling and recruitment methods, educational intervention design (pedagogy, content, modifications) and outcomes (individual and system level).

Two independent reviewers completed preliminary data abstraction and the lead author (B-JH) subsequently reviewed all abstractions and finalised tables 1–4. The lead and senior authors (B-JH and JS) independently appraised methodological quality using a tailored version of the Well Living House Quality Appraisal Tool (WLHQAT)^27–29 (online supplemental figure 3) and subsequently met to discuss and reach consensus on scores (table 3). WLHQAT includes three equally weighted assessment domains: local Indigenous community relevance of methods; rigour and validity; and strength of evidence and has a maximum total score of 12. Studies with a total score of <7 were not included in the full synthesis. The interdisciplinary nature of included studies added complexity to the quality appraisal, in that the research team, study design, concepts and priorities, data collection, and measures were wide-ranging.

Table 3.

Well Living House quality appraisal scores

Citation	Scoring range 1–3/4–6/7–9/10–12
Barajas 202137	7–9
Barnabe et al, 202138	7–9
Brewer et al, 202039	7–9
Chapman et al, 201440	7–9
Crowshoe et al, 201841	7–9
Delbridge et al, 201842	4–6
Durey et al, 201743	4–6
Hinton et al, 201444	7–9
Hulko et al, 202145	7–9
Kerrigan et al, 202046	7–9
Kerrigan et al, 202247	7–9
Liaw et al, 201548	10–12
Liaw and Wade, 201949	10–12
McMichael et al, 201950	4–6
Sauvé et al, 202251	7–9
Wheeler et al, 202152	7–9

Table 4.

Summary of Indigenous involvement in curriculum development, curriculum delivery and evaluation/research activities

Citation	Study design	Curriculum development	Curriculum delivery	Curriculum evaluation	Study analysis	Dissemination	Positionality
Barajas 202137	Yes	Yes	None listed	Yes	Yes	Yes	Yes
Barnabe et al, 202138	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Brewer et al, 202039	None listed	Yes	None listed	None listed	None listed	Yes	None listed
Chapman et al, 201440	None listed	None listed	Yes	None listed	None listed	None listed	None listed
Crowshoe et al, 201841	Yes	Yes	Yes	Yes	Yes	Yes	Limited
Hinton et al, 201444	None listed	None listed	None listed	None listed	None listed	None listed	None listed
Hulko et al, 202145	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Kerrigan et al, 202046	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Kerrigan et al, 202247	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Liaw et al, 201548	None listed	Yes	Limited	Yes	None listed	None listed	None listed
Liaw and Wade, 201949	None listed	Yes	Limited	Yes	None listed	None listed	None listed
Sauvé et al, 202051	Yes	Yes	Yes	None listed	None listed	None listed	None listed
Wheeler et al, 202152	Yes	Yes	Yes	Yes	None listed	None listed	None listed

Synthesis

We applied an iterative narrative approach to our synthesis.³⁰ This method was a good fit with the heterogeneity of study designs and outcomes and our secondary aim to understand which specific training approaches were impactful for whom and in what ways. In addition to our primary aim of identifying, summarising and assessing the design and outcomes of existing published evaluations of Indigenous cultural safety education programming for healthcare professionals, we were particularly interested in documenting underlying pedagogies, instructional strategies, formats, and content and how these might be related to programme success across participant groups and contexts. We were also interested in the involvement of Indigenous instructors and Indigenous communities and how this might have contributed to programme success.

The lead author led the synthesis of study design, participants, quality and outcomes, drawing on data abstraction and with regular input from the other authors. Refinement of secondary narratives regarding (1) the role of underlying pedagogies and (2) Indigenous instructor and community involvement was achieved through iterative discussion of independently identified themes among the authorship team followed by in-depth re-examination of the included studies by the first author.

Throughout the analysis, we applied a critical decolonising lens where we intentionally centred the distinct and diverse knowledges and strengths present in Indigenous communities’ practices of health and well-being.^31–34 The authors sought to acknowledge and critique the systemic power dynamics that so often inform existing health programme evaluation models, particularly when applied to oppressed populations, including Indigenous peoples in what is now known as Australia, Canada, New Zealand and the United States. In so doing, we drew on the foundational Indigenous principles of relationships, reciprocity, responsibility, respect and relevance (known as the five R’s),^{35 36} and applied our decolonising approach to our consideration and analysis of the inclusion (or lack thereof) of Indigenous knowledges and practices in the evaluation of identified studies. Research that looks to learn about Indigenous experiences of health programmes and policies requires acknowledging the unique and distinct relations and interconnections held by Indigenous peoples that are so often decontextualised through the application of Western methodologies.²⁷ In keeping with our decolonising approach, it is important for us to self-locate the authorship team as comprised of two Indigenous women (JS and DS), one racialised settler ally (B-JH), and two non-racialised settler allies (SF and CZ).

Patient and public involvement

We did not involve patients or the public in the design, or conduct, or reporting, or dissemination plans of our research.

Results

Literature search

The literature search strategy resulted in 2442 citations (following removal of any duplicates), from which 2250 were deemed ineligible based on title and abstract screening. A total of 192 articles were selected for full-text review from which 176 were excluded based on the primary inclusion criteria (n=147) or secondary inclusion criteria (n=29) (online supplemental figure 1). We were left with 16 unique studies that described and evaluated Indigenous cultural safety training for health professionals and were deemed eligible for full synthesis inclusion^37–52 (table 3).

Quality appraisal

Among the 16 studies that were included, three scored <7 on the WLHQAT^{42 43 50} (table 3). These studies were excluded from the synthesis. Lower scores reflected a combination of the following: limited, to no involvement of Indigenous community partners in the evaluation; inadequate sample size and/or lack of participant uptake and/or retention in the evaluation; and/or weak evaluation study design.^{43 50} For instance, a low score could reflect that Indigenous scholars or community members were involved in the design and/or delivery of the training programme but not in the design and/or implementation of the evaluation. Another study did not triangulate their qualitative study results.⁴²

Study and population characteristics

The 13 analysed studies were published between 2014 and 2022. The majority (n=7) were conducted in Australia.40 44 46–49 52 A smaller number (n=4) took place in Canada.38 41 45 51 Of the last two studies, one was conducted in the United States³⁷ and the other was conducted in New Zealand.³⁹

Evaluation design varied widely. Nine of the studies applied mixed methods37 38 41 44–46 48 49 52 including various combinations of surveys, open-ended questions, semistructured interviews and talking circles. One of these was a randomised trial that incorporated a participatory action research approach, in which the research team cooperated with the communities, supporting institutions and participants.⁴⁹ Two studies were qualitative.^{39 47} Another two were quantitative.^{40 51} Eight studies incorporated preintervention/postintervention surveys.38 40 41 45 48 49 51 52 Six of the studies incorporated some measure of longer-term impact as part of the evaluation with varied follow-up periods: across 3 years⁴⁴; 12 months⁴⁹; 6 months^{39 48} and 3 months.^{38 41} The remainder of the studies (n=7) collected postintervention data immediately following the intervention. One intervention was described and evaluated across multiple publications as part of a larger research programme.^{48 49} Most (n=10) but not all of the studies, provided access to and/or a detailed description of their evaluation tools.37–41 44 48 49 51 52 Of the 11 studies that used survey tools, 8 employed previously validated evaluation tools,38 40 41 45 48 49 51 52 2 of these, although validated, were adapted by the research team.^{41 51}

Sample sizes varied widely, ranging from 6 to 621, and studies took place in various settings. The majority (n=8) occurred in clinical settings and the remainder were either online (n=3) or a mix of online and in a classroom (n=2). Three of the studies recruited specialised practitioners: rheumatologists,³⁸ pharmacists⁵² and speech language therapists.³⁹ One study recruited only family medicine residents⁵¹ whereas another focused on nurses.⁴⁵ Four of the studies delivered interventions tailored to providers serving a specific health service user population: arthritis,³⁸ psychiatric care and mental health⁴⁴; residential care⁴⁵ and Māori adults with aphasia.³⁹

Reported impacts of Indigenous cultural safety education or training

Study outcomes were almost exclusively learner-focused (n=10) and included learner self-reports regarding: quality of the learning experience; changes in knowledge or awareness; shifts in beliefs; attitudes regarding Indigenous peoples and their care experiences; and/or confidence and skill to care for Indigenous peoples.37–41 45–47 51 52 (table 2) A subset of learner-focused studies (n=4) included measures of self-reported changes in practice.38 39 45 47 These impacts were assessed using proxy measures of clinical behaviour including post-intervention interviews with learners,^{39 47} or through the use of scenarios³⁸ or vignette-based care plans.⁴⁵ Although many of the studies reported significant changes in participants’ attitudes, knowledge and awareness, these findings were tempered by limitations in study design and implementation, such as self-selection bias,38–40 45–47 51 52 small sample size, low uptake and retention,37–39 41 47 51 52 the lack of randomisation and/or controls (all, except for⁴⁹) and potential social desirability response bias.³⁹ Conclusions regarding sustained impact over time, were limited by a paucity of studies (n=6) that included longitudinal measurements.38 39 41 44 48 49

Few studies reported on clinical outcomes, and most were based on self-assessments (n=4) as described above.38 39 45 47 Three studies described externally assessed, patient-based practice outcomes through the use of file audits^{44 48 49} and qualitative interviews with patients at the participating clinics.⁴⁸ Of note, the one study that included a randomised control and externally assessed, patient-based practice outcomes did not demonstrate any significant intervention impact.⁴⁹

Terminology varied widely across the studies, a phenomenon that has already been described elsewhere by Curtis et al ⁵³ as negatively impacting the quality of the evaluations and the ability to draw evidence-based comparisons. Some studies referred to cultural safety37 39 45 47 while others used terms such as: cultural awareness,⁴⁶ cultural security,⁴⁴ cultural respect,^{48 49} cultural competency,^39–41 cultural humility,³⁸ cross-cultural education and cultural capability,⁵² and intercultural empathy.⁵¹ A few studies relied on proxy measures to assess cultural safety. For example, Crowshoe et al ⁴¹ described an increase in learners’ ‘confidence’ as a proxy for cultural safety. Kerrigan et al ⁴⁶ focused on behaviour change and self-reported aspiration as indicative of positive clinical outcomes, and noted that although ‘it was impossible to assess’ whether their intervention shifted behaviour, they could ‘surmise that health professionals aspire to transfer learning to the workplace’.^{46 (p7)} Similarly, in a later paper, Kerrigan et al ⁴⁷ suggested, based on postintervention interviews with learners, that ‘[D]octors changed behaviour in relation to building rapport with patients, asking patients questions, working with Aboriginal interpreters, gaining informed consent.’^(p13) In conclusion they noted that there is ‘still a need to assess if training improves patient experience and outcomes’^(p14) to determine whether the intervention improved cultural safety.⁴⁷ A few authors reflected on the overall limitations of their findings, suggesting that they were not generalisable and/or that additional research is required.37 45 46 51 Hulko et al ⁴⁵ indicated that their intervention and evaluation was based on Secwepemc ways of knowing and being and doing and as such could not be scaled up whereas Barajas³⁷ acknowledged the value of specificity and context and warned against developing and implementing training programmes through a pan-Indigenous approach.

Training approaches and methods

Theoretical frameworks and pedagogical approaches were manifold. Studies referenced transformative learning theories^{38 47 51}; social-constructivist frameworks⁴⁴; diffusion of innovation theory³⁷; a public health framework³⁹ and Educating for Equity.^{38 41} Liaw et al ^{48 49} describe a transtheoretical approach in which they harmonised cultural intelligence frameworks, developments in cultural respect, safety and competence and a review of successful Aboriginal programmes alongside consultation with Aboriginal communities and others. Others (n=4) designed their programme with cultural safety and decolonising philosophies at their core.39 40 46 47 For example, Kerrigan et al ⁴⁶ place the responsibility for change on the ‘hegemonic individuals and institutions’.^{46 (p3)} Only one paper explicitly cited critical race theory⁴⁷ as a core component. A limited number (n=3) did not cite a conceptual theory or framework and instead reviewed cultural safety, competency and awareness in healthcare training and the possible benefits related to training programmes.^{40 45 52} Lastly, some of the training programmes applied participatory action approaches or community-based approaches to the development and delivery of the training.44 45 47–49

Participation for all programmes was voluntary. Overall, there were similarities in course content across programmes. Training delivery modalities varied and included combinations of online modules, didactic lectures, interactive group discussions, workshops, simulations and reflections. (table 1) Only one was delivered as a series of online podcasts, an approach which was well received by learners.⁴⁷ Although some in-person trainings (n=3) were delivered by non-Indigenous instructors,^{44 48 49} most (n=7) were codelivered/facilitated by a mix of Indigenous and non-Indigenous facilitators38 41 45 51 or delivered only by Indigenous facilitator(s)/instructor(s) (table 4).^{40 46 52} Some of the more innovative approaches incorporated story-telling and talking circles with elders⁴⁵; podcasts developed and voiced by elders⁴⁷ and simulation training facilitated with Indigenous community members.⁵¹ Liaw et al ^{48 49} delivered an integrative programme, Ways of Thinking, Ways of Doing, which in addition to a short workshop, participants were also provided with a case study reference toolkit and a cultural mentor.

With one exception,⁴⁹ all of the training programmes reported some level of impact, though only a few of the authors linked the observed impact to their training approaches and methods. Some directly attributed action-oriented^{44 48 49} and community-based^{37 45 51} approaches to the impact of the interventions. However, the same authors also noted that the participatory components to the learning materials were not incorporated consistently (eg, AIMhi care plans and engagement of Aboriginal Mental Health Workers⁴⁴ and cultural mentors⁴⁹). Crowshoe et al ⁴¹ suggested that the impact of their training programme was related to ‘interactive educational techniques and intentional facilitation strategies’^(p54) including a combination of Indigenous and non-Indigenous facilitators. Notably, this study had a high drop-out rate with less than half of the registered learners completing the postsurvey.⁴¹ Chapman et al,⁴⁰ who applied a multi-modal training delivered by an Indigenous trainer, described how the impact of their training programme was limited to significant changes in learners’ perceptions whereas learners’ attitudes remained unchanged. Kerrigan et al ⁴⁷ claimed their online elder podcast changed both learner attitudes and behaviours among a small, convenience sample of 14 learners, based on the analysis of semistructured interviews postintervention.

Indigenous community understandings of measures of success

Indigenous cultural safety can only truly be assessed through the lens of Indigenous patients and communities who ultimately are the recipients of clinical care.⁵⁴ It follows that Indigenous patient and community understandings and measures of success are critical to assessing the impact of any Indigenous cultural safety training programme. However, the degree of involvement of local Indigenous peoples and communities in the development, implementation and evaluation of the educational interventions was limited overall and differed across the studies. Table 4 (Summary of Indigenous Involvement in Curriculum Development, Curriculum Delivery and Research Activities) provides a summary overview. Six out of the 13 peer-reviewed papers included statements describing the ethnic and/or Indigenous identity of the authors. Of these, half (n=3) covered the entire authorship^{37 45 47} and the remainder (n=3) limited self-location to Indigenous coauthors.^{38 41 46} For the most part, Indigenous individuals and/or community members contributed to the development and delivery of the curriculum, either as members of the research team or as local Indigenous community members engaged through participatory and partnered approaches.

Contributions by local Indigenous communities to study evaluations were far more limited, and rarely drew on healthcare delivery and/or patient experience. Some established partnerships with Indigenous run organisations^{48 49} whereas others relied on survey tools that were developed in partnership with Indigenous advisors and communities,^{40 52} however, these were not always locally informed. Others involved Indigenous elders in the evaluation process.^{45 47} In these examples, the elders were involved in both the development and the evaluation of the curriculum. Lastly, only one evaluation focused on healthcare delivery and/or patient experience and included interviews with Indigenous patients and cultural mentors.⁴⁸

Discussion

The rapid growth of Indigenous cultural safety training for healthcare professionals is linked to a global movement to interrupt Indigenous/non-Indigenous health inequities, which are rooted in persistent colonial attitudes and systems, including anti-Indigenous stereotyping and racism.¹⁵ The majority of the papers included here provide a rich description of Indigenous cultural safety training programme approaches, content and implementation. In contrast, analysis and synthesis of the accompanying evaluations of these same training programmes revealed clear and cross-cutting gaps in the demonstration of clinical-level and/or system-level impacts, even though these are commonly referenced as desired outcomes. The majority of evaluations were limited in focus to learner experiences and self-reported practice outcomes. For example, Kerrigan et al ⁴⁷; Brewer et al ³⁹ and Barajas³⁷ all suggested, through their evaluations, that the training programmes resulted in changes in self-reported behaviour and as such, intention and practice. These outcomes, however, are subject to self-reporting response bias such as social desirability. While many of the studies were able to demonstrate some level of impact on knowledge and attitudes towards Indigenous peoples by learners, none of these studies were able to establish an observable impact with respect to a shift towards more culturally safe and clinical practice guideline adherent healthcare for Indigenous patients.

Evidence of shifts in knowledge and attitudes; but evidence base is limited

Self-reported shifts in knowledge and attitudes regarding Indigenous peoples did improve across most of the studies.37–41 45–47 51 52 Although limited, two of the studies suggested that these shifts may be sustained over time.^{38 39} However, when considering the stated impact of these studies, it is also important to take into account the many limitations inherent in the study design. Evaluation studies relied on voluntary self-selection. Sample sizes were generally small and those that were longitudinal showed significant baseline to postintervention lost to follow-up. Eight of the 13 evaluations involved pre–post assessments involving surveys and/or focus groups.38 40 41 45 48 49 51 52 Only one of these included a control group.⁴⁹ In addition, only eight of the studies included validated quantitative surveys that employed scales.38 40 41 45 48 49 51 52 As a result, the shifts in knowledge and attitudes can ‘at best’ be correlated with the described intervention and are limited by several biases arising from the dynamics of course evaluation and marking, participant optimism and in some instances, the lack of anonymity as well as voluntary and low response rates. For the most part, when the described impact was an observable increase in knowledge or shift in attitudes, studies also tended to focus on participant experience of the programme. These measures highlight how participants expressed gratitude regarding what they learnt and spoke to how this might have improved their confidence in working with Indigenous patients going forward. These shifts in confidence, although surely positive, cannot be interpreted as evidence of improved quality of care towards Indigenous patients in the healthcare system.

Very little evidence of patient-focused impacts and no measures of systems-level impact

Cultural safety by definition can only be determined and evaluated by the person receiving the care and their family,⁵⁴ yet only 3 of 13 studies included tools designed to evaluate patient experience: a subset of patient interviews postintervention⁴⁸ and pre/post file audits.^{44 49} Interestingly, Liaw and Wade saw no impact, and concluded, that ‘the lack of effect of the intervention may be attributable to study design limitations, complex and indirect relationship between the intervention and the outcome measures, or contextual factors that influenced the fidelity of the intervention at the Medicare Local/PHN level and its ability to achieve measurable changes in the target behaviours.’^{49 (p267)} None of the studies attempted to measure adherence to clinical practice guidelines, a critical outcome measure which is typically associated with provider training outcomes and could be evaluated through the use of standardised patients,^55–57 ideally unannounced, or through file audits of clinical care.^{58 59} Kirkpatrick has argued that it is ‘difficult, if not impossible to evaluate the impact of training on an organisation due to an inability to separate the variables which could be attributed to other factors’.^{60 (p59)} In this study, we focused on interventions implemented at the level of the healthcare provider, however, the approach does not limit the evaluation to individual level measures, as cultural safety training of healthcare providers can have organisational-level impacts. None of the studies evaluated systems-level changes that may have been associated with individual training. Understanding the networked effect of how training participants subsequently influence their colleagues will be important going forward. Hulko et al ⁴⁵ noted that cultural safety research in general needs to advance tools that will measure these effects, and noted that organisational change will require institutional supports and policy changes that encourage healthcare professionals to implement culturally safe practices.

Impactful specific training approaches, strategies, formats or content

The application of purposeful, evidence-based, pedagogical theory and practices that advance prerequisite knowledge, self-awareness and skills is critical to the success of cultural safety training and education programmes. A number of the reviewed studies described how specific training approaches, formats or content may have contributed to impact, however, most of the authors were also careful to note the limitations of their outcomes and the need for further research to clarify whether and if so, how, approach and content of the training programme contributed to the outcomes. Some authors also described how variation between past and current evaluations of Indigenous cultural safety, including conceptual frameworks, measurement tools and aims, resulted in an overall lack of consensus and limited the development of an evidence base.^{39 46}

Hinton et al ⁴⁴ spoke to the value of a participatory action-oriented study design that incorporated institutional leadership as change agents and clinical champions to encourage recruitment and uptake. This was further supported by Brewer et al ³⁹ who observed low uptake and argued that incentives, particularly over the longer term, were not always effective and that to improve uptake, and consequently evaluation, training ought to be ‘compulsory or obligatory’ and recommended organisational commitment and team involvement. Implementing mandated training alongside appropriate evaluations using file audits, simulation and/or standardised patients will undoubtedly require training and evaluation protocols that address arising concerns of participant healthcare professionals.

The evidence was limited as to whether or not inclusion of Indigenous peoples and communities contributed to successful outcomes, although a number of the studies referenced various components, such as Indigenous vodcasts, guest speakers, cultural mentors and academic lecturers as key to the programmes they evaluated. Liaw et al concluded that the strength of their programme may have been resultant from the inclusion of cultural mentors who, when ‘working with practice staff in their own environment, were effective translators of cultural respect theory and knowledge, as formalised in the toolkit and delivered by the workshop, into practice’.^{48 (p391)} Hinton et al ⁴⁴ also made similar observations regarding cultural advisors, who were involved in the action-oriented programming and group sessions.

Strengths and limitations

We acknowledge that classic systematic review methods have been developed outside of Indigenous contexts, without explicit alignment to Indigenous worldviews, community requirements and methodologies. Our team of Indigenous and allied scientists and Indigenous health service leaders built on existing tailored Indigenous systematic review methodologies^27–29 to implement a method aimed at optimising relevance for Indigenous peoples through: (1) codesign, coleadership and coauthorship by leading Indigenous methods scholars and Indigenous cultural safety educators, ensuring that their expertise and knowledge was centred throughout the project; (2) direct involvement of a senior Indigenous scholar and methodologist (JS) in all stages of the review, analysis and synthesis and (3) application of a data extraction tool developed in consultation with Indigenous community partners: the Southern Ontario Aboriginal Health Access Centre (online supplemental file 2) and the WLHQAT, a quality appraisal tool that was designed at an Indigenous-led research centre in partnership with Indigenous community members.

The review is limited to Indigenous cultural safety programmes with evaluations that have been published in the peer-review and grey literature and as such, may not have captured the true breadth of existing Indigenous cultural safety training programmes and related evaluations. To optimise feasibility and study coherence, we did not include organisational level interventions as for this initial study. Instead, we limited our focus to interventions directed towards healthcare providers. We do recognise that it is likely that lasting system-level impacts will require interventions that are implemented and evaluated at both the individual and organisational levels and would like to highlight the need for additional research focused on advancing and evaluating system-level interventions. Lastly, the review was conducted over a lengthy period of time due to the required extensive and iterative consultation with community partners and Indigenous study team members in the development and implementation of the final screening protocol to ensure that we were centring Indigenous worldviews, experiences and community considerations.

Concluding remarks

Overall, there is a paucity of evidence linking existing Indigenous cultural safety training interventions to enhancements in non-Indigenous healthcare professionals’ knowledge, culturally safe engagement skills and clinical practice guideline adherence when caring for Indigenous patients. As researchers and practitioners in this field, we note that these gaps in rigorous patient outcome focused scholarship are rooted in systemic limitations in the resources available to organisations leading this work to carry out and disseminate comprehensive and cost-intensive evaluations. This systemic under-resourcing and the linked implementation of non-evidence-based interventions is problematic, inconsistent with the evidence standards required in other domains of clinical training, and is commonly associated with the same harmful anti-Indigenous, colonial policies and practices that training is designed to disrupt. Further research investment, with funds directed towards Indigenous-led agencies and organisations that are leading the work in this field, is required to advance training programme evaluation design, implementation, analysis and dissemination. These investments would ensure that both the training programmes and their evaluations meet the dual criteria of excellence in Indigenous health research: (a) methodological rigour and (b) alignment with and connection to local, regional and/or national Indigenous priorities and needs.

Data availability statement

Data are available on reasonable request. Most of the data generated or analysed as well as the WLHQAT applied during this study are publicly available. Additional data and/or materials are available on request from the corresponding author.

Ethics statements

Patient consent for publication

Not applicable.