Stakeholder perceptions and experiences of competency-based training with entrustable professional activities (SPECTRE): protocol of a systematic review and thematic synthesis of qualitative research

Abstract

Background

Competency-Based Medical Education (CBME) aims to align educational outcomes with the demands of modern healthcare. Entrustable Professional Activities (EPAs) serve as key tools for feedback and professional development within CBME. With the growing body of literature on EPAs, there is a need to synthesize existing research on stakeholders’ experiences and perceptions to enhance understanding of the implementation and impact of EPAs. In this synthesis, we will address the following research questions: How are Entrustable Professional Activities experienced and perceived by stakeholders in various healthcare settings, and what specific challenges and successes do they encounter during their implementation?

Methods

Using Thomas and Harden’s thematic synthesis method, we will systematically review and integrate findings from qualitative and mixed-methods research on EPAs. The process includes a purposive literature search, assessment of evidence quality, data extraction, and synthesis to combine descriptive and analytical themes.

Discussion

This study aims to provide insights into the use of EPAs for competency-based education, reflecting diverse contexts and viewpoints, and identifying literature gaps. The outcomes will guide curriculum and policy development, improve educational practices, and set future research directions, ultimately aligning CBME with clinical realities.

Trial Registration

Not required.

Background

Medical education has undergone significant transformations in recent decades, driven by the urgent need to align educational outcomes with the dynamic and complex demands of modern healthcare systems [1, 2]. Traditional structure- and process-based educational frameworks, prevalent at the turn of the 21st century, have increasingly been deemed inadequate for preparing medical graduates for real-world clinical environments [3]. This shift has spurred the global adoption of Competency-Based Medical Education (CBME), an approach emphasizing the attainment of essential competencies for delivering high-quality patient care [3, 4].

Entrustable Professional Activities (EPAs) are pivotal tools within CBME, serving as practical mechanisms for professional development through feedback and competency-based assessments [5]. EPAs, defined as professional responsibilities entrusted to learners once they become clinically competent, translates the broad competencies of CBME’s theoretical frameworks into daily practice [5]. Despite their theoretical promise and growing endorsement, the implementation of EPAs faces challenges in its practical application, partly due to stakeholders engagement and resistance [6–8]. Although there is an increasing body of literature on EPAs—evidenced by over 1,000 articles and approximately 100 new publications annually—there is a notable gap in synthesizing this extensive work to provide coherent, actionable insights. Most research has focused on the theoretical foundations, development, and curricular integration of EPAs [7, 9], neglecting the nuanced context-specific challenges and successes experienced by diverse stakeholders. Furthermore, the rich, in-depth perspectives offered by qualitative research in medical education have not been systematically integrated into a comprehensive synthesis, hindering the ability of educators, policymakers, and researchers to effectively navigate available literature and apply findings to their specific contexts.

Addressing this gap is critical. A systematic review of qualitative studies will clarify the multifaceted experiences and perceptions of various stakeholders, providing a holistic understanding of EPAs’ impact on educational outcomes, professional development, and patient care quality [10, 11]. By synthesizing qualitative findings, we can uncover underlying themes and patterns that quantitative research might overlook, offering deeper insights into the implementation and perception of EPAs across different sociocultural settings [10, 12]. This synthesis will yield evidence-based recommendations to guide policymaking, educational practice, and future research, ensuring that EPAs fulfill their potential in transforming medical education and ultimately improving patient care [11, 13]. By systematically reviewing and integrating these qualitative findings, we can bridge the current knowledge gap, making the vast and complex literature on EPAs in CBME more accessible and actionable for all stakeholders involved.

Research questions

In this synthesis of the published peer-reviewed literature, we will address the following research questions: How are Entrustable Professional Activities experienced and perceived by stakeholders in various healthcare settings, and what specific challenges and successes do they encounter during their implementation?

Methods

Study design

We will use a qualitative synthesis, a systematic review process that collects, organizes, and analyzes qualitative and mixed-method research data, and then provides comprehensive understanding of complex issues or phenomena [12]. We will draw on Thomas and Harden’s principles of thematic synthesis, which entails four steps: systematic search of the literature, assessing quality and relevance of evidence, extracting data, and synthesizing data to integrate descriptive and analytical themes [14].

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines will inform the process of completing and reporting this planned review [15]. This protocol is reported according to the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) statement [16].

Eligibility criteria

• Study Designs: We will include qualitative studies and qualitative components of mixed-method studies.

• Participants: Studies involving medical educators (faculty, attendings, clinical teachers), medical learners (medical students, residents), administrators (program directors, competence committee members), and other stakeholders in EPAs across various healthcare settings will be included.

• Outcomes: Studies that explore the experiences, perceptions, challenges, and successes related to the implementation and use of EPAs.

• Settings: Any healthcare or educational setting where EPAs are implemented.

• Language: We will include articles reported in English or French.

Information sources and search strategy

The following databases were searched by a health sciences librarian (LS): Medline and Medline in Process (Ovid), Embase Classic + Embase (Ovid), APA PsycINFO (Ovid), CINAHL (EBSCOHost), and Education Source (EBSCOHost). A search strategy was developed in Medline and then translated into the other databases as appropriate (see Appendix 1). The searches in all databases were run until August 12, 2024. There were no publication restrictions for the search. All references were entered into an Endnote file for processing (n = 9496), and then were uploaded into Covidence for duplicate removal, and then screening.

Database searches (numbers for PRISMA flowchart):

Medline in Process and Medline (via Ovid): n = 2206.

Embase (via Ovid): n = 2988.

APA PsycINFO (via Ovid): n = 254.

CINAHL (via EBSCOHost): n = 2010.

Education Source (via EBSCOHost): n = 2038.

Total: n = 9496.

Duplicates: n = 2797.

Total with duplicates removed: n = 6699.

The literature search will be updated every six months and prior to submission for peer-review to capture any new relevant publications.

Selection process

Search results will be uploaded to Covidence for screening and data extraction. Two reviewers will screen titles and abstracts independently. Full texts of potentially eligible studies will be retrieved and assessed for inclusion. Disagreements will be resolved through discussion or with a third reviewer if necessary.

Data extraction

Using a standardized data extraction form (Appendix 2), two reviewers will extract data on study characteristics, participants, settings, methodologies, interventions, and key findings. A third reviewer will be consulted where discrepancies occur. For the thematic synthesis, all text labelled as “results” or “findings” will be extracted, including participants’ quotes, and entered in NVivo software for analysis.

Data synthesis

We will follow Thomas and Harden’s three stages of thematic synthesis, an inductive process involving the systematic coding of data and generating of descriptive and analytical themes [14]. The initial stage will begin with line-by-line coding performed independently by two reviewers. Codes will be created inductively to capture the meaning and content of each sentence. All text with a given code will be examined to check for consistency of text interpretation and whether additional levels of coding are needed. In the next stage, descriptive themes will be developed. The reviewers will look for similarities and differences between the codes to group and structure them appropriately, ensuring the meaning of initial coding groups are captured as codes are added and new descriptive themes are developed. The last qualitative synthesis stage involves the generation of analytical themes. Conducted independently then as a group, the reviewers will infer answers from the previously created descriptive themes that address the research questions. Through repetition of this cyclical process, new abstract themes will be generated that sufficiently describe and explain all initial descriptive themes. Evidence-based recommendations can thus be derived from the generated analytical themes. NVivo software will be used to facilitate the entire data coding and synthesis process.

Quality appraisal

The Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Qualitative Research will be used to assess the quality of the include articles [17]. Two reviewers will independently evaluate the quality of the included studies, with discrepancies resolved through discussion or consultation with a third reviewer. Each paper will be categorised as “low” (scores of 0–3), “moderate” (scores of 4–7), or “high” (scores of 8–10) quality. The results of the quality appraisal will not be used as an eligibility criterion for synthesis but will instead be reported descriptively to provide context to the findings. Lower-quality studies will be included in the synthesis process as they may still provide important insights into the phenomena of interest related to the interpretation and reporting of key findings [12]. This approach allows for transparency in how the quality of the included studies may influence the synthesis, enabling readers to interpret the findings with an understanding of the methodological strengths and limitations of the source material.

Confidence in cumulative evidence

We will use the Grading of Recommendations, Assessment, Development and Evaluation – Confidence in the Evidence from Reviews of Qualitative research (GRADE-CERQual) approach to assess the confidence in the findings from the qualitative evidence synthesis [11, 18]. This involves evaluating methodological limitations, relevance, coherence, and adequacy of the data. Assessments for each finding will be made through discussion among all authors through team meetings.

Reflexivity

The study will be informed by a constructivist epistemology, which posits that knowledge is co-constructed through social interactions and shaped by individuals’ experiences, values and contexts. This approach is particularly suited for exploring stakeholder experiences and perceptions, as it acknowledges the subjective and situated nature of these phenomena. Our team comprises individuals with diverse professional backgrounds allowing us a comprehensive and balanced examination of the research questions, but with a common focus on medical education research and practice. JP is a medical student with interest in medical education research. LC is a research coordinator with expertise in qualitative research and knowledge translation. LS is a research librarian in medicine and health sciences. KAL is a PhD qualitative researcher with expertise in medical education research but minimal experience with EPAs. SHM is a medical education researcher, competence committee chair, previous program director, and has local and national expertise with designing and using EPAs. CT is a clinician educator with international expertise on EPAs. RK is a medical education researcher and program director with experiencing in using and implementing EPAs. Each team member will engage in reflexive practices, including group discussions and critical reflections, to surface and mitigate potential biases.

Ethical considerations

This study will not require ethical approval as it involves secondary analysis of published data.

Discussion

The results of this systematic review will provide a comprehensive synthesis of qualitative research on the implementation of EPAs within CBME. Thematic synthesis is particularly suited for this context as it allows for a detailed, nuanced understanding of stakeholders’ perspective and experiences with EPAs. This method focuses on generating rich, in-depth and actionable insights, offering evidence-based recommendations that are crucial for addressing stakeholder concerns and fostering collaboration among educators, administrators, and policymakers [14]. Ultimately, these insights can lead to more effective and sustainable implementation of EPAs, enhancing the quality of medical education and patient care in the CBME era [19].

Several practical and operational challenges need to be addressed to ensure the success of this study. Managing the extensive volume of literature requires a comprehensive search strategy across multiple databases. Using Covidence for screening and data extraction helps streamline this process, although there is a risk of missing relevant studies. To mitigate the risk of missing relevant studies, we have implemented a dual-screening approach to minimize errors and ensure that all relevant studies are identified and included. Additionally, limiting the search to English and French may introduce language and context-related bias. However, we cannot overcome this limitation given the cost of translation and lack of funding for this study. Ensuring consistency among reviewers during screening, extraction, and synthesis is crucial; regular calibration exercises and consensus-building sessions will help mitigate discrepancies [15]. Synthesizing qualitative research presents its own set of challenges, such as the variability in study methodologies, quality and reporting. The subjective nature of quality appraisal and thematic synthesis requires careful and iterative processes, including discussion among the research team members [12]. Despite these challenges, conducting a thematic synthesis remains essential. It provides a robust framework for integrating qualitative findings from diverse contexts, uncovering deeper insights that might be overlooked with alternate methods [14].

Studying the perceptions and experiences of the implementation of EPAs is essential for optimizing educational practices, aligning training with healthcare needs, and informing policy and decision-making [7, 9, 19]. We aim to identify effective practices and context-specific challenges for enhancing competency-based assessments and ensuring graduates are well-prepared for patient care [9]. By conducting a robust qualitative evidence synthesis, we can guide resource allocation, address stakeholder concerns, and foster collaboration. Additionally, we will discuss research gaps and promote continuous quality improvement, advancing the field of medical education and ultimately improving the quality of healthcare delivery.

Appendix 1 – Search Strategies

Medline

1. "entrustable professional activit*".ti,ab.

2. EPAs.ti,ab.

3. 1 or 2

4. ("Environmental Protection Agency" or “Eicosapentaenoic Acid”).ti,ab.

5. 3 not 4

Embase

1. "entrustable professional activit*".ti,ab.

2. EPAs.ti,ab.

3. 1 or 2

4. ("Environmental Protection Agency" or “Eicosapentaenoic Acid”).ti,ab.

5. 3 not 4

APA PsycINFO

1. "entrustable professional activit*".ti,ab.

2. EPAs.ti,ab.

3. 1 or 2

CINAHL

Search ID#		Search Terms
	S5	S3 NOT S4
	S4	TI Eicosapentaenoic Acid OR AB Eicosapentaenoic Acid
	S3	S1 OR S2
	S2	TI ( epa or epas ) OR AB ( epa or epas )
	S1	TI entrustable professional activit* OR AB entrustable professional activit*

Education Source.

Search ID#		Search Terms
	S5	S3 NOT S4
	S4	TI Eicosapentaenoic Acid OR AB Eicosapentaenoic Acid
	S3	S1 OR S2
	S2	TI ( epa or epas ) OR AB ( epa or epas )
	S1	TI entrustable professional activit* OR AB entrustable professional activit*

Appendix 2 – Data Extraction Form.

Authors.

Year.

Journal.

Country.

Study Aim(s).

Methodology.

Methods.

Sampling strategies.

Participants (Sample size, Demographics).

Setting (Undergraduate vs. Postgraduate medical education).

Context of the study (Educational program(s), Country(ies), etc.)

Key findings.

Abbreviations

CBME

Competency-Based Medical Education

EPA

Entrustable Professional Activities

GRADE-CERQual

Grading of Recommendations, Assessment, Development and Evaluation – Confidence in the Evidence from Reviews of Qualitative research

JBI

Joanna Briggs Institute

SPECTRE

Stakeholder Perceptions and Experiences of Competency-Based Training with Entrustable Professional Activities

PRISMA

Preferred Reporting Items for Systematic Review and Meta-Analysis

PRISMA-P

Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols

Author contributions

All authors contributed to the conceptualization of the study. J.P. and R.K. drafted the protocol. L.S. conducted the literature search. All authors provided critical revisions to the protocol. All authors have approved the submitted version and have agreed both to be personally accountable for the author’s own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, even ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature.

Funding

JP received the Medical Student Education Research Grant (MSERG) Award by the Ontario Medical Students Association.

Data availability

No datasets were generated or analysed during the current study.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.