Defining the capabilities and competencies of high-performing family physicians: a mixed methods study

Abstract

Introduction

High-performing primary care is recognised as the foundation of an effective and efficient healthcare system. Many medical graduates report they are not prepared for independent practice. To date, no research has been conducted to identify the key capabilities and competencies of high-performing family medicine graduates in Canada. This pilot project aims to identify the capabilities and competencies of high-performing early-career family physicians in Ontario, Canada, and explore opportunities for enhancing learning, teaching and assessment within family medicine residency programmes.

Methods and analysis

Employing a mixed-methods explanatory sequential study design, this research will use a theory-driven Professional Capability Framework, previously validated in studies across nine professions, to guide the investigation. The first (quantitative) phase involves surveying ~50 high-performing early-career family physicians identified as high performing by educators, colleagues and leaders. The objective of the survey is to identify the key competencies and personal, interpersonal and cognitive capabilities of high-performing family physicians. The second (qualitative) phase involves conducting workshops with stakeholders, including educators, professional associations, regulators and colleges, to test the veracity of the results. Quantitative data will be analysed using descriptive statistics, and qualitative data will be analysed using Braun and Clarke’s thematic analysis. The first and second phases will identify the key capabilities and competencies required to confidently adapt to the independent practice of comprehensive family medicine and inform fit-for-purpose educational strategies for teaching, learning and assessment.

Ethics and dissemination

The study is approved by the University of Toronto’s Health Sciences Research Ethics Board (#41799). Research findings will be discussed with professional bodies, educators responsible for family medicine curricula and universities. Study findings will also be disseminated through academic conferences and academic publications in peer-reviewed journals. Project summaries and infographics will be developed and disseminated to key stakeholders.

Strengths and limitations of this study.

• The study design is informed by a validated theory-driven conceptual framework informed by nine professions.

• An explanatory sequential mixed-methods design will allow for rich data to be generated by exploring themes emerging from quantitative and qualitative data.

• A limitation of this pilot study is that it focuses on a small sample of early-career family physicians in Ontario.

Introduction

High-performing primary care is recognised as the foundation of an effective and efficient healthcare system.¹ Countries with a strong primary care orientation have demonstrably better health outcomes and health equity, lower mortality rates and overall costs of healthcare.^2–4 Family physicians (FPs) are integral to the Canadian healthcare system since they are typically the first point of contact with the primary care system and are responsible for coordinating other specialty services.¹ FPs are trained to practise with a broad scope and are uniquely placed to meet the evolving healthcare needs of patients and their families.¹

The overarching goal of medical education programmes is to prepare physicians to provide the highest quality of care and meet the needs of patients and populations.^5–10 In Canada, the goal of family medicine (FM) residency programmes is to prepare and train residents who are competent to enter, effectively deliver and adapt to the independent practice of comprehensive FM anywhere in Canada.¹¹ However, the healthcare needs of modern society are changing, as a result, the definition of preparedness for practice is continually evolving, and as such, education should change to help prepare professionals to adapt to and successfully navigate complex and changing circumstances.⁶

In Canada, curricula have shifted from time-based training to outcome-based or competency-based medical education (CBME).⁸ CBME is an approach used to design, implement, assess and evaluate medical education and residency programmes. Despite implementing CBME, medical graduates may not adapt to the skills needed in practice even though they are considered to be competent.^{8 12–21} The lack of preparedness has been associated with physician burnout^{22 23} and narrowing scope of practice.²² It has been suggested that in addition to competence, professionals must be capable of adapting to constantly changing health employment systems and be able to face and negotiate the complex and changing circumstances of their practice.¹⁴

Studies of early-career graduates in a wide range of professions consistently show that both competence and capability are important elements of forming the high-performing graduate.^24–31 Whereas competence is the possession of a key set of ‘correct answer’ skills and knowledge and the performance of specific tasks in relatively predictable circumstances, capability is about responsiveness, creativity, contingent thinking and growth and the ability to adapt to relatively uncertain situations.^{30 32} The literature on higher education frequently discusses qualities such as integrity, courage and passion, trustworthiness, responsiveness, the ability to successfully navigate dilemmas and times when things go awry and the ability to adapt and change.³⁰ To be effective, the work environment and its leaders must also support these graduates.³³

To date, no research has identified the key capabilities of high-performing FM graduates. Thus, the objectives of this study are: (1) to determine if the characteristics of high-performing graduate in other professions applies to FM; (2) to identify the key capabilities and competencies of high-performing FM graduates; and (3) and highlight how medical curricula can foster the development of these capabilities and competencies through learning, teaching and capability-focused assessment. We are defining a high-performing FM graduate as a graduate that is competent and able to confidently transition and adapt to the independent practice of comprehensive FM, in which adapting is the ability to navigate effectively diverse and unfamiliar situations, settings and patient populations. This study will: (a) identify the key capabilities and competencies required to confidently adapt to the independent practice of comprehensive FM; (b) review the extent to which these are integrated into medical curricula and given focus in assessment at university and in clinical placements; and (c) highlight how to improve learning and teaching by providing feedback on the best aspects and areas to be improved in medical education in Canada.

Methods and analysis

Study design

This pilot study will build on previous studies of high-performing graduates across professions^{34 35} and use the Professional Capability Framework validated in these studies to identify the key capabilities and competencies of high-performing early-career FPs. A mixed-methods explanatory sequential study design will be used to collect quantitative and qualitative data in two phases.³⁶ A quantitative survey will be followed by two qualitative workshops with stakeholders to explore and explain the survey data results. A sequential explanatory mixed-method study design corroborates the findings, provides a more in-depth exploration of identified relationships and has been used in previously published health research.^{37 38} We will use the Good Reporting of a Mixed Methods Study checklist.³⁹ We will conduct this study from August 2023 to July 2024.

In this study, ‘competence’ will be defined as the possession of the skills and knowledge necessary to perform the duties for a particular role successfully in relatively predictable conditions. ‘Capability’ will be defined as the possession of that mixture of personal, interpersonal and cognitive capabilities which permits confidence to adapt to unfamiliar and complex situations, settings and populations (eg, the ability to work productively with people from a diverse range of professions and backgrounds, manage the unexpected, successfully adopt and deploy new technology, to be change implementation savvy, inventive, sustainability responsive, to learn from experience and to operate with a clear understanding of one’s ethical position).³⁰

Guiding framework

The Professional Capability Framework that underpins the survey used in the study is a conceptual framework that has been developed, tested and validated^24–31 (refer to figure 1). The framework is driven by change management theory for higher education and empirical findings of professionals, practitioners and leaders identified as experienced and effective. The Professional Capability Framework serves as a guiding framework in this study, providing a theoretical foundation for understanding the capabilities and competencies of high-performing FPs. While the framework itself is not being tested for its applicability, it is being used to identify the specific and potentially unique personal, interpersonal and cognitive capabilities that contribute to the success of early-career FPs.

Figure 1.

Professional capability framework. Adapted from Scott.⁴⁰

Three overlapping aspects of professional capability are identified—personal (comprised of factor analysed items clustered into three subdomains: self-awareness and regulation, decisiveness and commitment), interpersonal (with two subdomains: influencing and empathising) and cognitive (with three subdomains: diagnosis, strategy and flexibility and responsiveness). These domains and subdomains are underpinned by relevant role-specific and generic competencies (the skills and knowledge found to be essential to the specific role being studied).

The findings from the successful early-career graduate research have been used as follows: (1) the capabilities with the highest ratings are used to sharpen the focus of learning and assessment during training; (2) the dilemmas identified are used to produce a capstone subject called key dilemmas of early-career professional practice and; (3) existing practitioners are alerted to the top-ranking capabilities and are invited as part of continuing professional development to use the capabilities as a diagnostic framework for reflecting on situations in practice when things went awry, and the situation was not handled well.⁴⁰

First phase: survey of early-career family physicians

Participants

We will use purposive and convenience sampling to recruit participants from Ontario, Canada.⁴¹ First, we will reach out to programme directors and preceptors from medical universities to identify early-career FPs that are 1–5 years into practice. We know from studies by people like Alan Tough (1978), and other scholars, that students particularly value input from those further down the path they are on who are performing effectively.^42–44 In addition, a study among emergency medicine residents at a single training site found the assessment of resuscitation performance in a simulated setting approximated the assessment of resuscitation performance in the clinical workplace.⁴⁵ We will also ask for nominations from early-career FPs and primary care leaders in primary care organisations.

We will recruit programme directors, educators, early-career FPs and primary care leaders by contacting programme and site directors in the postgraduate programmes, identifying educators through public websites and the contacts of the research team, making presentations at programme director or site director meetings, reaching out to a national committee of early-career FPs, advertising through newsletters in stakeholder organisations and journals, the snowballing technique with colleagues and stakeholders and promoting the study through social media.

Based on criteria of high-performing graduates from previous studies across different professions,^{31 46–55} we will ask nominators to identify early-career FPs in their first 5 years who demonstrated the following during their training or their first years in practice. Initially, we will ask participants to identify individuals based on the following criteria: (a) consistent delivery of allocated work on time and to a specified standard in the time allotted; (b) high levels of client (in this case, patient satisfaction); (c) high levels of co-worker satisfaction; (d) being motivated and enthusiastic; (e) demonstrated an eagerness to learn/advance skills; (e) being pro-active (eg, initiating talk with colleagues or reviews of patients); (f) sound communication and liaising skills; (g) good organisation and time management skills; and (h) high levels of competency/professionalism. Participants will be asked to identify candidates that have demonstrated the ability to confidently transition and successfully adapt to diverse and unfamiliar situations, settings and patient populations. We will follow-up with participants to validate whether the criteria set by Scott et al are reflective of high-performing FPs.³¹ We will record the similarities and differences in the attributes identified by participants.

We will include at least 50 early-career FPs, consistent with the methodology of Scott et al.^{31 51–53} We will purposely select a representative sample of ethnically diverse and gender-diverse early-career FPs.

Data collection

A survey based on the Professional Capability Framework questionnaire, with strong validity evidence from earlier studies will be used to identify the capabilities and competencies that are perceived to be most important for high-performing early-career practice FPs and the extent that FPs perceive residency programmes to focus on these capabilities and competencies.³¹ The survey also permits early-career FPs to provide insights into what they had in mind when ranking items and to identify a wide range of suggestions on what universities might want to focus on to make their training more relevant to the needs of medical graduates in early-career practice.⁵⁶ Specifically, respondents will be asked to explain their high ratings for each of the quantitative areas surveyed (‘best aspect’ comments) and to suggest improvements (‘needs improvement’ comments). They will also be asked to identify one of the most challenging situations they have encountered in their early years of professional practice and how they addressed it. This will provide valuable information on real-world scenarios for case-based, dilemma-based and problem-based learning and assessment. The final questionnaire will be administered to the 50 participating early-career FPs, using REDCap—a secure web application for online surveys.⁵⁷ All eligible participants will be sent an open survey link (see online supplemental file 1).

Before administration of the survey, we will pre-test the questionnaire to ensure response process validity by conducting cognitive interviews with a sample of three to five early-career FPs that are gender and ethnically-diverse to understand their comprehension and interpretation of questions. We will recruit participants through our team, colleagues and professional associations. The technique of cognitive testing is often used to prompt detailed explanations of participants’ feelings and experiences, comprehension, recall and judgement.⁵⁸ In individual interviews, we will ask questions about clarity, comprehension and interpretation of questions, the relevance of questions, missing content, time to complete the survey and feasibility of using the survey technology. We will modify the survey and invite participants to a focus group to discuss the findings and ask similar questions about the modifications.

Data analysis

For the quantitative study, we will compute descriptive statistics, frequencies and means and SD for all descriptive and outcome measures⁵⁹ from the survey to identify the key competencies and personal, interpersonal and cognitive capabilities underlying the high performance of FPs. The qualitative interview data on the educational experience of FPs will be analysed using Braun and Clarke’s six steps for thematic analysis.^{60 61} Thematic analysis will provide a detailed description of key ideas/messages by identifying commonalities in the data.⁶⁰ We will use open coding to read and code the survey data line-by-line. A codebook will be developed through a series of discussions with the research team. The codebook will be used for survey responses, which will be reviewed continually by the primary investigator. Next, the research team will review the coded survey responses independently and through a series of meetings to help identify the emerging themes of the similarities and differences in the results and educational strategies to improve the FM curriculum. NVivo V.12 software will be used to organise the data.⁶² We will also analyse by educators, programme directors and early-career FPs for high-performing FPs. The findings for these analyses will be compared with criteria identified from earlier studies of other professions.

Phase 2: workshops

Participants

The second phase of the study will invite university-based educators, professional associations, regulators and colleges to participate in workshops. Participants will be purposively selected based on their expert knowledge or experience with medical education and their role in the healthcare system (professional colleges, universities, regulators, associations). Participants will be identified through publications, reports and organisations that represent physicians (eg, Ontario Medical Association, Ontario College of Family Physicians, College of Physicians and Surgeons of Ontario, College of Family Physicians of Canada, Medical Council of Canada, Resident Doctors of Canada, Association of Facilities of Medicine in Ontario).

Our goal is to recruit 40 participants (at least 10 each from professional colleges, universities, regulators and associations) based on previous guidance on the number of participants to reach thematic saturation.^{63 64}

Data collection

The findings of the study’s first phase will be instrumental in informing the study’s second phase, which will focus on obtaining feedback and building consensus among stakeholders on the key competencies and capabilities of high-performing FPs and the educational strategies and assessment tools for curriculum improvement. Researchers will conduct two half-day workshops with consenting participants to test the veracity and implications of the data produced.⁶⁵ These workshops will take place virtually.⁶⁶ First, the primary investigator will present the findings from the study’s first phase. Next, a trained facilitator will lead the discussions on key findings and how FM curriculum design, teaching and assessment can be enhanced in Canada. The facilitator will monitor the group dynamics to ensure the views of all participants are adequately represented.

Field notes will be taken throughout the sessions.⁶⁷ Halfway through the meeting, the key ideas will be shared with key stakeholders, potentially stimulating further discussion until a consensus is reached. If an agreement cannot be reached, this will be noted. The workshop will be audiotaped and professionally transcribed for analysis.⁶⁸ Transcripts will be reviewed for accuracy by a member of the research team.

Data analysis

Transcribed data from the workshops will be analysed using Braun and Clarke’s six steps for thematic analysis.^{60 61} The data from the workshop transcripts will be reviewed and open-coded. The transcripts will be independently examined by the research team. Themes will be identified in a series of meetings on similarities and differences in agreement or disagreement with the capabilities and competencies of high-performing FPs and educational strategies to improve learning, teaching and assessment. In the final phase of analysis, we will triangulate the findings of the survey data with themes that emerged from the workshop data, which will provide a more detailed and balanced analysis.⁶⁹ Preliminary results will be sent to participants for feedback and ideas before final recommendations.⁷⁰ This work will culminate in an improved understanding of the normative competencies and capabilities needed to enable FPs to adapt to the many challenges and dilemmas of professional practice that will emerge over the coming decades in Canada.

Patient and public involvement

No patients or the public were involved in developing this study’s research question or design. Education stakeholders are part of the research team and will be engaged in different phases of the research.

Discussion

Effective and comprehensive primary care is one of the foundations of a high-performing healthcare system.^{71 72} Comprehensive care has been shown to reduce hospitalisation and overall system costs.^{73 74} Several factors shape the practice choices of FPs, including medical education and residency training. However, preparing medical graduates for practice is challenging for various reasons.⁶

This mixed-methods study will collect data through a combination of surveys and workshops in Ontario. The first phase will involve completing surveys with early-career FPs identified as performing effectively in the first 1–5 years of professional practice. The second phase will involve workshops with key stakeholders, including university-based educators, colleges, regulators and professional associations, to test the veracity of the results and identify optimum ways of building the findings into the curriculum and assessment. The first and second phases will identify the key capabilities and competencies required for adaptability for comprehensive FM and highlight the learning and teaching requirements for medical education in Canada.

The anticipated outcomes of this study include the following:

1. Determine if the criteria of high-performing graduates in other professions apply to FM.

2. The theoretical identification of the key capabilities and competencies required to confidently adapt to the independent practice of comprehensive FM.

3. To understand the extent to which these key capabilities are integrated into medical curricula and given focus in assessment at university and clinical placements.

4. To provide feedback on how to improve learning and teaching in various aspects and areas of medical education in Canada.

This work will contribute to the literature by augmenting the existing conceptualisations of competencies in FM by expanding on the theoretically driven key clinical and non-clinical competencies and personal, interpersonal and cognitive capabilities needed by FPs to adapt to changing healthcare landscapes. This theoretical advancement of knowledge has the potential to make higher education more relevant to FPs as well as those in other professions.⁵¹ Our long-term goal is to use the results from this pilot study to justify a national study across 12 jurisdictions in Canada to develop a validated capability framework for FM. This knowledge will inform recommendations on educational strategies for teaching, learning and assessing capabilities, including adaptability in FM. Following the completion of the national study, we will develop similar frameworks within medicine across other healthcare professions (ie, physician assistants, public health professionals, mental health workers). Our study can also inform future research on the long-term effects and relationships of high-performing FPs to further enhance our understanding of their contributions to the healthcare system.

Ethics and dissemination

This project has received ethics approval from the University of Toronto Research Ethics Board. All participants will sign written consent forms. Implied consent will apply to the return of completed questionnaires. Informed consent will also be sought from participants at the start of the workshop, whereby they will be reminded of their ability to decline participation and ability to withdraw at any point.

After the study, we will disseminate findings to international and national medical education and curriculum reform stakeholders through presentations. Many of our team members are educators and advisors to the College of Family Physicians of Canada and have a strong history of scholarship targeting university educators and the public. We will also conduct several webinars through our affiliated universities for a range of academic, community partners and public audiences, which will be available for download as podcasts. We anticipate presentations and publications to a range of FM and community knowledge users. The results of this study will be presented at leading Canadian medical education conferences (eg, the Canadian Conference on Medical Education and Family Medicine Forum), and we will seek publication in a leading peer-reviewed medical (ie, Canadian Medical Association Journal) and/or medical education (ie, Advances in Health Sciences Education; Medical Education) journals.

Ethics statements

Patient consent for publication

Not applicable.