Uncharted territory

Knowing is not enough; we must apply. Willing is not enough; we must do.

Goethe

Much of the recent discussion around CanMEDS–Family Medicine (CanMEDS-FM) competency-based curriculum design and development focuses on residents: how to implement and evaluate CanMEDS roles in family medicine residency programs.^1–6 To our knowledge there are no scholarly studies examining how family physicians demonstrate the CanMEDS-FM competencies on a day-to-day basis in the workplace. In short, we suggest that family medicine has yet to systematically generate any evidence of actual assessments of family physicians’ performance within the context of the CanMEDS framework as part of continuing education. Further, family medicine has not developed or evaluated any best-practice implementation models and approaches to delivering the CanMEDS-based performance assessment once in practice. With the recent drive to create and implement competency-based medical education in continuing professional development (CPD) to improve the quality of care and patient safety, this lack of evidence could seriously hinder assessment and implementation practices well into the future. This is troubling given what is known about the importance of workplace-based assessments in which core competencies have to be articulated and subsequently measured across the learning continuum.⁷ We suggest that the paucity of literature assessing the CanMEDS-FM competencies of practising family physicians requires our close attention now, and that there are still many questions that need to be answered (Box 1) before the wave of implementation activities truly commence in earnest across Canada.

Box 1. Questions to consider.

To understand the factors that enable or inhibit the use of CanMEDS-FM competency-based assessments in routine practice settings, the following questions need to be answered:

• What is the prevalence of competency-based assessments in Canadian family practices across the country?

• How can CanMEDS-FM roles be incorporated into or be the foundation for assessment strategies?

• What factors facilitate or hinder the implementation of competency-based assessment systems in the family medicine workplace context?

• Are all CanMEDS-FM roles appreciated and valued in terms of physicians’ approaches to CPD in family medicine?

• Are competency-based assessments in the workplace effective at altering family physicians’ behaviour? What is the mechanism by which we expect a competency-based assessment to change their behaviour? How do we best measure clinical behaviour patterns here?

• What are the specific strategies involved in this type of assessment? Do those strategies have evidence for changing practice? Are there opportunities for incorporating other underused but potentially effective strategies based on evidence from other literature sources?

CanMEDS-FM—CanMEDS–Family Medicine, CPD—continuing professional development.

Evaluating CanMEDS competencies in medical practice

Designing, implementing, and maintaining workplace assessment, including tools for multisource feedback and audit and feedback, is pivotal for ensuring lifelong learning, enhancing the quality of doctors’ performance, and identifying underperforming physicians.^8–12 Further, the assessment of physicians’ performance aids in achieving the highest possible standards of clinical practice, thus improving the performance of health care organizations.^9,13 Performance measures at the micro level (ie, individual clinician performance) integrated with performance measures at the meso level (ie, family practice context) reflect and affect the overall health system performance (ie, the macro level).⁸

In spite of this widely held assumption about the systemic interconnectedness of quality, there are concerns about how little attention is devoted to evaluative research, especially to the evaluation of competencies in medical practice,^14,15 even though there is a recognition that such competency frameworks are difficult to implement.^16,17 In addition, given the importance of workplace assessment, namely multisource feedback and audit and feedback, as a method of formative performance evaluation, which can be effective but varies widely in terms of improving professional practice and clinical behaviour,^9,10,18 there have been few attempts to use competency-oriented testing to assess the performance of practising physicians in an organized and systematic manner.^16,17 For example, recent evaluative studies focusing on the assessment of family physicians’ performance demonstrate assessment of competencies, but they are not explicitly within the context of the CanMEDS-FM framework.^19,20

In essence, we are concerned with the question “How does outcome-based education work, for whom, and in what circumstances?” which in itself is not new.²¹ But if we want to know how close we are to the goals of CanMEDS-FM competency-based medical practice and if we want to train physicians who deliver high- quality health care, there is a time-limited opportunity for the education community to embrace the importance of evaluating its educational interventions across the entire learning continuum to include CPD.^15,22 By using a variety of evaluation approaches, we suggest it would be important to address the following question: “Does competency-based medical education work and how, why, and when?” To do this, we need to turn to knowledge translation and implementation (KTI) research as a conceptual, methodologic, and evidence-based resource.

Focusing on practice change

The ultimate goal of CPD is to implement new clinical behaviour and to eliminate or de-implement outdated, non–evidence-based, and “low value” practices to improve patient outcomes.²³ However, answering a key question has proven to be a considerable challenge²⁴: “How do we go about changing the existing practice patterns of physicians, thereby providing high-quality care consistent with best evidence and reducing medical errors to a minimum?” Before we can determine how to successfully change behaviour, we must be able to measure physician behaviour in the clinical setting.⁸

There are a number of related fields that focus on practice change (eg, knowledge translation research, patient safety, and quality improvement). In this article, we use the term knowledge translation and implementation as an umbrella term to cover these fields. Monitoring professional behaviour in situ and providing physicians with detailed comparative feedback including clinical performance data has been shown to serve as a stimulus for changing their practices.¹⁸ For example, several Cochrane reviews suggest that educational interventions (eg, educational outreach visits, educational meetings, printed educational materials), as well as audit and feedback and reminders, change professional behaviour,^10,25–27 but with considerable variability between trials.

Some have argued that substantial progress cannot be made until we have built a better theoretical foundation to understand the determinants of professional behaviour,^28,29 to clarify the hypothesized mechanisms of behaviour change,³⁰ and to promote a cumulative knowledge using shared language. There are now many theories borrowed from disciplines such as psychology, sociology, and organizational theory, as well as theories, models, and frameworks that have emerged from within implementation science to gain insights into the mechanisms by which implementation is more likely to change behaviour.^18,30–34 In spite of this, the medical education community has yet to effectively implement a knowledge translation approach to develop and deliver educational interventions that would successfully change clinical behaviour. Clearly, opportunities for novel approaches in CPD that incorporate and build on KTI need to be explored, as has been previously suggested,^35–39 but which has yet to be acted upon in the domain of family medicine.

Conclusion

Continuing professional development and KTI science share a common goal: to improve medical and health professional practice and hence patient care and population health outcomes.^39,40 However, the pathways to reach this goal differ,³⁹ most notably in terms of the much broader focus of KTI science. Knowledge-to-action models emphasize identifying barriers to desired practice (including but not limited to individual health care professional education needs) and tailoring interventions to addressing these barriers. Professional education is only one potential solution, addressing a specific set of potential barriers that are usually incomplete (eg, we need education, plus work process change, plus system change).

Given the fact that “knowledge to action” is complex and challenging,⁴⁰ closer collaboration among researchers in the areas of continuing education, knowledge translation, patient safety, and quality improvement appears desirable if not essential.^41,42 Integration of efforts across these 4 domains, as well as efforts within stakeholder organizations responsible for each level of medical training and practice, should aim to develop, implement, and rigorously evaluate the implementation of more holistic and successful interdisciplinary interventions to improve professional performance and patient outcomes.^12,41–43 We suggest that implementing these kinds of interventions will result in more evidence-based and observable practice mirrored in changes in professional behaviour patterns and quality indicators.²⁴

As the CanMEDS framework is just beginning to be implemented, this is an exciting, albeit time-limited, opportunity to increase our efforts to see whether our interventions do result in changes in physician practice in the context of the drive to incorporate CanMEDS–FM competencies into CPD interventions among family physicians. However, we still need to answer many questions (Box 1) in order to have a complete understanding of the factors that enable and inhibit the use of CanMEDS-FM competency-based assessments in routine practice settings.