Replacing high-stakes summative examinations with graduated medical licensure in Canada

KEY POINTS

• Disruptions to medical licensing exams in Canada during the COVID-19 pandemic have raised an opportunity for reexamination of processes for licensure.

• No evidence supports the effectiveness of current licensure exams in ensuring patient safety; the current process causes undue stress and directs valuable and limited resources toward passing an exam, potentially detracting from the more meaningful goal of preparing for independent practice.

• Best practices in medical and continuing education support the use of competency-based assessments, guided by robust programmatic assessment models, to judge fitness to practise, followed by life-long self-directed learning.

• Credentialing and licensing authorities should advocate for the resources required to replace current high-stakes summative assessments with graduated licensure and to develop quantifiable, nationally synergized, specialty-specific practice standards that support both the oversight of graduated licensure and maintenance of competence.

The COVID-19 pandemic has disrupted the medical assessment system in Canada. Examinations delivered by The College of Family Physicians of Canada (CFPC) and the Royal College of Physicians and Surgeons of Canada (RCPSC) were delayed, cancelled or adapted. As these examinations are required for independent practice, this had a negative impact on trainees, supervisors and patients during a time of great stress within our health care system.¹ Although disruptive, these challenges provided an opportunity for change by unfreezing the historical approach to medical licensure in Canada — an approach that can be characterized as arduous, expensive, logistically challenging, poorly aligned with clinical practice and potentially biased.

The current medical licensure practices in Canada depend on high-stakes, standardized, summative examinations that were developed to uphold the medical social contract to guard patient safety and benefit society.² However, no evidence has shown that these exams contribute to this outcome. Although psychometrically sound exams provide superficial reassurance as a safety net that “catches” trainees who are not prepared for independent practice, the incredibly high pass rates of graduates from Canadian training programs suggest that this function is largely redundant.³ In contrast with this approach, best practices in medical and continuing education support the use of competency-based assessments, guided by robust programmatic assessment models. Programs guided by these models employ frequent, low-stakes assessments within the clinical environment, along with specialty-wide or internally created local exams; such programs are being implemented throughout the medical education system,⁴ and have resulted in increased assessment volume⁵ and quality.⁶ Although further evidence of competence in clinical practice may be required to maintain the social contract, licensing exams, as currently structured, do not provide this evidence.

The ongoing use of high-stakes examinations for licensure has numerous unintended consequences. Standardized exams test nonsalient variables and are at risk of biases (e.g., financial hardship from paying to write and attend examinations, structural racism affecting examination literacy and preparation) that may inhibit the movement of our health care institutions toward equity, diversity and inclusion.⁷ The medical licensing process directs valuable and limited resources (e.g., time, energy, focus) toward passing an exam, potentially detracting from the more meaningful goal of preparing for independent practice. Moreover, the focus on a single high-stakes examination does not support the development of lifelong, self-directed learning. These exams can also increase stress among trainees to a degree that threatens mental health and wellness.⁸ Lastly, the exams remain painfully expensive for both the trainee and the regulatory body, at a time when student debt continues to soar.

Although landmark changes have been made in response to vigorous advocacy and debate,⁹ including the removal of the Medical Council of Canada’s second qualifying examination from licensure requirements, passionate voices urge further modernization to increase flexibility and alignment with intended outcomes. A modern model of assessment for licensure that integrates both training programs’ competency-based assessments and clinical practice data derived from trainees’ clinical work would better align with evidence-informed educational practices, underscore the importance of continuous quality improvement for practitioners and refocus licensure on real-world clinical practice.^10,11

The implementation of such a system would be challenging. Most international medical training programs use high-stakes, gatekeeping examinations similar to those used in Canada, so cannot provide robust examples to guide the way forward. Fortunately, the scaffolding for such a model already exists in the Competence By Design assessment programs (RCPSC), the Triple C Curriculum (CFPC) and the Core Professional Activities of the Residency Training Profile (CFPC). The direct observation of real-world clinical activities within these training programs occurs frequently and, when competency-based programs are implemented with fidelity, they provide substantive evidence of competence in the clinical setting. The formalization of “transition to practice” periods within these training programs could occur if this evidence was used to grant a graduated licence that allowed trainees to take on greater clinical autonomy. Trainees with graduated licences would have increased ownership of clinical outcomes that could be tracked through practice audits, supported by their training program and hospital system. The incorporation and oversight of such quality metrics would engage individuals in continuing to employ lifelong learning skills while supporting quality improvement practices within their discipline.

Developing and socializing the cultural changes and quality improvement infrastructure required for this model would be resource intensive. Culturally, the creation of a graduated licence would need to be implemented without increasing medicolegal risk or decreasing compensation for supervisors. The successful socialization of this approach would offload some patient care responsibilities to the trainees with graduated licences, thereby allowing their supervising consultants to develop and oversee their practice audits. The oversight of clinical practice during this stage would require increased sophistication in how health care systems report clinical outcomes and conduct quality-improvement initiatives. These costs could be contained through other synergizing innovations. For example, nationally developed, locally implemented, discipline-specific standards for practice audits could be incorporated into both maintenance-of-competence programs and the assessment of a graduated licensure period.

Logistically, the implementation of these changes would require substantial collaboration between the CFPC, RCPSC and provincial regulatory bodies. From a credentialing perspective, specialty-specific examination boards would need to be replaced with boards trained to review standardized practice audits at the end of the graduated licensure period. Additionally, new accreditation standards would need to be developed to ensure adequate oversight and support of trainees during the graduated training period. From a regulatory perspective, provincial medical licensing bodies would need to evolve to define and supervise a form of graduated licensure that provides trainees with greater clinical autonomy. If these changes were made collaboratively across Canada’s provinces and territories, they could facilitate the harmonization of provincial licensure standards, or even the development of a portable national licensure that would improve physician portability.¹²

By disrupting traditional approaches to examination, the COVID-19 pandemic has provided an opportunity to imagine alternative approaches to licensure that better align with best practices in education assessment, while also upholding the medical social contract. Credentialing and licensing authorities should advocate for the resources required to replace the current high-stakes summative assessments with graduated licensure; to develop quantifiable, nationally synergized, specialty-specific practice standards that support both the oversight of graduated licensure and maintenance of competence; and to streamline licensure requirements between jurisdictions.