Scholarly activity in Canadian Residency Matching Service criteria: do Canadian programs really care about applicant research?

Abstract

Background

Residency programs across Canada evaluate applicants based on written applications, reference letters, and interviews. One key factor many institutions consider is “scholarly activity.” To improve transparency, the Canadian Residency Matching Service (CaRMS) recently revised its program description pages, adding a dedicated section outlining expectations for research and academic work. This study examines whether these changes have made program criteria clearer for applicants.

Methods

For all 17 Canadian faculties of medicine, 2023 R1 entry, —internal medicine, family medicine, pediatrics, general surgery, psychiatry and anesthesiology—program descriptions were reviewed on the CaRMS website, looking for keywords related to scholarly activity.

Results

Although most residency programs now include scholarly activity in their CaRMS descriptions, several programs provide vague descriptions of this requirement. In 2023, nearly all family medicine (94%), internal medicine (100%), and pediatrics (100%) program descriptions referenced requiring or considering scholarly work as part of their selection process—up from 41%, 65%, and 71% in 2019. Programs commonly mentioned scholarly activity in two or three sections of their selection criteria, with key themes including active scholarly work, scholarly deliverables, and future scholarly potential.

Conclusion

Canadian medical schools should set clearer expectations for scholarly activity in residency applications to ensure transparency and equal opportunities for all applicants. Programs could also explain why research matters—whether as a core component of training or to develop critical thinking and initiative. Greater clarity would help applicants see research as more than just a “check-the-box” requirement, fostering genuine engagement in scholarly work.

Introduction

Canadian medical students face numerous challenges throughout their journey to becoming physicians, with one of the most significant being the residency matching process. In past years, this matching process, which determines the specific residency program—including the school, location, and specialty—where they will train, has become very competitive, with declining position to applicant ratio, with a nadir in 2018.¹ For example, across all specialties, there were 1.2 positions per applicant in 2010, compared to 1.08 positions in 2023.¹ The high stakes of matching places immense pressure on students, driving them to seek ways to optimize their applications and enhance their competitiveness throughout this process.

The Canadian Residency Matching Service (CaRMS) is a centralized application system responsible for facilitating the matching process between senior medical students and residency programs across Canada.² CaRMS has a website that includes information provided by each school’s residency programs regarding application requirements and preferences.² Matches are determined using a match algorithm that aligns the residency programs’ rankings of candidates and the candidates’ program rankings. Thus, in the current model, it is important that students make well-informed decisions about what programs best align with their interests and career goals.

Residency programs across Canada select applicants based on written applications, reference letters and interviews.² One aspect of a student's application package that many institutions consider when ranking candidates is “scholarly activity.”³ The Canadian Medical Education Directives for Specialists (CanMEDS) considers scholarly activity to be an essential competency of practising physicians.⁴ Scholarly activity is defined by CanMEDS as “a lifelong commitment to excellence in practice through continuous learning and by teaching others, evaluating evidence and contributing to scholarship.”⁴ The purpose of including scholarly activity as a requirement for residency program applications is to encourage balance, altruism and scholarly development, which are considered important competencies for future medical doctors.⁵

A 2015 review of 20 published studies found that engaging in scholarly activity increased a student’s likelihood of matching into their preferred residency program.⁶ In a 2020 Canadian study by Lukings and colleagues, the presence of scholarly activity in the 2019 CaRMS program's requirements was evaluated in three specialities: family medicine, internal medicine and pediatrics.³ The study revealed that 41% (7 of 17) of family medicine, 65% (11 of 17) of internal medicine, and 71% (12 of 17) of pediatric residency programs expressed interest in applicants with scholarly activities. Moreover, the study highlighted that the terminology used to describe scholarly activity on the CaRMS website was often vague and non-specific, making it difficult for applicants to discern which activities align best with program expectations.

Since then, CaRMS has restructured its program description pages, introducing a detailed table subsection for scholarly activity in which programs can opt to provide greater clarifications regarding scholarly activity expectations. Thus, this study aimed to evaluate whether the updated CaRMS structure has improved clarity in scholarly activity requirements. To achieve this, the previous study was replicated, and the results were compared. Additionally, the scope of the current study was expanded to include psychiatry, general surgery and anesthesiology residency programs across Canada, providing a broader analysis of programs varying in size and discipline.

Methods

Study design

Following up on previous work, we performed a cross-sectional study to characterize the research suggestions present in the 2023 CaRMS website descriptions for six R1-entry programs.³ While the previous study assessed three specialties (internal medicine, family medicine, and pediatrics) across all 17 Canadian medical schools, we elected to expand on this to further include general surgery, psychiatry, and anesthesiology. General surgery was chosen as we wanted to evaluate a surgical specialty with a larger number of residency positions. Psychiatry was chosen as it bridges medical and psychosocial approaches to care, which offers a valuable contrast to more procedural or biomedical specialties. Anesthesia was chosen for its unique interdisciplinary position, interacting with surgery, critical care, internal medicine, obstetrics, and emergency medicine. Ethics board approval was not required for this study as all the data were publicly available on the CaRMS website.

Data collection

For the six R1-entry programs, the authors collected data by visiting each of the 17 respective Canadian medical schools’ program pages on the CaRMS website.² In cases where a medical school offered multiple streams for a single R1-entry program, the Canadian medical student stream with the greatest number of seats was assessed.

For each school’s program page, the authors identified relevant descriptions referring to scholarly activity. Equivalent French terms were also identified. A complete codebook containing all captured terms is included in Appendix A. Importantly, only the subsections ‘Selection Process Goals’, ‘File Review Process’, and ‘Interview Process’ within the CaRMS program page were assessed, as other sections do not provide information regarding a program’s selection criteria.

Following a pilot extraction to ensure consistency across reviewers, data were collected by six authors (AE, HV, NC, CM, AP, SD). Extraction for all medical schools’ program descriptions was performed in duplicate by two independent reviewers. In cases of discrepancies between the two reviewers, the rest of the group was consulted, and a consensus was reached.

Data analysis

Quantitative Analysis. The frequency of scholarly activity-related language in each of the residency program CaRMS website descriptions was summarized by specialty and region. To avoid isolating and revealing the data from individual institutions, the 17 medical schools were grouped into four categories: Western Canada, Ontario, Quebec, and Atlantic Canada. This was also performed to remain congruent with the previous study, thereby facilitating direct comparisons across time.³ Within these geographic categories, data were presented as percentages of programs to include references to scholarly activity across each of the six R1-entry specialties.

The extent of each program’s research requirements was further explored by analyzing the three subsections (‘Selection Process Goals,’ ‘File Review Process,’ and ‘Interview Process’) within the stated program description. For each subsection, the authors assessed for the presence of descriptors of scholarly activity requirements. Programs were then scored based on how many of these three subsections included at least one reference to scholarly activity, and analyses were performed by comparing findings across specialties and geographic categories.

Qualitative analysis. We conducted the qualitative assessments upon completion of the data extraction; all reviewers independently evaluated the dataset to identify recurring descriptors, which informed global themes across all R1-entry specialties. The reviewers subsequently met to establish a consensus. Based on collective agreement, groupings of common scholarly activity descriptors and themes were established.

Findings

Quantitative

Over 95% of programs (16/17) across all six specialties referenced scholarly activity in their CaRMS program descriptions. However, exceptions were noted both among family medicine programs in the Western region and general surgery programs in Ontario, where only 80% included mentions of scholarly activity (Figure 1; Figure 2).

Figure 1.

Percentage of programs (family medicine, internal medicine, and pediatrics) that mentioned scholarly activity by geographic region in 2023.

Figure 2.

Percentage of programs (anesthesia, general surgery, and psychiatry) that mentioned scholarly activity by geographic region in 2023.

When comparing the three specialties investigated in 2019 (family medicine, internal medicine, and pediatrics) to 2023, more programs mentioned scholarly activity across all three specialties.³ In 2023, 94% (16/17) of the family medicine, 100% of the internal medicine and 100% of pediatrics programs mentioned scholarly activity compared to 41% (7/17) of family medicine, 65% (11/17) of internal medicine, and 71% (12/17) of pediatrics programs in 2019 (Figure 3).

Figure 3.

Comparison of programs (family medicine, internal medicine, and pediatrics) that mentioned scholarly activity in 2019 vs 2023.

When analyzing the number of programs that mentioned scholarly activity in none, one, two, or all sections (‘Selection Process Goals’, ‘File Review Process’, and ‘Interview Process’) on the CaRMS description pages across specialties, the majority of programs mentioned scholarly activity in two or three sections (Table 1). For family medicine, one program did not mention scholarly activity at all, while most programs mentioned scholarly activity in one or two sections. For general surgery, two programs did not mention scholarly activity.

Table 1.

Number of programs (n = 17) that mentioned scholarly activity in none, one, two, or all sections – ‘Selection Process Goals’, ‘File Review Process’, and ‘Interview Process’–on CaRMS program description pages.

Specialty	Zero Sections	One Section	Two Sections	Three Sections
Family medicine	1	6	8	2
Internal medicine	0	2	9	6
Pediatrics	0	1	10	6
Anesthesia	0	2	8	7
General Surgery	2	2	6	7
Psychiatry	0	2	7	8

Qualitative

When analyzing qualitative data, four keywords were consistently found to be used to describe or refer to scholarly activity: research, scholarly, academic, and scientific. For each keyword, there were various modifiers that further contextualised details of the scholarly requirements. These modifier words were found to fit into three themes: Active Work, Deliverables, and Future Desire (Table 2).

Table 2.

Summary of keywords and modifiers used to describe the three identified themes in CaRMS program description pages.

Keywords	Major Themes Identified
	Active Work	Deliverables	Future Desire
“Research”	Experience Involvement Productivity Commitment Engagement	Achievements Publication Presentation Productivity Conference	Interest Potential Aptitude
“Scholarly”	Experiences Work Activities Pursuits CanMEDS*	Impact Scholarship CanMEDS*	Interest Inquiry Lifelong learning Potential CanMEDS*
“Academic”	Work Commitment Performance Pursuits	N/A	Inquiry Potential Aptitude
“Scientific”	Involvement Rigor	Achievements	Curiosity

Theme 1: Active work

Active work referred to any past or ongoing research endeavours. For example, program descriptions used phrases such as:

Evidence of commitment and engagement in research, medical education, leadership and/or extracurricular activities. (Pediatrics, File review process)

We are evaluating the applicant’s past scholarly activities. (Pediatrics, File review process)

These examples contain the keywords “Research” and “Scholarly” to describe that programs are evaluating scholarly work. Modifiers such as “commitment,” “engagement,” and “activities” were used to further contextualise this work to be active.

Theme 2: Deliverables

Deliverables included any tangible outcomes from any past or current scholarly activity. Throughout the various program description sections, but particularly within the “File Review” section, phrases are used such as:

Research projects/publications pertinent to family medicine. (Family medicine, File review process)

Research productivity during medical school including research projects, presentations and publications will also be considered. (General surgery, Selection criteria)

The keyword “Research” was present to indicate scholarly related work, and modifiers such as “publications”, and “presentations” were used to specify the tangible outcome.

Theme 3: Future desire

Future desire referred to the willingness of candidates to participate in scholarly work during residency. Examples include:

Scholarship potential including interest in participating in future research. (Psychiatry, File review process)

Demonstrates interest in research participation. (Internal medicine, File review process)

Demonstrates interest in building scholarship in education and/or research during training. (Internal medicine, Interview process)

Intellectual curiosity, an interest in academic and research related endeavors… are also highly prized in our Program. (Anesthesia, Selection criteria)

These examples use the keywords “academic” and “research” along with modifiers such as “interest,” “potential,” or “curiosity” to encourage candidates to demonstrate a future desire to participate in scholarly activity.

Discussion

Our findings reveal that nearly all 2023 Canadian R1 residency programs in the six specialties examined—family medicine, internal medicine, pediatrics, anesthesia, general surgery, and psychiatry—reference scholarly activity in their program descriptions. Exceptions to this pattern are one family medicine program in Western Canada and three general surgery programs in Ontario. These findings suggest that most of the Canadian residency programs we studied seek information about scholarly activity, evidence, or interest when evaluating applicants.

Interestingly, our analysis reveals that many programs reference scholarly activity across multiple sections of their descriptions—including “Selection Process Goals,” “File Review Process,” and “Interview process”—rather than limiting it to a single section. The variation in frequency and placement of scholarly activity mentions across sections may influence applicants’ interpretation of the associated requirements. For example, references in the “File Review Process” may suggest its importance for interview consideration, while mentions in the “Interview Process” could reflect its importance in final matching decisions. However, the exact significance of scholarly activity in an applicant’s overall CaRMS application remains unclear.

This study builds on a 2019 review, which found that scholarly activity was more commonly considered in internal medicine and pediatrics programs relative to family medicine.³ Our study’s findings show an overall increase in mentions of scholarly activity, with no notable differences between specialties. This aligns with existing literature that indicates that both applicants and program directors increasingly indicate that they value scholarly activity.^7-12 Reports show that program directors use metrics such as publications and research experience as indicators of academic potential, a strong work ethic, and commitment to a specialty.¹³ This perception of scholarly activity as a key factor in residency competitiveness has driven an increasing number of students to dedicate time to research, including taking research years.¹⁴

Despite the perception that scholarly activity improves competitiveness, there is contradicting evidence as to whether applicants who demonstrate evidence of scholarly activity perform significantly better in the match process.^6,15-16 For instance, a 2017 study of a pediatrics program found no significant association between pre-residency scholarly activity—such as publications, post-graduate degrees, and presentations—and matching success.¹⁶

Although residency programs increasingly reference scholarly activity in their descriptions, its role in the CaRMS match remains unclear. The term itself lacks a consistent definition and is often conflated with scholarship, despite important differences.^17-20 This ambiguity may be contributing to the vague or inconsistent language used in many program descriptions, making it difficult for applicants to gauge expectations. For example, a 2021 cardiac surgery survey revealed a mismatch between applicants’ and selection committees’ perceptions of how many publications were considered competitive.²¹ This discrepancy may lead students to overemphasize certain forms of scholarly output. Alternatively, selection committees may underemphasize other valuable activities.

This mismatch in perception may also help explain the marked rise in scholarly activity among matched applicants over the past 15 years.²² In our analysis (Table 2), references to scholarly activity fell into three themes: Active Work (ongoing efforts), Deliverables (e.g., publications), and Future Desire (interest or potential). Of these, Deliverables were the most transparent, with explicit statements such as “research projects/publications pertinent to family medicine.” The other categories remained open to interpretation, potentially fostering uncertainty and, in some cases, prompting applicants to list superficial or exaggerated achievements to appear competitive.^23–26 Simpson and colleagues argue that applicants can demonstrate scholarly qualities aligned with the CanMEDS framework without engaging in formal scholarship.¹⁷ Thus, clearer, standardized descriptions of valued scholarly attributes could help applicants better prepare, engage meaningfully in scholarly activity, and reduce the hidden curriculum that disadvantages first-generation and non-native English-speaking applicants.

Moving forward, Canadian medical schools should provide specific expectations for scholarly activity on application sites to increase transparency in CaRMS requirements. In our view, engagement in research should be considered valuable not solely for producing specialty-specific knowledge, but for cultivating generalizable skills that support evidence-based clinical practice. As medicine continues to evolve toward an evidence-informed model, the ability to critically appraise and apply literature is essential across all specialties. Even for students who do not pursue academic careers, experiences working with research promote habits of inquiry and an orientation toward continuous improvement. These competencies are broadly transferable and support quality care across diverse settings. Therefore, residency programs should be transparent about why they value scholarly activity: is it to signal commitment to a specific discipline, to foster intellectual skills relevant to any physician’s practice, to build specific skills that will be used in residency and future practice or other reasons? Additionally, programs may consider a minimum level of engagement in scholarly activity to demonstrate these valuable skills, rather than allowing the volume of research completed to influence their rankings. This transparency will ensure equal opportunities for success for all applicants. Finally, greater clarity would help applicants understand how research skills align with program goals, discouraging the perception of research as a “check-the-box” requirement, but rather, fostering genuine interest in scholarly pursuits. We urge Canadian residency programs to disclose their reasoning for evaluating scholarly activity, methods for assessing scholarly activity, and how this affects candidates’ rankings.

This study was limited to six specialties and relied solely on standardized CaRMS program descriptions. Investigation of additional information on associated university websites may provide greater transparency. Furthermore, the analysis was subject to the researchers' interpretation of the program descriptions, which may differ from other individuals.

Conclusion

This study demonstrates that scholarly activity is increasingly emphasized in Canadian residency program descriptions, with nearly all programs across six specialties referencing it in their CaRMS pages. While the introduction of a standardized template in 2020 has improved consistency, the language used to describe scholarly activity remains vague and open to interpretation. There is an increasing incongruence between applicants’ and selection committees’ valuing of scholarly activity. Clearer definitions and explicit expectations regarding scholarly activity and transparent discussion of how this is considered in the ranking process are essential to ensure equitable access for all applicants and to guide them in aligning their experiences with program expectations. By providing transparency and rationale, residency programs can help applicants move beyond viewing scholarly activity as a mere requirement and instead foster genuine engagement in academic pursuits. We call on Canadian residency programs to explicitly describe why scholarly activity is valued by the faculty involved in the selection process (that is, if it is indeed valued), as it is currently unclear to applicants who are hoping to be successful in the next step of their career. Greater transparency will help demystify the residency selection process, ultimately benefiting both applicants and programs.

Appendix A. Codebook Research productivity

Clinical research

Translational research

Research activity/activities

Interest in research

Research experience

Research potential

Undergraduate research

Research endeavours

Research

Researcher

Basic research

Research opportunities

Research interest

Scholarly activity/activities

Scholarly impact

Scholarly project

Scholar

Scholarship

Scholarly interest

Manuscript

PhD

Quality and safety

Regueur scientifique/scientific rigor

Abstract

Publication

Poster

MSc

CanMEDS

Conference

Quality improvement

Presentation

Project

Experience with methodology

Curiosité scientifique/scientific curiosity

Funding Statement

Funding: None

Conflict of Interest

The authors do not have any conflicts of interest due to financial or personal relationships that could bias their work.

Authorship

Authors El Galad and Van Dierdonck contributed equally to this work as first author.