Canadian urology resident scholarly performance

Abstract

Introduction:

Scholarly research is a key component of Canadian urology residency. Through comparison of scholarly performance of urology residents before residency with that achieved during residency, we aimed to elicit predictive factors for completion of research activities.

Methods:

Electronic surveys were sent to 152 urology residents of 11 accredited Canadian programs. Survey questions pertained to post-graduate training year (PGY), formal education, scholarly activity completed before and after the start of residency, protected/dedicated research time, structured research curriculum and pursuit of fellowship training.

Results:

Surveys were completed by 42 residents from 10 programs. Only 26% of residents had a structured research curriculum, 38% a dedicated research rotation and 43% protected research time. We found that 45% of residents published at least 1 manuscript so far during residency (mean 1.14 ± 0.32), and 43% submitted at least 1 manuscript (mean 0.86 ± 0.25). During residency, 62% of residents completed ≥1 formal research presentation (median number 1.5; range: 0-≥10). Only the level of PGY significantly affected the number of manuscripts published (p < 0.001) and number of formal research presentations (p < 0.001) completed during residency. In total, 86% of residents planning to pursue fellowship training had a mean number of publications and presentations during residency of 1.25 ± 0.37 and 2.25 ± 0.54, respectively.

Interpretation:

Level of PGY significantly affected quantitative scholarly activity, but the numbers and types of presentations performed prior to residency, completion of an honours or graduate degree and plans to pursue fellowship training did not.

See related article on page e448.

Introduction

Research activities are an integral component of urology residency programs in Canada. Since the adoption of the Canadian Medical Education Directives for Specialists (CanMEDS) framework of essential physician competencies by the Royal College of Physicians and Surgeons of Canada (RCPSC), the objectives of the CanMEDS “Scholar” have become a mandatory component of all accredited residency programs and objectives of training.¹ The “Scholar” objectives specify that upon completion of the specialty program each resident should be able to pose a research question, apply appropriate methods to address the question, and disseminate the research findings.^1,2 However, there are no set standards by the RCPSC.

Most Canadian academic urology programs have internal expectations for achieving these “Scholar” objectives by evaluating the quantity and quality of research presentations and peer-reviewed publications. Program directors complement these accomplishments in scholarly performance with in-training evaluation reports (ITERs) that focus on other aspects of this CanMEDS core competency.

Through comparison of scholarly performance of urology residents before residency with that achieved during the residency training period, our study aimed to elicit predictive factors for successful completion of research activities during residency. Since urology program structures vary across the country, we also examined the relationship between scholarly performance and protected research time, presence of a dedicated research rotation, existence of a structured research curriculum and access to a biostatistician.

Methods

After full institutional ethics board approval, program directors of all 13 accredited Canadian urology training institutes were approached with e-mail invitations for participation and permission to approach their current urology residents. Anonymity was guaranteed to program directors and residents. Institutions that did not respond were sent the invitation 2 additional times and, if necessary, 2 further attempts were made to contact them by telephone. Following approval by each institution, the consent form and link to the electronic survey were forwarded by the respective program directors to 152 current urology residents in all years of training. Two reminder emails were sent to each resident over the study period (August–September 2011).

Select Survey (ClassApps.com, Overland Park, KS), accessed through Nova Scotia Health Information Technology Services, was used to create an electronic survey entitled “Urology Resident Scholarly Performance Survey.” Survey questions pertained to current year of post-graduate training, formal education completed prior to residency (e.g., Bachelor’s, Honours, Master’s, PhD degrees), and scholarly activity completed before and after the start of residency training (i.e., research presentations and publications). Publications of all levels of authorship were included. We examined the following factors: amount of protected research time, existence of a dedicated research rotation, research expectations of the program, access to a biostatistician, program funding, the existence of a structured research curriculum and the pursuit of fellowship training.

Descriptive statistics were used to characterize the study population. Generalized linear regression analysis was used to evaluate between the numbers of manuscript submissions and publications during residency versus the numbers of pre-residency submissions and publications, completion of an honours or graduate degree, plans to pursue fellowship training and variation in urology program structure. Multivariate exploratory logistic regression analysis and Hosmer and Lemeshow goodness-of-fit test were used to analyze the effect of the number of manuscripts submitted and published during residency on the decision to pursue fellowship training, factoring in the conduct of prior research presentations and number of publications, level of prior educational training and variation in urology program structure. Incomplete surveys were excluded from data analysis. Calculations were performed using the SAS statistical package (SAS Institute Inc., Cary, NC).

Results

Eleven Canadian programs participated in this study and electronic surveys were sent to 152 residents in all 5 years of urology residency training. In total, 42 residents (28%) from 10 (77%) training programs completed the entire survey (Table 1).

Table 1.

Sample characteristics

	No. total residents (%)
Post-graduate year
1	11 (26.2%)
2	8 (19.1%)
3	7 (16.7%)
4	8 (19.1%)
5	8 (19.1%)
Total no. residents	42 (100%)
Undergraduate degree
None	1 (2.4%)
BSc	30 (71.4%)
BMSc	3 (7.1%)
BBA	1 (2.4%)
B Med & Surg	1 (2.4%)
BEng	1 (2.4%)
BHSc	4 (9.5%)
MB BCh BAO	1 (2.4%)
Honours degree	29 (69.1%)
Graduate degree
Master’s	4 (9.5%)
PhD	2 (4.8%)

Thirteen residents (31%) published at least 1 article before entering medical school. The mean number of articles published during this time was 0.67 ± 0.23, including 1 individual with a PhD who published 8 articles. Of these 13 residents, 12% completed a first-author manuscript and 21% an intermediate-authorship paper (Fig. 1).

Fig. 1.

Completed scholarly performance at each stage of training.

Twenty-seven residents (64%) had at least 1 article accepted or pending before the start of residency (mean number of articles 1.02 ± 0.15. Among these authors, 37% completed a cohort study, 30% case report, 15% basic science paper and 11% case series. Of the total residents, 55% prepared a first-author manuscript and 17% an intermediate authorship by this stage of training.

Thirty-five (83%) residents were currently involved in research with most projects predominantly clinically based (Fig. 2). Nineteen (45%) had published at least 1 manuscript during their residency (mean 1.14 ± 0.32), 38% as first- and 12% as intermediate-author. There were 18 residents (43%) who had at least 1 article currently accepted or with acceptance pending (mean 0.86 ± 0.25). The types of manuscript are shown in Fig. 3.

Fig. 2.

Type of active research involvement during residency.

Fig. 3.

Manuscript production during urology residency.

Thirty residents (71%) had formally presented research prior to the start of residency (median number of presentations 1.5; range: 0–≥10). Two completed ≥10 presentations, including 1 resident with a PhD. Overall, 60% of residents completed a poster and 48% a podium presentation prior to residency. In comparison, 62% of total residents completed at least 1 formal presentation so far during their residency (median 1.5; range: 0–≥10), with 45% completing a poster and 43% a podium presentation (Fig. 1).

Of the residents surveyed, 86% indicated they intended to pursue fellowship training. Of these, the mean number of publications and presentations so far during residency was 1.25 ± 0.37 and 2.25 ± 0.54, respectively.

Pertaining to urology program structure, 98% of residents had funding available to attend conferences and meetings. In total, 83% had access to a biostatistician to assist with research analysis, but in 81% of these instances this individual was external to the urology program. Only 26% of participating urology residents indicated a structured research curriculum, 38% a dedicated research rotation and 43% protected research time.

Generalized linear regression analysis revealed that only post-graduate training year significantly affected the number of formal research presentations (p < 0.001) and number of research publications completed during residency (p < 0.001). The number and type of research presentations performed prior to residency, number and type of research publications completed prior to residency, completion of an honours or graduate degree and plans to pursue fellowship training did not significantly affect the quantity of scholar activity preformed during urology residency.

Discussion

The literature regarding urology resident scholarly performance is limited since investigators have sampled mainly chief or graduated residents,^3,4 examined only first authorship manuscripts or limited study cohorts to the top 50 American urology hospitals.⁵ We report the scholarly performance of Canadian urology residents, including an analysis all 5 years of residency, research publications of all authorship and successful completion of research presentations.

Scholarly research is a key component of urology residency and viewed with importance by residency committees in the annual Canadian Residency Matching Service (CaRMS). Within the CaRMS application, candidates indicate their research activities, publications and research presentations. This provides valuable information of a candidate’s research interest, previous research performance and possible future research potential. However, we found no correlation between number and type of research publications completed prior to residency or volume of manuscripts completed prior to residency and scholarly activity during residency. These findings are in agreement with some previous reports⁶ and in disagreement with others.³ Neither completion of an honours nor graduate degree correlated with increased research productivity in our study. Publishing record of candidates has not been shown to correlate with clinical performance.⁷

A previous analysis of Canadian and American chief urology residents and recent graduates indicated most residents submit and publish at least 1 first-author paper in a peer-reviewed journal during residency.³ We found only 45% of residents published at least 1 manuscript during residency and 43% had at least 1 manuscript submitted at the time of our study. These numbers appear low but could reflect the fact that 45% of our respondents were junior residents. Although most residents (83%) surveyed indicated current involvement in research, some junior residents may not have completed a research project or were possibly not yet involved in a research project, as the time of our study was early in the academic year. We did find that post-graduate year of training was positively correlated with scholarly performance during residency. This seems logical because as a resident advances through a training program, they enhance their research resumé, either secondary to personal motives and/or program expectations/encouragement. Residents performed more formal research presentations compared to manuscript submissions, which could reflect the low publication rate of presented abstracts following attendance at urological meetings.^8–12

Our low number of publications may reflect potential barriers to scholarly performance in the Canadian system: access to research money, resident time constraints, limited physical space to support resident research activities and lack of research mentors.¹³ It has been recognized that barriers to completing scholarly projects in internal medicine residency include insufficient time, inadequate research skills and lack of a research curriculum.¹⁴

In our study, 98% of residents had access to funding for attendance at conferences and meetings to formally disseminate research findings. This does not address whether money is equally available to support new research initiatives. This latter concept could limit the research potential of residents at some centres, as research funding may be limited in today’s economy secondary to growing restraints on Canadian healthcare spending and the global recession. Furthermore, 81% of residents indicated their biostatistician was outside their program reflecting the small size of most individual research programs and probable increased cost for data analysis.

Despite encouragement from the RCPSC, Canadian residents lack dedicated time; only 38% of respondents had a dedicated research rotation and 43% had protected research time. These findings confirm a trend over the past decade away from dedicated research time during urology residency which could be detrimental to scholarly endeavours.¹³ Greater research time during residency has been shown to be associated with increased research productivity during and after residency.⁵ Months of protected research time have been correlated with volume of manuscripts submitted by urology resients.³ Number of months of dedicated research time was positively correlated with grants obtained and publications by Canadian orthopaedic residents.¹⁵ It is unknown if any Canadian institutions could regularly support >6 months of dedicated research time, but providing more research time to busy trainees would likely lead to increased research productivity.

Only 26% of the study participants indicated their program included a structured research curriculum. This, combined with lack of dedicated research time, identifies key factors that likely affect not only the quantity of scholarly activity of residents at such institutions, but also the acquisition of fundamental research principles, understanding of biostatistical methodology and learning of effective critical appraisal of research data. We feel that all urology residents benefit from research exposure regardless of individual future intentions of community or academic practice. Learning to critically appraise research literature is a key component of residency training and teaches the skills necessary to practice successful evidence-based medicine during his/her career.

Although 86% of our participants intended to pursue fellowship training, we found no correlation between number of research presentations or manuscript submissions during residency and pursuit of fellowship training. This disagrees with previous reports^3–5 and may be a reflection of our small sample size or a reflection of our sample population as previous studies examined chief residents, fellows and recent graduates. It has been reported that a 5- versus a 6-year residency program and manuscript publication during residency were each independent predictors of urology residents pursuing fellowship training and that residents who published a manuscript during residency were nearly sixfold more likely to pursue fellowship training.⁴ In our study, residents planning to pursue a fellowship had so far completed a mean of 1.25 ± 0.37 publications and 2.25 ± 0.54 presentations.

We were limited by the failure of 3 Canadian universities to share their research framework and scholarly activities of residents despite multiple attempts to contact relevant program directors from 2 institutions and lack of participation of all residents at another. We successfully sampled 77% (10/13) of urology training centres in Canada, but only 42 participants responded; this prevented us from performing multivariate analysis. There is the potential for response bias because residents who had successfully completed research projects may have been more inclined to respond to a survey regarding scholarly performance. The anonymous nature of our study prevents us from characterizing the responders and non-responders. Our survey was not formally validated before distribution. We did not specifically examine the journals in which manuscripts were accepted or published.

Conclusions

This is the first Canadian survey examining scholarly activity of all 5 years of urological residency. Level of post-graduate training year significantly affected quantitative scholarly activity during urology residency, but the numbers and types of presentations performed prior to residency, completion of an honours or graduate degree and plans to pursue fellowship training did not. There are potential opportunities to improve scholarly activity of urology residents by improving the availability of a dedicated research curriculum, research rotation, protected research time and biostatistical support.