Study habits of Canadian urology residents: Implications for development of a competence by design curriculum

Abstract

Introduction

The purpose of this study was to evaluate the study habits of Canadian urology residents throughout their residency training.

Methods

A survey was administered to all final-year Canadian urology residents over a two-year period. Sixty-seven respondents answered a survey consisting of 54 questions scored on a 10-point Likert score. The survey addressed study habits throughout training, motivations for studying, and preferred resources used.

Results

Dedication to studying was directly correlated with proximity to the Royal College of Physicians and Surgeons of Canada (RCPSC) exam. Ninety-six percent of residents reported studying over 10 hours per week during their chief year compared to 0% during their junior year. As residents progressed in their training, preparation for the Royal College exam became the greatest motivator for studying. There was considerable variability in study methods and study resources used throughout training. In their chief year, residents found such resources as the textbook Campbell-Walsh, AUA updates, CUA and AUA guidelines, and the study notes of former trainees to be valuable for their preparation. Teaching rounds, journal clubs, and reading current urological literature were found to be les helpful. Forty-six percent of all residents surveyed indicated that they would prefer writing their RCPSC exam one year earlier than the current timing.

Conclusions

This study provides insight into study habits of Canadian urology residents. This data may be helpful in shaping the future of urology training programs and examinations within Canada and elsewhere.

Introduction

Urology residency in Canada is currently driven by a time-based model, consisting of five years of structured clinical training culminating in a summative licensing exam in the final year. In response to growing societal demands for greater accountability and that residents are adequately prepared for increasingly complex medical practice,¹ competency-based medical education (CBME) is receiving more attention worldwide.^2–4 In 2015, the Royal College of Physicians and Surgeons of Canada (RCPSC) launched Competence By Design (CBD), a multiyear initiative to implement a CBME approach to residency training across Canada.⁵ Departments of urology in Canadian universities are scheduled to implement a CBD curriculum for all residents entering into the July 2017 cohort and beyond.

The CBD framework has been conceptualized to move away from the practice of credentialing physicians solely on the basis of time spent training, and towards a model where learners will demonstrate achievement of defined milestones of competence at their own pace.⁶ Each of the seven roles first described in the 1996 Canadian Medical Education Directions for Specialists (CanMEDS) Physician Competency Framework — medical expert, communicator, collaborator, leader, health advocate, scholar, and professional — will be expressed as a series of enabling competencies. Attainment of specific criteria that describe the knowledge, skills, and attitudes residents are expected to achieve will serve as the focus of learning experiences. Well-defined milestones for progression at each stage will provide learners and educators with a transparent understanding of the requirements to maintain and improve their performance for advancement.

As scholars, in addition to the rigorous demands of clinical care, residents are expected to master a large body of factual knowledge during their residency.¹ As we shift from a time- to a competency-based curriculum, it is more imperative than ever for residents to develop efficient methods of studying and learning, and for programs to ensure training is more focused, effective, efficient, and cost-effective. Furthermore, in the most recent edition of the CanMEDS framework released in 2015,⁵ there is increased emphasis on the lifelong learner component, recognizing the need to have a planned approach for active and continuous learning as part of a sustained commitment to excellence in practice.

A small number of studies involving residents in other surgical specialties have correlated resident study habits with performance on in-training examinations. In particular, structured reading habits^7,8 and the use of specific study materials^7,9 were found to be positive predictors of successful performance. Studies have established that resident performance on these in-training examinations goes on to correlate with subsequent performance on licensing examinations.^10,11

To our knowledge, no data currently exists on how Canadian urology residents study and prepare for their RCPSC examination in urology. With the upcoming implementation of the CBD framework in residency training, an increased interest in urology education and curriculum development is reflected in many current articles addressing these topics.^12,13 We set out to explore the study habits of Canadian urology residents during their training.

Methods

This prospective study consists of an anonymous, cross-sectional, self-reported survey that was administered to all final-year Canadian urology residents (n=67) at a review course for two consecutive graduating years (2013 and 2014). Participation was voluntary and confidential; no identifying information was recorded in the survey results. Ethics approval was attained from the Queen’s University institutional research ethics board and explanations for the objectives of the study and assurance of confidentiality were distributed to all respondents.

The survey consisted of 54 closed-ended questions on a 10-point Likert scale regarding study habits. Residents were asked to rate the following: volume of studying, sources of motivation, resources used, studying methods, and overall satisfaction with training. Questions addressed three distinctive levels of training: junior resident (PGY1, PGY2), senior resident (PGY3, PGY4), and chief resident (PGY5). Questionnaire development resulted from previous experience with survey construction for residents. Residents and educators involved in the department were asked to assess and modify the survey for clarity.

Basic statistical analysis was completed on quantitative data. For ease of reporting, results presented are those determined to reflect high importance with Likert scores of 8, 9, or 10, grouped by training year (junior, senior, or chief resident).

Results

A total of 67 final-year residents (100%) across Canada responded to the survey. Comparing the two annual cohorts of residents, there was very little difference in responses.

Respondents indicated that the volume and structure of their studying increased dramatically throughout residency. Over 60% of respondents denied dedicated study time in junior years, whereas 96% of respondents reported greater than 10 hours per week in their final year, with 73% spending greater than 25 hours per week studying (Fig. 1). Motivation for studying shifted with seniority, from preparation for clinical duties in junior years to RCPSC exam preparation as seniority increased (Fig. 2). In the final year of training, 99% of respondents indicated that exam preparation was the biggest motivator.

Fig. 1.

Weekly studying time. Bar graph shows the percentage of respondents at each stage of training (junior, senior, chief) reporting number of hours (range <2 to >40) spent studying per week; 96% of respondents reported greater than 10 hours per week in their chief year, with 73% spending greater than 25 hours per week studying.

Fig. 2.

Highly motivating factors for studying. Bar graph shows percentage of respondents at each stage of training (junior, senior, chief) reporting factors for studying as highly motivating (Likert score ≥8). In junior years, preparation for rounds, OR, and clinic were the primary motivating factors. In the final year of training, 99% of respondents indicated that exam preparation was the biggest motivating factor. AUA: American Urological Association; OR: operating room.

A variety of resources were used during the chief year, including: the Campbell-Walsh Urology textbook, the American Urological Association (AUA) and Canadian Urological Association (CUA) guidelines, the AUA updates, review notes from former trainees, and review courses. Less useful for chief residents were didactic lectures, journal clubs, and current urological literature (Fig. 3). Among junior residents, there is considerable variability in resources used with no one dominant source. In terms of study methods, there is a trend for the increasing popularity of independent and group studying, as well as review courses as residents progress in seniority. Conversely, didactic and interactive teaching maintain the same level of popularity throughout training (Fig. 4).

Fig. 3.

Highly useful study resources. Bar graph shows percentage of respondents at each stage of training (junior, senior, chief) reporting study resource as highly useful (Likert score ≥8). Campbell-Walsh Urology and AUA/CUA guidelines were reported as highly useful by greater than 80% of chief residents. Core teaching and lectures were rated as less useful than other methods. AUA: American Urological Association; CUA: Canadian Urological Association.

Fig. 4.

Highly preferred study methods. Bar graph shows percentage of respondents at each stage of training (junior, senior, chief) reporting study method (independent study, group study, interactive teaching, didactic teaching, and review courses) as highly preferred (Likert score ≥8). There is considerable variability in the study methods used. Independent study was most consistently ranked as highly important across all levels of training.

Interestingly, 46% of residents indicated that they would favour writing their exam a year earlier than the current timing at the end of the final year of training.

Discussion

This study is the first to explore the study patterns of Canadian urology residents. CBD represents the application of research in education to optimize human performance in healthcare. Encouragingly, the results of this study support many of the proposed changes in the upcoming CBD model, and can be further used to guide the design and stepwise implementation of new learning experiences, teaching methods, and assessment systems under the new model.

Overall, this study demonstrates the important role that the RCPSC certifying examination has in promoting studying, with the average volume of studying increasing 10-fold from junior to chief residency, although there is little consensus on preferred study methods. These results suggest that in addition to structured preparatory examinations in the final year,¹⁴ consideration may be given to instituting examinations throughout training to spur motivation for studying during the junior years of training. Importantly, well-established medical education literature on test-enhanced learning supports giving tests often and spaced out in time to promote better long-term retention of knowledge.¹⁵ Feedback is also crucial to learning from tests, and a system of ongoing evaluation would facilitate opportunities for faculty to provide tailored recommendations, as well as remedial education and counselling for residents who show signs of academic weakness. This more rigorous and detailed tracking of residents’ progress is in line with the learner-centred CBD framework, a key strength of CBME.

This data also calls attention to the need to increase efforts to ensure appropriate and useful didactic sessions in training programs. While didactic lectures are a mainstay of medical education, it has been observed that medical residents fail to show learning benefit from lecture series in terms of effect on standardized examination scores⁹ or long-term practice behaviours.^16–18 Our results reflect that junior residents find these sessions to be valuable. As they progress into training, residents tend to favour independent and group studying. It appears that during their chief year, when most studying is focused on preparation for the certification exam, residents favour focused sessions, where topics perceived to fill specific knowledge gaps are covered. Teaching sessions are fixed in time and space and often cover a topic only once per year in typical didactic curricula. Attendance can be problematic, with residents being post-call, away on off-site rotations, or being paged out for patient care. It is, therefore, not surprising that these sessions become less relevant for chief residents, who are mostly concentrating on passing their exams. With the introduction of the CBD framework, it is more important than ever to ensure the allocation of limited time-maximized learning.

Perhaps the most interesting finding was that nearly half of all residents indicated that they would rather write their final exam at the end of their fourth year of training, a year earlier than the current system. Whether this reflects a proximity bias in that residents who are three months away from their Royal College exam are understandably apprehensive and would rather that this burden has been done away with already is difficult to ascertain. Nonetheless, in the new CBD framework, the phases of training have been redefined to reflect the stages of progress throughout a physician’s career. These phases include transition to discipline, foundations of discipline, core of discipline, transition to practice, and ultimately, to continuing professional development and transition out of professional practice.⁵ The findings of this study may lend some credence to moving the examination to the end of the core of discipline stage, allowing residents to focus on navigating the challenges of transitioning from senior resident to independent surgeon in their final phase of residency. However, given the critical role that a certification exam seems to play in motivating studying, further work may be necessary to elucidate motivators to properly transition to practice, and ultimately, staying up-to-date through continuing professional development during practice.

It is important to note that this study examines the study habits of urology residents. While fund of knowledge is a component of competence, other areas, such as acquisition of surgical skills (through simulation or otherwise), and other non-medical expert roles of CanMEDS were not assessed here.

Our study does have some weaknesses. The observations are limited by the nature of a self-reported survey, and there is a lack of previously validated surveys. In addition, there are a relatively small number of respondents. Another limitation of the study was that the survey only captures a snapshot of self-reported attitudes and experience close to the RCPSC certification exams, and relies on the residents’ accurate recollection of study habits in earlier years of training.

Despite the limitations pertaining to the present study, the response rate of the residents and the similarity of results between the two cohorts could attest to the veracity of the findings. Further studies with larger numbers of residents may be useful to better assess the variable experience and attitudes towards study habits during urology residency. In addition, studies correlating study habits with final exam results may be useful to identify preparation strategies associated with higher performance. However, correlation of test performance on future performance as an attending surgeon remains unclear.¹⁹

Conclusion

This study provides initial insight into the study habits of Canadian urology residents. With the upcoming RCPSC’s initiative to implement a CBME model to all residency training programs across Canada, this information can help in the development of effective, evidence-based curricula to maximize resident success.