Resident-run urology clinics: A tool for use in competency-based medical education for teaching and assessing transition-to-practice skills

Abstract

Introduction

In a competency-based approach to resident education, a component of training should focus on skills needed for the transition from residency to independent practice. The ability to run an outpatient clinic represents one such skill. Resident-run clinics (RRC) have been implemented in family medicine programs to allow residents to practice this skill and have enhanced learning while providing excellent patient satisfaction. To date, there has been little experience with RRCs in surgical residency programs. We describe a urology RRC and report assessments of both resident performance and patient satisfaction.

Methods

The RRC was run independently by a senior urology resident. All cases were reviewed with faculty at the end of the day and an evaluation form assessing resident performance was completed. Residents also completed a brief self-assessment. All patients completed an anonymous survey to assess aspects of patient satisfaction.

Results

Overall, resident performance was excellent, with changes to the management plan in 6% (3/47) of cases after faculty review. All clinics finished within 30 minutes of planned end time. Residents reported confidence in their ability to manage the clinic (8.25/10). Forty-three patient surveys were completed. On a five-point scale, patient ratings of wait time, clinic environment, and appointment duration were 3.91, 4.23, and 4.12, respectively. Patient ratings of resident skills (communication, sensitivity, treatment options, and answering questions) were 4.30, 4.35, 4.40, and 4.42, respectively. Overall, confidence in residents was 9.07/10 and 100% of patients would recommend the RRC.

Conclusions

Based on our ongoing experience, RRCs provide well-received, safe patient care and serve as a learning tool for residents as they prepare for independent practice. Given these results, residency programs could consider inclusion of a RRC as a component of the transition-to-practice training within a competency-based curriculum.

Introduction

Competency-based post-graduate medical education has emerged over the past two decades to rapidly become the standard of training worldwide^1–4 and it is expected to reshape medical education in the 21st century.⁵ The Royal College of Physicians and Surgeons of Canada (RCPSC) has recently launched a competency-based approach to residency education termed Competence by Design (CBD). Under the CBD framework, residents pass along a competence continuum consisting of four stages of training: transition to discipline, foundations of discipline, core of discipline, and transition to practice.⁶ The transition to practice stage occurs during the final few months of residency wherein the resident “demonstrates the readiness to enter autonomous practice.”⁶

One crucial aspect of the transition to practice stage is the ability to independently run an outpatient clinic. Resident physician-run outpatient clinics have been a valuable tool in primary care residency training programs across North America for decades.^7,8 These resident-run clinics (RRCs) give residents the opportunity to assess and treat patients in a mostly autonomous fashion, from generating diagnoses to developing management plans, without immediate review by a staff physician.

While RRCs have been implemented and studied for the past several years in primary care residency programs, they have only recently been reported in the literature among surgical residency training, most notably in plastic surgery^9–12 and general surgery.¹³ Generally, non-surgical clinics have been shown to enhance trainee learning while also providing excellent patient satisfaction.^14,15 However, to date, there has been very little documented in the literature about using RRCs in surgical residency training programs, and no reports specific to urology training. Such studies are essential in that they can demonstrate whether RRCs are effective ways of training and assessing surgical residents while ensuring patient safety.

We hypothesized that RRCs in urology could provide well-received patient care and function as a safe and effective method for resident transition to practice training. We describe the implementation of a urology RRC and report assessments of both resident performance and patient satisfaction.

Methods

This cross-sectional, questionnaire-based study was conducted at one Canadian academic hospital (The Ottawa Hospital - Civic Campus). The urology RRC was piloted starting November 2017, and all patients (outpatient referrals and emergency room referrals) referred to urology for nephrolithiasis and urinary retention were included. Referrals were triaged by attending staff and deemed appropriate for the resident clinic, at which point the patient was given an appointment within the resident clinic. Data capture for this study ran from the clinic’s initiation until April 2018.

This clinic was held on two or three half-days per month and was staffed by a single senior resident (post-graduate year 3–5), who was given a single clinic room for patient interviews and examinations. All cases seen during this clinic were reviewed with a staff surgeon at the end of the clinic and any changes to the resident’s treatment plan were noted. During the clinic, residents were instructed to attempt to make all clinical decisions, including ordering investigations and tentatively booking surgical procedures. The supervising staff urologist was always present in the hospital building and available if the resident in the RRC felt that he needed immediate assistance or advice on a case.

Patient satisfaction survey

Following their clinic visit, patients voluntarily completed a brief, anonymous survey to rate their satisfaction with the clinic experience, as well as confidence in the resident physician using Likert scales (Supplementary Fig. 1). The questions sought to evaluate resident performance based on CanMEDS roles.¹⁶ Patient demographic data captured included age, sex, and reason for referral to the clinic.

Resident evaluation

At the end of every clinic, residents were assessed by their attending physician to note any changes in diagnosis or management of patients. Residents then completed a brief self-assessment evaluating their confidence in being a medical expert and comfort in managing the clinic based on 10-point Likert scales (Supplementary Fig. 2).

This study was approved by The Ottawa Hospital Research Ethics Board.

Results

Patient satisfaction and confidence

The voluntary, anonymous patient satisfaction survey was completed by 43 of 47 patients who were assessed in the resident clinic. Patient demographic data was gathered via the voluntary survey and so was available only for those who completed the survey (n=43) (Table 1). Overall patient satisfaction with the RRC was high; 100% of patients said they would recommend the clinic to their family or friends. Mean (standard deviation [SD]) patient confidence in resident physicians was also high (9.07/10, SD 1.33).

Table 1.

Characteristics of patients seen in urology resident-run clinic

	n	%
Gender
Male	36	83.7
Female	7	16.3
Age
<60 years	19	44.2
≥60 years	24	55.8
Reason for referral
Renal colic	17	39.5
Urinary retention	13	30.2
Unspecified	13	30.2
Management
Diagnosis changed	1
Plan changed	2
No changes to resident diagnosis or plan	44

Patient satisfaction ratings of their clinic experience were distributed highly towards satisfaction (Fig. 1). Mean (SD) ratings of clinic attributes were as follows: wait time to obtain appointment: 3.79/5 (0.86); wait time at clinic: 3.91/5 (0.95); clinic environment: 4.23/5 (0.92); and sufficient appointment duration: 4.12/5 (0.93).

Fig. 1.

Histogram of patient satisfaction in clinic experience.

Patient confidence ratings of resident physician skills were similarly distributed highly towards confidence (Fig. 2). Mean (SD) ratings of resident skills were as follows: communication: 4.30/5 (0.74); respect for patients: 4.37/5 (0.72); sensitivity to patient needs: 4.35/5 (0.72); treatment options well-explained: 4.40/5 (0.82); and allowing and answering questions: 4.42/5 (0.76).

Fig. 2.

Histogram of patient confidence in resident physician.

Resident performance and patient safety

A total of 10 residents, three post-graduate year (PGY) 5, three PGY 4, and four PGY 3, participated in the study. Following case review by staff, residents’ diagnosis or management plan was changed for three of the 47 patients (6%) (Table 1). All three changes were felt to be minor; two changes involved optimization of medication and one involved interpretation of imaging.

All clinics finished within 30 minutes of planned end time, with an average clinic duration of 180 minutes. The supervising staff urologist was never called by the resident during the clinic for advice/intervention.

Residents reported confidence (mean, SD) in their ability to function as a medical expert (3.75/5, 0.96) and manage the clinic as efficiently as possible (3.75/5, 0.96). The majority (60%) of resident participants found this a useful tool during their transition to practice; some residents felt that the clinic was not fully representative of the scope of a practicing urologist’s clinic due to the limited case types (nephrolithiasis and urinary retention only).

Discussion

Competency-based medical education (CBME) is rapidly becoming a standard component of residency training worldwide. ^1–4 In Canada, as of July 2018, all urology residency training programs have implemented CBME by adopting the RCPSC CBD framework. Within the CBD competence continuum, transition to practice represents the final stage of residency training before certification.⁶ A large part of urological practice occurs in the outpatient clinic setting and, therefore, the ability to run an outpatient clinic efficiently and safely is a key competency required of every practicing urologist. Resident-run clinics allow trainees to learn and practice the necessary skills to independently run an outpatient clinic while still having some oversight to ensure patient safety. Reports of RRCs come almost exclusively from the primary care setting^7,8,15 and there is limited data available concerning educational and patient safety outcomes for RRCs in surgical specialties.^9–13 Notably, there have been none reported in urology.

In our study, we demonstrate that patient satisfaction and confidence are maintained at a high level in RRCs. Following case review by staff, changes to resident diagnosis or management plans were only occasionally made and were minor in nature. This is not just an indicator that patient safety is maintained, but that residents of the PGY 4–5 level (the majority of the participants in this study) are “ready” to have the type of independence provided in the RRC. Finally, the majority of resident physicians themselves found these RRCs to be a valuable component of their transition to practice training. Given the positive results of this study, a logical next step is to expand the breadth of patient presentations to include more diverse urological presentations.

Our findings are important since they provide evidence that RRCs are viable and feasible as a component of residency training. Hospital administrators and medico-legal advisors may require this type of evidence before approving the initiation of RRCs in the future. The high level of resident satisfaction we demonstrated indicates that trainees themselves are likely to “buy in” to participating in RRCs as a component of their training.

A limitation to our study includes a possible selection bias inherent to the patient surveys being voluntary, so respondents may have been those who had strong feelings about their clinic experience, either positive or negative. Notably, there have been no strong negative responses so far and there were no outliers significantly skewing our results. The relatively small sample size is another limitation of this study. However, among survey sampling studies, while 30 respondents is the mathematical minimum required, we have obtained a sample closer to the 50 respondents needed for a patient satisfaction industry standard, where the margin of error is only 14%.¹⁷ Moreover, the trends in patient satisfaction were quite evident, given the overwhelmingly positive response to this clinic. Although it could be seen as a limitation that the scope of patients within this study were limited to nephrolithiasis and retention, this was a pilot study attempting to establish safety and patient satisfaction and so the authors did not feel that more complex patients were appropriate. Nevertheless, we believe the ability to see patients over several visits provides residents with experience in continuity of care not otherwise gained in day-to-day residency training. Finally, other than assessing resident satisfaction and opinion regarding the educational benefits of participating in the RRC, we did not have any objective measures of learning or improvement over time. Future studies could objectively assess factors such as efficiency (time spent per patient) or diagnostic accuracy over time.

Conclusions

Based on this study, RRCs provide safe, well-received patient care and serve as a well-received learning tool for residents as they prepare for independent practice. Given these results, urology residency programs should consider formal inclusion of RRCs as a component of transition to practice training within a competency-based curriculum.

Supplementary Materials

Supplementary Fig. 1.

Patient satisfactory survey.

Supplementary Fig. 2.

Resident self-assessment survey.