Core competencies in surgery: evaluating the goals of urology residency training in Canada

Abstract

Background

In Canada and the United States, the relevance and utility of training objectives as perceived by practising surgeons is rarely examined. We sought to determine if urology residency training objectives reflect the broad realities of urologic practice.

Methods

A survey, based on the Royal College of Physicians and Surgeons of Canada training objectives for urology, was designed and validated. All 418 full-time practising members of the Canadian Urological Association were surveyed.

Results

The overall response rate was 63%. Many specialized clinical areas of urology that receive little emphasis in the training objectives were rated as useful by the majority: laparoscopic surgery (92%), percutaneous renal access (86%), transrectal ultrasonography (84%), pediatric urology (81%), extracorporeal shock-wave lithotripsy (70%), urethral reconstruction (66%) and adrenal surgery (62%). Microsurgery and transplantation were perceived as less useful (54% and 22% respectively). Virtually all nonsurgical training objectives were regarded as useful components of training; however, in the opinion of the majority of respondents residency did not prepare them for many of these: the challenges of office and hospital administration (91% and 89% not prepared [NP]), building a referral base (67% NP), time management (60% NP) and providing care under financial constraints (60% NP).

Conclusion

The study results support the current training objectives and indicate areas requiring increased emphasis.

Surgery is a dynamic, constantly changing field.¹ Newer approaches to specific problems often require advanced training and expensive technology. In conjunction with these advances in surgery significant changes have occurred in the delivery of health care in Canada and the United States: an increased emphasis on outpatient surgery, office procedures and nonoperative management of various conditions, concomitant with diminishing access to hospital resources.² Surgeons are becoming increasingly involved in administrative and managerial roles, often as part of a change-management team. Residency training should adequately prepare trainees for both surgical and nonsurgical challenges of practice. The latter are associated with communication and administrative skills, and the frustrations of small-business management.

Recognizing the changing landscape of medical care, the Royal College of Physicians and Surgeons of Canada (RCPSC) set out in 1993 to revise the training objectives for specialist education. This project was initiated to ensure that skills acquired in residency responded to societal needs.² CanMEDS 2000 outlines a competency framework that is a guide to training in all specialty areas, not just urology. The 7 specific competencies emphasized were chosen to prepare clinicians for the many challenges they face as health-care providers. They include the following: medical expert/clinical decision-maker, communicator, collaborator, manager, health advocate, scholar, and professional. In developing this framework, the RCPSC collaborated with educational experts to devise objectives, as well as learning and evaluation methods, to implement into residency training.

In the US also, the evaluation of residency training has undergone major change with the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project. This represents a shift toward the evaluation of resident performance in a set of core clinical competencies that closely resemble those outlined by the RCPSC^3,4 (Fig. 1). There has been considerable debate about what strategies should be employed to evaluate resident performance in these core competencies. Furthermore, their validity and ability to improve the quality of resident training has been questioned.⁴

FIG. 1. Core competencies as outlined by the Royal College of Physicians and Surgeons of Canada (RCPSC) and the Accreditation Council for Graduate Medical Education (ACGME) in the United States.

Given that training residents and evaluating their performance in these core competencies is resource intense, our goal was to evaluate the perceived utility of these competencies among practising urologists. We not only sought to examine training in specific specialized surgical areas of urology but also in the nonclinical areas outlined under the CanMEDS competencies.

We examined surgical skills that were listed as either “B” or “C” under the medical expert competency in the RCPSC training objectives for urology residency.⁵ The “B-list” skills include those that the resident should know how to do but may not have actually done during residency training. The “C-list” skills are those for which the resident should be able to describe the principles of the procedure. We chose these specific competencies because they represent novel or specialized skills in urology, which may not be adequately emphasized during training. They are distinct from the “A-list” skills that the graduating resident is expected to perform competently and independently.⁶

Regarding the nonsurgical CanMEDS competencies, our hypothesis was that emphasis on these communicator, collaborator, manager, health advocate, scholar and professional competencies may be very important but may not be adequately taught in residency.

Methods

Survey instrument

The 2-page, semistructured questionnaire contained 4 sections. A modified 6-point Likert scale was weighted from “not useful” to “highly useful” to gather quantitative data on the relevance of surgical and nonsurgical aspects of practice as well as specific challenges facing practitioners (see Appendix 1).

In part I we questioned respondents about the location of residency training. In part II we asked them to rate the components of residency training based on their relevance to clinical urologic practice, intentionally excluding surgical components from the “A list” of RCPSC objectives in our survey (e.g., transurethral resection, cystoscopy).

In part III we asked respondents to rate the nonsurgical components of residency training based on their relevance to clinical urologic practice.

In part IV we asked respondents: “How well do you feel residency prepared you for the following challenges of practice?” by employing the same 6-point Likert scale weighted from “not prepared” to “highly prepared.” In the interpretation of the results the terms “relevance” and “utility” are used interchangeably due to the structure of the survey.

Two additional questions asked respondents about their “biggest challenge in practice today” and how they felt residency training could be improved.

The survey was screened for face validity on a group of 14 urology residents. The contents of the survey were scrutinized for clarity, ambiguity of terminology and ease of administration. A reliability analysis was conducted in which 11 urologists completed the survey at 2 separate times. Averaging across the survey questions and the 11 respondents, 81% of the re-test responses were within 1 category of the original test response. The Spearman rank correlation was calculated for each test and re-test pair, revealing statistically significant positive correlations for all 11 respondents, based on a 0.05 significance level. Krippendorff's r was calculated for each test and re-test pair, revealing agreement over and above chance for all respondents.

Conducting the survey

The survey was mailed to all 418 Canadian Urological Association (CUA) members practising full-time urology in Canada in 2003. A cover letter explained the goals of the survey. Each survey was marked with a unique number to identify the respondent and allow subgroup analysis. French and English surveys were sent to members practising in the provinces of Quebec and New Brunswick. After 6 weeks, a second copy of the survey was mailed to nonrespondents to ensure an adequate sample.⁷ All surveys included in the final analysis were collected within 12 weeks of initial mailing.

Data analysis

Quantitative Likert data was organized into the following subgroups based on the setting of practice: urban (≥ 100 000 population), rural (< 100 000 population) and academic (full-time practice in a university setting with a postgraduate urology residency training program). Responses were also analyzed on the basis of province of practice.

Statistical analysis

The Likert scale was interpreted as a continuous scale from 1 (“not useful”) to 6 (“highly useful”). A “useful” rating in the results section corresponds to a rating of 4, 5 or 6 (adjacent to the weighted end “useful”) whereas a “not useful” rating corresponds to a rating of 1, 2 or 3 (adjacent to the weighted end “not useful”). The standard errors (SEs) of the proportions were calculated to account for sampling error of the population using the finite population correction (fpc) to account for a population (n = 418) (fpc = square root [1–0.63 = 0.37] = 0.608). The SE for each proportion was calculated as follows: SE = 0.608 × [square root of {p × (1 – p)/n}]. In this formula “p” represents the tabulated proportion from the data and “n” the total number of responses (n = 264). Statistical significance was set at a p value of 0.05 and 95% confidence intervals (95% CIs) were calculated by multiplying the SEs by the z score (= 1.96).

Results

The overall response rate was 63%. There was no statistically significant difference in response rate according to setting or province of practice (Table 1).

Table 1

Part I. Location of residenct training

The majority (91%) of respondents stated that residents should spend at least 3 months outside academic training centres. Most (68%) agreed that this component of training was best experienced in the fourth year of a 5-year training program.

Part II. Surgical components of training

The perceived utility of the surgical components of training are presented in Table 2. Laparoscopic surgery and pediatric urology were rated as “useful” by the majority of all respondents, as were transrectal ultrasonography, percutaneous renal access surgery and extracorporeal shock-wave lithotripsy, and urethral reconstruction. Adrenal surgery and microsurgery were rated as “useful” by approximately 60% of the respondents. The majority of respondents (78%) rated renal transplantation as “not useful.”

Table 2

With the exception of microsurgery, there were no differences of opinion between the different subgroups of urologists (rural, urban and academic). Academic urologists did not rate microsurgery as useful whereas urban and rural urologists did.

Part III. Nonsurgical components of training

All nonsurgical components were rated as “useful” (Table 3). Components of training that were assessed included: office management, administrative responsibilities, time management, critical appraisal of the literature, ethical decision-making, communication skills with colleagues and patients and accessing medical information with computer technology. There was no statistical difference between the proportions of respondents rating hospital administration as “useful” and those rating it as “not useful.” There were no differences between subgroups of urologists.

Table 3

Part IV. Preparation for the challenges of practice

The majority of respondents felt that residency adequately prepared them for the following facets of practice: appraisal of the scientific literature, ethical decision-making, communication with colleagues and patients, and accepting ultimate patient-care responsibility (Table 4). A majority felt unprepared for the challenges of time management, office management, hospital administration, building a referral base, building confidence in the medical community and providing care in a financially constrained system. Analysis of the responses based on the setting of practice revealed no significant subgroup differences.

Table 4

Open-ended questions

Forty-four percent of comments included mention of time management (time for administration, reading, running an office, leading a balanced life). As well, 30% of comments related to the challenge of providing care to meet unrealistic patient demands with limited access to resources such as operating-room time, diagnostic radiology and hospital beds. Many remarked on the challenges of dealing with politics and administration, bureaucratic interference in the delivery of care, loss of control, loss of respect and a general disregard for doctors' time.

Thirty percent of respondents suggested that increased exposure to community urology and more elective time during residency would allow acquisition of skills in areas of special interest as well as assist in finding future employment. Twenty-two percent suggested formal training in time management and administration. A number of comments were made about increasing the emphasis on the common problems of operative and office urology along with de-emphasizing high-intensity surgery.

Discussion

Surgical practice has been significantly affected by innovation in medical and surgical therapies at the same time that the delivery of care has been altered by changes in resource allocation, patient demographics and expectations. The objectives of residency training should remain congruent with these changes. The goal of our project was to examine the perceived utility of specific training objectives in urology as they pertain to these realities.

The overall response rate of 63% is excellent for such a survey and is higher than that of other similar surveys.^6,8,9 We speculate that this likely relates to a sense of interest and concern about resident education among the population of practising urologists that has allowed us to make statistically valid intergroup comparisons with small margins of error and to generalize the results to our entire population.

The 2 surgical components that were rated “useful” by the highest proportion of respondents were laparoscopy and pediatric urology. Laparoscopy is an expanding field in urology. It is being driven by many factors including patient preference, shorter hospitalization, more rapid postoperative recovery and decreased morbidity.^10–12 Our results suggest that this is an important component of training that should receive increasing emphasis.

The high perceived utility of pediatric urology appears to be consistent with the reality of urologic practice, at least in the US. In a 1999 American Urological Association Gallup Survey, 83% of respondents treated pediatric patients, including 92% who performed simple urologic procedures rather than referring the child to a pediatric urologist.¹³

We also noted that adrenal surgery, transrectal ultrasonography and percutaneous renal access were surgical components valued by the majority of respondents. In academic centres, adrenal surgery is often performed laparoscopically by general surgeons who have acquired the necessary skills and are receiving the majority of referrals. Therefore exposure of urology residents to this facet of surgery is limited during training. Referral lines in community practice may differ, however. Similarly, TRUS and percutaneous renal access are specialized techniques that are performed exclusively by interventional radiologists in many academic centres. It seems that for the majority of respondents interventional radiologists may not be available, and urologists are required to assume responsibility for these procedures. Given the high perceived value of these skills we conclude that resident training in these areas may require increased emphasis.

The majority of Canadian urologists rated renal transplantation as “not useful” for surgical practice. This finding may relate to the fact that renal transplantation is performed only in academic centres. However, exposure to renal transplantation is associated with the acquisition of vascular surgical skills, exposure to the pelvic ureter and ureteral reimplantation surgery, techniques applicable to many other areas of urology.

All respondents rated virtually all components of nonsurgical training as useful and relevant. Hospital administration was the only nonsurgical component for which there was no statistical difference between respondents who rated this as “useful” versus those who rated it as “not useful.” In the challenges section (part IV), however, the majority of respondents identified this as a component of practice for which they were not prepared. Such a finding would seem paradoxical in that the respondents rated administration as a significant challenge in practice at the same time that this facet of practice is not clearly rated as a relevant component of training. This observation may relate to the fact that urologists do not see their role as administrative, but the reality of practice necessitates that they deal with administration. Also, one could postulate that administration training may be important for future practice, but respondents felt that it was not a useful component of training to become a clinical urologist. Further investigation would help clarify how this component could be best implemented into residency. An alarming finding was that close to one-third of respondents did not feel that their formal training prepared them for ethical decision-making.

In comparing the utility of nonsurgical components of practice with the challenges section of the survey, we noted that time management and office management were rated as “useful” and were also rated as components for which urologists were not prepared for upon completion of training. As well, in the open-ended portion of the survey many respondents noted that time and office management were their biggest challenges. Currently, there is very little instruction in Canadian residency training on time management and the business aspects of running an office. Similarly, in the US, a recent survey of American neurosurgical residents and program directors revealed that virtually all respondents felt unprepared for the business aspects of medicine. A significant proportion (58%) of program directors indicated that residency did not deal effectively with this facet of clinical practice.¹⁴ The experience of some postgraduate program directors in surgery is that residents are more interested in spending time to learn the technical aspects of surgery than in learning the business or administrative skills needed to practise. Motivating residents to become interested in this facet of medicine during their residency is a challenge. We suggest that mentorship programs in which residents are exposed to the day-to-day work of urologic practice may help motivate learning in these areas by demonstrating that surgical skills make up 1 component of a host of skills required to practise urology.

At the undergraduate level, there clearly is an increased recognition of the need for formal training in the managerial sciences as they apply to medicine. Over the last decade, there has been an explosion in the number of universities offering a combined MD–MBA degree. Currently, 31% of North American medical schools offer this option.¹⁵ However, it will likely be a decade or more before this education translates into practising clinicians with strong business and administrative skills. Strategies to address this deficiency in today's residency education are required now.

Summary and conclusions

We have identified a number of surgical and nonsurgical areas that Canadian urologists perceive as having high utility in all settings of practice. Some surgical areas of urology that are underemphasized in our training objectives are perceived to be of high importance. There are many nonsurgical components of practice that are emphasized in our training objectives but for which many Canadian urologists feel inadequately prepared by their residency training. These findings highlight the perceived importance of the RCPSC and ACGME core competencies for training and emphasize the need for resident education not only in specialized clinical areas but also in the managerial aspects of clinical practice.

Appendix 1.

Presented at the 59th annual meeting of the Canadian Urological Association, Whistler, BC, June 28, 2004.

Competing interests: None declared.

Correspondence to: Dr. Andrew E. MacNeily, Division of Pediatric Urology, K0-134, BC Children's Hospital, 4480 Oak St., Vancouver BC V6H 3V4; fax 604 875-2721; amacneily@cw.bc.ca