A needs assessment study of undergraduate surgical education

Abstract

Background

There is compelling evidence to suggest that undergraduate surgical education may fail to provide appropriate instruction in basic surgical principles and skills.

Methods

We completed a descriptive, cross-sectional survey of stakeholder groups (surgeon educators and recent medical school graduates) to assess the perceived relevance and learning for surgical principles, surgical skills, teaching environments and teaching interventions.

Results

Graduates returned 123 surveys, and surgeons returned 55 surveys (response rates: graduates 46%, surgeons 45%). Both graduates and surgeons considered 8 of 10 surgical principles highly relevant to current medical practice. Despite this, the surgical clerkship seemed to enable proficiency in far fewer principles (graduates: 3, surgeons: 5). Graduates believed that each of the 15 basic surgical skills is relevant to current medical practice, whereas surgeons indicated that more invasive skills (i.e., central venous lines, thoracentesis) are much less relevant. Graduates and surgeons indicated that medical students will achieve proficiency in only 3 basic skills areas as a result of the surgical clerkship. Graduates and surgeons considered each surgical specialty relevant and effective in undergraduate surgical education. According to graduates and surgeons, the most effective teaching environments are outpatient settings (emergency department, outpatient clinics). Graduates and surgeons ranked resident teaching as the most effective teaching intervention, and traditional interventions (grand rounds, formal rounds) and electronic resources (computer-assisted learning, web-based learning) were ranked the least effective.

Conclusions

In this study, we assessed the learning needs of contemporary medical students in surgery. The results suggest that respondent graduate students and surgeons believe that the level of proficiency achieved in surgical principles and basic skills through undergraduate surgical educations is much less than anticipated. Outpatient settings and resident teaching are believed to provide the most effective teaching for medical students. Information from this study has important implications for Canadian undergraduate surgery programs and curricula.

There is compelling evidence to suggest that undergraduate surgical education may fail to provide appropriate instruction in basic surgical principles and skills. In 1988, Reznick and colleagues¹ identified important deficiencies in the surgical education of family physicians and proposed broad recommendations for curricular change. More recently, Spratt and others² found that some basic surgical problems and skills require greater emphasis during the surgery clerkship. These concerns may be compounded by a limited exposure to key surgical specialty problems and skill sets within undergraduate surgical curricula.^3–5

DaRosa and colleagues⁶ emphasized the need for program evaluation in undergraduate surgery to assess the needs of learners and the impact of contemporary undergraduate surgical curricula. We feel these data should encourage program directors in undergraduate surgical education to assess whether the needs of contemporary medical students are being met within their own curricula. Thus, we evaluated a Canadian undergraduate surgery clerkship curriculum through a needs assessment.

Methods

We completed a descriptive, cross-sectional survey of stakeholder groups (surgeon educators at McMaster University and recent graduates from McMaster University medical school, not limited by postgraduate program). Recent medical school graduates were surveyed with the assumption that they will recall their undergraduate surgery education with acceptable clarity and will have developed appropriate insight into the basic surgical knowledge and skills necessary for general medical practice. In this study, graduates' responses were considered the perceived learning needs for undergraduate surgery, and surgeons' responses were considered a surrogate for the true learning needs. We assessed 4 broad educational areas in the survey: knowledge of basic surgical principles, basic surgical skills, teaching environments and teaching interventions.

The section on basic surgical principles (i.e., wound healing, fluids and electrolytes) was derived from several sources, including the guidelines produced by the Committee for Undergraduate Surgical Education in Canada (CUSEC), the Association for Surgical Education (ASE), a literature review and textbooks. We used expert opinion to refine and summarize the list of topics chosen.^7,8 In a similar manner, we developed an appropriate list of key surgical skills.

For each of the specific educational topics and skills detailed above, the respondents were asked to score the quality of education during their clerkship and the significance or relevance of each topic for general medical practice. Topics were scored with a 5-point Likert-type rating system.⁹ Prior to the study, select-ed surgeon-educators reviewed the questionnaire to establish content validity and assure clarity. The questionnaires were mailed to the stakeholders for completion (3 mailings at 3-week intervals).

The data were entered into a Microsoft Excel^© spreadsheet and analyzed by ranking each topic ac-cording to its mean score. Confidence intervals were calculated with the Excel data analysis package.

Results

The questionnaires were mailed to 284 McMaster graduates and 118 surgeons working within 4 teaching hospitals in Hamilton. Graduates returned 123 surveys (3 discounted and 16 returned with incorrect addresses). Surgeons returned 55 surveys (2 discounted). The overall response rate for the graduates was 46% and for the surgeons was 45%.

The graduates and the surgeons both considered 8 of 10 surgical principles to be highly relevant in their current medical practice (Fig. 1). Despite this, graduates believed they achieved proficiency in only 4 of these principles (Fig. 1a), whereas surgeons thought most graduates were proficient in 6 principles (Fig. 1b).

FIG. 1. Perceived learning and relevance of surgical principles according to graduates (n = 120) (A) and surgeon educators (n = 47) (B). Likert scale anchors for learning: 1: learned poorly, 3: became proficient, 5: excelled. Likert scale anchors for relevance: 1: irrelevant, 3: relevant, 5: extremely relevant.

Graduates thought that each of the 15 surgical skills on the survey were relevant in their current medical practice (Fig. 2a). However, they indicated that they developed proficiency in only 3 skills. Moreover, graduates think they might not have learned many “basic” surgical skills adequately during their undergraduate training (wound care, wound débridement and casting). Surgeon educators identified more “advanced” skills as less relevant to undergraduates (fine needle aspiration, thoracentesis, paracentesis and chest tube placement) (Fig. 2b). Surgeons also indicated that medical students acquire proficiency in only 3 of the surgical skills listed.

FIG. 2. Perceived learning and relevance of basic surgical skills according to graduates (n = 120) (A) and surgeon educators (n = 36) (B). Likert scale anchors for learning: 1: learned poorly, 3: became proficient, 5: excelled. Likert scale anchors for relevance: 1: irrelevant, 3: relevant, 5: extremely relevant. ABG = arterial blood gas; CVL = central venous line; I/D = incision and drainage; IV = intravenous; NG = nasogastric.

Graduates and surgeons considered all surgical specialties relevant in undergraduate surgery education (Fig. 3). General surgery, orthopedics, plastics, urology and pediatric surgery are ranked as the most relevant specialties. Both the graduate and the surgeon respondents identified a disparity between the perceived relevance and the perceived impact on learning for general surgery and orthopedics.

FIG.3. Educational impact and relevance of surgical specialties according to graduates (n = 120) (A) and surgeon educators (n = 30) (B). Likert scale anchors for impact: 1: does not help, 3: usually helps, 5: extremely effective. Likert scale anchors for relevance: 1: never, 3: occasionally, 5: always. ENT = ear, nose, throat; Ortho = orthopedics; Neuro = neurosurgery; Peds = pediatrics.

Although graduate and surgeon respondents considered each teaching environment highly relevant to medical student education, the emergency department was ranked the highest (Fig. 4). The environments considered most effective for teaching were outpatient settings, including the emergency department and outpatient clinics.

FIG. 4. Effectiveness and relevance of teaching environments according to graduates (n = 120) (A) and surgeon educators (n = 44) (B). Likert scale anchors for effectiveness: 1: did not learn, 3: usually effective, 5: extremely effective. Likert scale anchors for relevance: 1: never, 3: occasionally, 5: always. ER = emergency room; OR = operating room.

Resident and surgeon teaching (informal and formal) were considered the most relevant interventions by graduates and surgeon respondents (Fig. 5). Graduates ranked informal teaching by residents as the most effective teaching intervention for medical students, whereas surgeons ranked informal resident teaching and informal surgeon teaching equally. Graduates considered the grand rounds and Web-based learning to be ineffective means of learning. Grand rounds were ranked lowest in terms of effectiveness by surgeons.

FIG. 5. Effectiveness and relevance of teaching interventions according to graduates (n = 120) (A) and surgeon educators (n = 42) (B). Likert scale anchors for effectiveness: 1: did not learn, 3: usually effective, 5: extremely effective. Likert scale anchors for relevance: 1: never, 3: occasionally, 5: always.

Discussion

In this study, we assessed the learning needs of contemporary medical students in surgery by surveying recent medical school graduates. Laxdal and colleagues¹⁰ described a learning need as a gap between current and optimal competence or performance. The perceived learning need represents the perspective of the learner, while the true learning need is determined more objectively. Polling learners to capture their opinions and experiences is considered a highly effective approach to establishing learning needs and forms the basis for this needs assessment study.

Recent medical school graduates believed that basic surgical principles and skills are highly relevant in their current medical practice (Fig. 1, Fig. 2). However, they also believe that acquisition of these same basic principles and skills is largely inadequate during a surgery clerkship. Our data suggest that proficiency may be achieved by graduates in only one-third to one-half of surgical principles and one-third of basic surgical skills. There is good evidence from several other studies to suggest that basic surgical skills are not well taught during undergraduate surgery training.^1–5 In their needs assessment study of a large American undergraduate surgery curriculum, Da Rosa and others⁶ identified similar deficiencies in skills training. These findings led to important curricular changes at the authors' institution.

It remains to be seen why essential elements (basic skills and surgical principles) of an undergraduate education in surgery may not be given appropriate emphasis during a clerkship program. Surgeon educators may not realize the importance and significance of instruction in basic principles and skills or may defer to more complex surgical topics.⁶ Focused educational sessions on surgical principles may not always be offered at each educational site. Time constraints in the operating room often result in medical students having less opportunity to learn and practice basic surgical skills. Although this suggests that there may be a need for further training in basic surgical skills and principles, it may also imply a need for educational interventions directed at the surgeon educator and the surgery resident. The importance of learning various teaching techniques that suit the limited opportunities for discussion and informal teaching opportunities on a surgery service have been discussed previously.^11–13 This is further underscored by the data presented in Figure 4 and Figure 5, which indicate the importance placed on informal teaching environments (emergency department, outpatient clinics) and teaching interventions by residents and surgeons.

Currently, there are no guidelines in the literature regarding the appropriate mix of general surgery and specialty experience for undergraduate surgical education. Generalist physicians or primary care doctors have repeatedly called for a broad-based curriculum with focused learning objectives in specialty-specific topics in surgery.^2,4 Other than referencing traditional curricula and current “standards,” there is no compelling reason to restrict medical student exposure to a general surgery service during the surgery clerkship.¹⁴ Others have demonstrated that nontraditional subjects, i.e., cardio-thoracic surgery, may be successfully introduced into the surgery clerkship.¹⁵ Our data indicate the potential for all surgical services to play an important role in undergraduate surgical education (Fig. 3). In this study, graduates and surgeons indicated that subspecialty surgeons can offer an appropriate educational experience for medical students. We believe that an effective rotation on a surgical specialty service must be structured with site-specific goals, objectives and rotation descriptions. This would include an indication of teaching opportunities and experiences offered to medical students, along with chosen methods of evaluation. These formal descriptions have now been requested from all surgical services offering educational experience at McMaster University. Given the current time restrictions in a surgery clerkship, incorporating all surgical specialties as components of a curriculum is not feasible. To address these issues appropriately, McMaster University now offers structured elective experiences coupled with enhanced large group teaching sessions.

As a result of this needs assessment, additional large group sessions were added to the McMaster undergraduate surgery curriculum. These include sessions discussing basic perioperative patient care and surgical principles. These sessions have been added to the clerkship Web site and can be used as a reference site for medical students. Work is also underway to coordinate skills sessions specifically directed toward medical students' needs.

In this study, we assessed the perceived learning needs for contemporary medical students in surgery by surveying recent medical school graduates. Our results show that recent graduates and surgeon educators believe that basic surgical skills and surgical principles are highly relevant to a generalist physician in clinical practice. We found, as others have, that the level of proficiency in these basic surgical principles is less than anticipated. This suggests that many programs face similar challenges of integrating a wide array of surgical topics into undergraduate education while maintaining effective instruction in the basic surgical principles and skills most relevant to generalist physicians. Finally, the information from this study has directly influenced our program structure and has guided the implementation of important curricular changes.

Competing interests: None declared.

Correspondence to: Dr. Daniel W. Birch, The Royal Alexandra Hospital, 10240 Kingsway, Edmonton AB T5H 3V9; fax 780 735-4771; dbirch@ualberta.ca