Evaluation of a Transition to Practice Program for Neurosurgery Residents: Creating a Safe Transition From Resident to Independent Practitioner

Abstract

Background

In 2004, the Department of Neurosurgery at the University of Florida implemented a major curriculum innovation called the Transition to Practice program. This program was established to prepare residents to more safely transition to the role of independent practitioner.

Methods

A qualitative and quantitative evaluation of the program was conducted after its fifth year using online surveys and interviews. Study participants included Transition to Practice graduates, faculty, and current residents.

Results

Of the 26 respondents, 89% of faculty and all graduates were very satisfied with the program. Strengths identified included an independent yet mentored broad operative experience, the development of self-confidence, and a real sense of responsibility for patients. Medical billing and coding instruction and career mentoring were areas of the program that required additional attention.

Conclusion

Overall, this program is meeting the stated objectives and is well received by the graduates and faculty. Based on the results of this evaluation, curricular changes such as instructions in practice management and implementation of a career-mentoring program have occurred. The Transition to Practice program is a unique curricular response to change that other surgical specialties may find useful in addressing the current-day stresses on graduate medical education.

Introduction

The chief resident year represents the culmination of the neurological residency experience and is the final step prior to transition to independent practice. The Accreditation Council for Graduate Medical Education (ACGME) Neurological Surgery program requirements state that residents in this final year:

“… must… spend a 12-month period of time as chief resident on the neurological surgery clinical service in the sponsoring institution or its approved participating sites. The chief resident must have major or primary responsibility for patient management with faculty supervision. The chief resident should also have administrative responsibility as designated by the program director. The specific portion of the clinical training that constitutes the 12 months of chief residency must be specifically designated as the chief residency.”¹

Because of societal demands for more direct faculty involvement, medicolegal pressures, and government regulations regarding billing and compliance, the ability to meet these requirements has presented a major challenge to neurosurgical graduate education.² With the implementation of the ACGME Common Duty Hours Standards in 2003, additional pressure has been applied to neurosurgery and other surgical specialties. There have been conflicting reports regarding the impact of duty hours on residents' surgical experience. Some have asserted that surgical disciplines have been especially affected by the need for time-intensive, hands-on procedural training and the high acuity of many patient illnesses.^3–6 As we have previously reported, a diminution in scores of certification examinations has been observed.⁷ A survey of neurosurgery program directors and chief residents reported that 96% of neurosurgery residency directors and chiefs believed that the duty hours restrictions had compromised resident surgical training.⁸

There are additional external pressures on the educational processes necessary to develop safe and competent neurosurgeons. The Institute of Medicine, in its monograph, Resident Duty Hours: Enhancing Sleep, Supervision, and Safety,⁹ defines the underlying principles of graduate medical education to be: (1) gradual or graded assumption of responsibility for patient care while under supervision; (2) adequate time to engage in reflective learning; (3) sufficient continuity of individual patients to understand the natural evolution of illness; and (4) reinforcement of professionalism and its obligations.

Though the ACGME program requirements and duty hour limits and the Institute of Medicine principles of graduate medical education are sound, finding time to meet all goals and mandates has created significant stress on traditional pedagogical methods in surgical education. This conundrum led to an examination of our residency program and the resultant development of a new program, the Transition to Practice program (TTP). The TTP program provides senior-level residents the experience of primary responsibility for patient management and continuity of care in a mentored environment along with a series of regular activities to promote reflective learning and reinforce the behaviors of professionalism.

The University of Florida Transition to Practice Program

In 2004, the University of Florida Department of Neurosurgery responded to the challenges noted here with a major curricular change.² It was possible to satisfy all requirements for board eligibility for neurosurgery within 6 postgraduate years. Presently, the requirements include 1 year of fundamental clinical skills that includes 3 months of education in neurology and 3 months in general patient care, 42 months of clinical neurosurgery education, and the remainder with education in neuroradiology, neuropathology, research, or other appropriate subject matters. Our program had been previously approved for 7 years of training, and we could ensure that our residents were “board-eligible” at the end of their sixth year by transitioning to a rotation schedule that moved the required rotations to the first 6 years. In essence, the chief residency year was moved from postgraduate year-7 to postgraduate year-6.

Preparing residents to safely transition to the role of independent practitioner requires a broad set of curricular objectives (box). For the preceding 6 years, the resident's schedule was proscribed with an assigned time to be in the operating room (OR), a time to be on the wards, and to be in conference. For the first time, TTP residents must develop, implement, and manage a schedule of their own as junior members of the faculty. A weekly schedule for the TTP experience encompasses the spectrum of clinical care from initial outpatient evaluation to surgical treatment and to postoperative follow-up. Each week the TTP resident spends 1 day in an outpatient clinic setting evaluating new patients and seeing postoperative patients for follow-up. It is in this setting that the TTP resident has opportunities to learn the nuances of an outpatient practice such as time management, patient and family communication, and the essential components of coding and billing. During clinic time, TTP residents must pace themselves and coordinate the ancillary staff to ensure a smoothly functioning schedule. One designated OR day is assigned to the TTP resident each week. This segment of the curriculum allows the TTP resident to not only hone surgical skills but also to experience surgical planning, scheduling, and current procedural terminology coding. Each surgical case is reviewed preoperatively by the TTP resident with a faculty mentor and then postoperatively at a biweekly case presentation conference with all faculty. The biweekly postoperative conferences provide the TTP resident with an opportunity to review and reflect on each surgical procedure, not only from the technical perspective but also from the perspective of surgical decision making, outcomes, and management of complications.

box Transition to Practice (TTP) Objectives

1. Perform common neurosurgical procedures with minimal or no assistance.

2. Enter practice as competent neurosurgeons.

3. Successfully manage and run an outpatient clinic.

4. Add to the participant's fund of knowledge regarding neurosurgical patient management.

5. Handle a large patient volume.

6. Handle the business aspects of practice.

7. Master current procedural terminology coding.

8. Function as a member/leader of a health care team.

9. Use resources of the health care team in patient management.

10. Critically analyze patient complications.

11. Practice evidence-based medicine principles.

To date, 8 residents have completed the program, with 6 opting to leverage the built-in flexibility of the program length (ie, 6 or 12 months) to obtain additional specialty training. We conducted an evaluation of the program, now in its sixth year, and report our findings in this article.

Methodology

We used a survey and additional interviews to assess the TTP program. The evaluation included the following components: (1) determine if the objectives were met, (2) identify the strengths and weaknesses, and (3) determine what, if any, changes should be made. Quantitative and qualitative methods were employed. Tools to carry out the evaluation included an anonymous online survey and one-on-one interviews with program stakeholders, which are graduates, current departmental faculty, and residents. Participants responded to the survey questions using a 5-point Likert scale (5  =  strongly agree, 4  =  agree, 3  =  neutral, 2  =  disagree, and 1  =  strongly disagree), with additional open-ended questions. The purpose of the interviews was to gather any additional information that could either clarify or validate the survey results and to test any preliminary themes.

The interview questions focused on the respondent's overall evaluation of the program and suggestions for program improvements. Data from the interviews and open-ended questions on the surveys were coded and analyzed by 1 author (G.J.L.) using constant comparison and linguistic analysis to identify common themes.¹⁰ A second author (J.R.L.) reviewed the data sets and confirmed the themes. Surgical case data, including case logs, case mentors, and complications, were drawn from departmental resources from 2004 to 2008. The study received approval from the University of Florida Behavioral/Nonmedical Institutional Review Board.

Rigor is important in qualitative studies. Guba and Lincoln¹⁰ proposed 4 criteria for determining the trustworthiness of qualitative research: credibility, transferability, dependability, and confirmability. The credibility of this study was addressed by member checking.^10,11 After reviewing the survey results and interview data, 1 TTP program graduate and 1 current participant agreed the analysis adequately depicted the TTP program. Transferability was not an issue because the goal of this study was an objective-based evaluation of a program. Dependability of the study was established using overlapping methods of survey and interviews. Confirmability is concerned with objectivity of the study. Triangulation promotes conformability by reducing any potential investigator bias.¹² Triangulation strategies included interviews and anonymous surveys, multiple respondents, and surgical case logs. The surgical logs of the TTP graduates were analyzed to determine the number and type of surgical procedures performed. Complications data were received from the Shands at the University of Florida quality assurance department.

Results

Return rates for the survey were 68% (19 of 28) for faculty and residents and 88% (7 of 8) for program graduates. One-on-one interviews were conducted with 8 faculty members, 5 residents, and 4 graduates by 1 of the authors (G.J.L.). Scores on how well the programs' goals were achieved ranged from 3.00 to 5.00 (scale, 1 =  strongly disagree to 5  =  strongly agree) (tables 1 and 2). Graduates responded with strongly agree or agree on 95% to 100% in all items, except for 1 item on the business aspects of the program. Faculty and residents responded with 89% to 100% agreement on all items except for the same question on business aspects of handling a practice. Graduates of the TTP program rated the business aspects of practice lower than the faculty or residents, with only 28% of the graduates indicating that they agreed with the statement.

Table 1.

Survey Data: Current Residents and Faculty^a

Table 2.

Survey Data Graduates^a

The highest rated item by graduates for both surveys was the item, “Overall, I am very satisfied with the Transition to Practice program,” with 100% of the graduates responding that they “strongly agreed” with the statement. For the same item, 89% of the faculty and residents responded, “strongly agree” and 11% rated this item as “agree.” The lowest ranked item on both faculty/resident and graduate surveys was the item, “The TTP program prepares residents to handle the business aspect of their practice.” Only 45% of the faculty and residents rated this as “strongly agree” or “agree” (tables 1 and 2).

Operative and Postoperative Experience

Surgical case logs showed the TTP resident gained operative experience across the entire spectrum of neurosurgery. Eight TTP graduates performed a total of 1776 operative procedures (table 3). The largest individual experience was 417 cases, while the smallest, 125 cases, was experienced by a surgeon who spent only 6 months as a TTP resident before taking additional specialty training at another institution. The case mix wherein spine surgery was most frequent is consistent with what one would expect to see in a routine general neurosurgical practice. The large number of vascular cases reflects the additional time the TTP resident spent on our high-volume endovascular neurosurgery service. Twelve different faculty members served as mentors, with a range of 3 to 270 cases and a mean of 93 cases. Because of surgical specialization within our departmental faculty, the director of our spine surgery service mentored 26% (270) of all cases.

Table 3.

Transition to Practice Surgical Case Log 2004–2008

Review of postoperative follow-up records for patients cared for by the TTP resident revealed 1263 discharges. Infection rate for patients of TTP physicians was 1% (15) and the return to OR within 48 hours rate was 2%. Mortality rate for patients cared for by the TTP resident was 7%, with a combined mortality observed-versus-expected ratio of 1.07.

Program Strengths and Weaknesses

Program strengths identified by the respondents in the survey and interview included:

• Independent operative experience with back-up available

• Build self-confidence as a neurosurgeon

• Sense of responsibility for patients

• Continuity of patient care

• Ability to demonstrate competence

• Opportunity to lead the patient care team

• Opportunity to teach residents in the OR

• Flexibility in the length of the program and the opportunity to do fellowships

Program weaknesses identified included:

• Insufficient training in medical billing and coding

• Lack of an overall mentor

• Lack of subspecialty surgical cases

• Lack of scholarly output by TTP residents

• Lack of common format for recording of surgical cases by TTP residents

Discussion

Ensuring that a resident possesses the necessary knowledge and skills to be declared competent to engage in the independent practice of neurological surgery is a very complex process. Attaining competency in performing a certain operative procedure is but one of the skills necessary to function as a competent neurosurgeon. Skill in managing the preoperative outpatient and postoperative phases of clinical practice must be developed. Acquisition of the knowledge necessary to run the business aspects of a neurosurgical practice is essential. Our study suggests that all the aforementioned skills were satisfactorily attained by the graduates of the program, with the exception of some of the skills necessary to run the business. Faculty and current resident perceptions of the program were very similar. The Dreyfus model suggests a continuum of progressive stages through which one must pass to attain competency in a certain subject.¹³ Ericsson¹⁴ suggests that time is but one variable in the pathway of learning and that level of performance for individuals in a given domain is not attained automatically as a function of extended experience, but the level of performance can be increased even by highly experienced individuals as a result of deliberate efforts to improve.

The TTP program is a viable solution to the conundrum of educating neurosurgical residents in today's complex health care environment. Additional benefits beyond the stated objectives were also observed. “Experiencing for the first time, the full burden of patient responsibility” was reported as a significant benefit. Many TTP residents experienced enhanced self-confidence as the leader of a patient care team. Opportunities to tailor the program to individual needs and allowing the flexibility to pursue other specialty training were viewed positively.

A key question in evaluating any graduate medical education program is: “How does the TTP program address the principles of graduate medical education?” Gradual or graded assumption of responsibility for patient care is a hallmark of the program. Mentors provide as much supervision as is deemed necessary or desired by the TTP resident.

Engaging in reflective learning, a core component of any profession, is well met with the TTP Case Conference. The TTP residents must review all operative cases with the faculty and resident staff, including their thought process in devising and implementing a treatment plan and the results of that plan. Mentors provide the TTP residents with opportunities to reflect-in-action and reflection-on-action regarding their surgical decision-making skills.¹⁵

A physician and patient jointly pursuing high-quality and cost-effective management of the patient's problem is a basic tenet of continuity of care. During the TTP year, residents gain significant experience by following a patient from an initial outpatient consultation through preoperative assessment, the operative procedure, and postoperative care. This continuum, not easily accomplished in today's graduate medical education environment to the degree experienced in the TTP, is an invaluable experience and one of the noted strengths of this program.

There is no definitive definition of professionalism. The ACGME mandates demonstration of certain behaviors and the adherence to certain principles in pursuit of professionalism, yet it does not state a definition. Demonstrating respect, compassion, integrity, and accountability to patients, society, and the profession is evidence of professionalism. A commitment to ethical principles in the provision of care or withholding care is evidence of professionalism, as is a sensitivity and responsiveness to the culture, age, gender, and/or disabilities of a patient.¹⁶ During the TTP year, residents have the opportunity, through mentored autonomy, to clearly understand the responsibilities and behaviors of a true professional. As one TTP graduate stated, “You have more patient ownership–you are their doctor.”

Business-related objectives were not well met. The program's lack of education on the business aspects of being a neurosurgeon in dealing with coding and billing was frequently mentioned. With this feedback, programmatic changes have been implemented to send each TTP resident to nationally recognized courses on coding, billing, and compliance. Some graduates noted that additional education in contracts and negotiating would have been useful. The curriculum is being expanded to include instruction in additional business activities, such as the creation and maintenance of a referral base, working with a practice manager, and tools for marketing. Although surgical case mentoring was viewed as extremely beneficial, the graduates recommended assignment of a mentor with whom the TTP resident could discuss career questions and issues. This recommendation has been implemented.

Faculty members reported the TTP residents' lack of engagement in scholarly activity, which is an overarching goal of our residency program. We are addressing whether this is another manifestation of insufficient career mentoring. A practical issue identified during this study was lack of a common recording format for all TTP activities. A common format is being devised to assist in continued program evaluation.

Conclusion

The TTP program is a unique curricular response to changes in graduate medical education. Program graduates, faculty, and residents reported that the stated objectives were being well met except for instruction in certain components of medical business. All groups were very satisfied with the program, reporting that it provided an opportunity to “perform neurosurgery with a safety net” or with “supervised independence,” as described by 2 graduates.

This evaluation has resulted in 4 substantive program changes:

1. A formal mentoring program has been implemented.

2. A practice management curriculum has been implemented.

3. A standardized surgical case log for TTP has been implemented.

4. A scholarly activity objective has been added.

This program is an innovative response to change and is a model that other surgical specialties may find useful in addressing the current day stresses on graduate medical education.