Restructuring the Orthopedic Resident Research Curriculum to Increase Scholarly Activity

Abstract

Background

Limited time and funding are challenges to meeting the research requirement of the orthopedic residency curriculum.

Objective

We report a reorganized research curriculum that increases research quality and productivity at our academic orthopedic medical center.

Methods

Changes made to the curriculum, which began in 2006 and were fully phased in by 2008, included research milestones for each training year, a built-in support structure, use of an accredited bio-skills laboratory, mentoring by National Institutes of Health–funded scientists, and protected time to engage in required research and prepare scholarly peer-reviewed publications.

Results

Total grant funding of resident research increased substantially, from $15,000 in 2007 (8 graduates) to $380,000 in 2010 (9 graduates), and the number of publications also increased. The 12 residents who graduated in 2005 published 16 papers from 2000 to 2006, compared to 84 papers published by the 9 residents who graduated in 2010. The approximate costs per year included $19,000 (0.3 full-time equivalent) for an academic research coordinator; $16,000 for resident travel to professional meetings; reimbursement for 213 faculty hours; and funding for resident salaries while on the research rotation, paid through the general hospital budget.

Conclusions

The number of grants and peer-reviewed publications increased considerably after our residency research curriculum was reorganized to allow dedicated research time and improved mentoring and infrastructure.

What was known

Existing models for research curricula in orthopedics are challenged by limited time and lack of funding.

What is new

Research curriculum incorporated milestones for each training year, formal support structure, mentors, and protected time for residents to engage in required research activities.

Limitations

Curriculum was implemented at a single institution, limiting generalizability.

Bottom line

A restructured orthopedic resident research curriculum resulted in increased numbers of grants and peer-reviewed publications.

Editor's Note: The online version of this article contains more detailed descriptions of the outcome measures and the literature search on barriers to resident research ^{(34KB, doc)} and the development of the clinician-scientist that guided the curricular intervention described in this study.

Introduction

The research component of orthopedic resident curricula is challenged by the lack of supportive infrastructure, the diminished funding for research projects, and the perception that research is time taken away from both patient care and training in the basics of orthopedics.^1–5 All orthopedic training programs accredited by the Accreditation Council for Graduate Medical Education (ACGME) must provide an environment for scholarly activity, including research, which is an important element of several of the core competencies. For academic centers such as ours that are committed both to helping residents develop a critical knowledge of research and its methodology and to nurturing clinician-scientists, it is especially important to offer research opportunities as part of the training program. Nationwide, few residents enter a career as a clinician-scientist, and even these small numbers are shrinking.^4–7

At many institutions, resident research curricula are underdeveloped or even nonexistent, with low publication rates.^8–11 The factors that affect resident research productivity have been explored in a number of studies, but we were unable to find a curriculum in the literature that included all the elements considered most useful.^9–11 This article describes a model of a successful, revised resident research curriculum, as assessed by the primary benchmarks of number of grants received and papers published. One rationale is the benefit provided by exposing residents to research. Enhancing resident participation in research addresses 3 of these ACGME competencies:¹

1. Medical knowledge: research fosters an investigatory and analytical approach to clinical situations.

2. Practice-based learning and improvement: expertise with research techniques allows residents to apply their knowledge of study designs and statistical methods to the appraisal of clinical and improvement studies with relevance to patient care.

3. Professionalism: residents learn the importance of adhering to ethical principles in conducting research.

Methods

Setting

In 2010, our urban academic teaching center performed more than 25 000 surgical orthopedic procedures, and there were more than 275 000 outpatient visits.

Curriculum Development

In 2005, we created an interdisciplinary Resident Research Curriculum Committee to reorganize the research curriculum around a detailed series of milestones for each postgraduate year (PGY). Members included faculty with appointments in the research division, basic scientists, clinical orthopedists, statisticians, and epidemiologists. We instituted an extensive support network (eg, individual mentorship by research faculty, appointment of an expert research coordinator available for guidance) and protected research time. In addition, our core didactic research lectures were reevaluated and enhanced. The aim of these efforts was to teach basic research principles and supply the expertise needed to successfully navigate the challenges and administrative requirements of a research project and a grant proposal.

We describe the administrative infrastructure, the curriculum, and the block schedule for protected time that we developed to support residents' research activities.

The Hospital for Special Surgery Institutional Review Board (IRB) deemed this project exempt from review.

Prior to the changes, each resident had to find a research mentor, frame the topic, navigate the institutional review process, and apply for funding. We observed that given the work hour restrictions, residents had little time to go through this lengthy process, perform the research, and produce the subsequent scholarly papers without a formal structure. In addition, the research process has become more complex, partly because of the requirements set out by the Health Insurance Portability and Accountability Act, the IRB, and the Institutional Animal Care and Use Committee (IACUC).

Recognizing the difficulties residents were encountering, we endorsed the recommendations from our Resident Research Curriculum Committee and approved (1) the establishment of an administrative infrastructure with research expertise that would provide support and guidance to the residents for successful completion of their projects; (2) the development of specific requirements and criteria for successfully completing research projects (the curriculum); and (3) the provision of protected time for research.

Support Infrastructure

Administrative Support

A designated position of academic research coordinator was established, requiring a background in research and an advanced degree in health education, public health, or a similar field. The position serves as liaison between our Resident Research Curriculum Committee and the residents, coordinates the research education sessions led by hospital faculty, and guides residents through their IRB/IACUC submissions and grant applications, including the creation of a research budget. The academic research coordinator reports to the director of academic training.

Bio-Skills Education Laboratory

The bio-skills education laboratory (BSEL) provides state-of-the-art education to residents and other clinical staff and was certified by the American College of Surgeons in 2012. BSEL education includes surgical procedures with simulated use of sawbones, cadavers, and an arthroscopy simulator (with knee and shoulder components), as well as familiarization with surgical instrumentation and equipment. Residents have priority access to work on their research projects, especially biomechanical and anatomical projects. The BSEL operates under a blended funding model (ie, hospital operating budget, grants, and fees).

Faculty Support (Mentor)

Each resident chooses an established clinician-scientist who serves as a mentor to guide the project. Initially, mentors meet with residents at least twice to discuss the project and plan the details, and then they identify potential co-investigators, review and critique the grant application to assess quality and fundability, and use bio-statistical and methodology assistance from our research cores. The mentors ensure that the research can be completed during an approximate 24-month time frame. The whole process from education through publication takes 2 to 3 years. Residents are required to submit grant applications to various agencies. If a grant is not awarded, then mentors can either support the project through their subspecialty service or recommend the project to the surgeon-in-chief and the designated institutional official of graduate medical education for funding consideration. Mentors also write letters of support, which are used for funding proposals and other research-related initiatives.

Current Research Curriculum

We created the curriculum in 2005 and put the staff infrastructure in place. In 2006 and 2007, several phases were piloted and refined, and in 2008, the curriculum became a formal requirement. It is organized into annual milestones that residents are expected to meet for each of the 5 PGYs (box 1).

Box 1 Annual Milestones in the Resident Research Curriculum

• PGY-1: Certification course on the ethics of research involving human subjects; introduction to current research projects at our center; identification of potential mentors

• PGY-2: Basic or clinical research area and mentor chosen

• PGY-3

• • First quarter: Discussion with mentor to identify appropriate research project

  • Second quarter: Preparation of grant proposal

  • Third quarter: Submission of proposal to the institutional clinical review panel for critical feedback; after revision, submission to the Institutional Review Board or the Institutional Animal Care and Use Committee, if appropriate

  • Fourth quarter: Submission of grant proposal draft to mock grant review led by 8 to 10 basic and orthopedic surgeons/scientists who give feedback/direct mentoring

• PGY-4: Submission of revised application to the academic research coordinator 1 month before submission to Orthopaedic Research and Education Foundation or a comparable funding agency

• PGY-5: Completion of research, data analysis, and final manuscript by end of academic year; poster and podium presentations at the annual Chief Residents Presentations Meeting, with faculty scoring; submission of completed manuscript to a peer-reviewed journal

A handbook outlining the milestones and providing frequently asked questions and responses was created. The handbook includes expectations for both residents and mentors and is a road map for how to complete a research project. Residents are also provided with 10 criteria by which their presentations will be scored.

Core Didactic Research Lectures

Starting in 2005 under the direction of our Resident Research Curriculum Committee chair, the research methodology lectures were redesigned and now include a series of eight 1-hour lectures given at the start of the academic year to allow incoming orthopedic fellows and residents to participate. Topics include research design, navigating the IRB process, critical appraisal, and basic research methods (ie, statistical design). We offer an introduction to IBM SPSS software (IBM Corp, Pittsburgh, PA) and a hands-on SPSS clinic.

Protected Time

The rotation schedule for all residents includes a dedicated research block and an elective block that can also be used for research, which range from 6 to 7 weeks each (ie, 12 to 14 weeks, or at least 960 hours every year). These blocks are designed to meet the requirements for external funding, such as the Orthopaedic Research and Education Foundation (OREF) grant. Residents also have the option to pursue a research year free from clinical training to gain additional academic and/or research experience.

Grants and Grant Selection

All residents are required to apply for the OREF Resident Clinician Scientist Training Grant or comparable grant. We provide support at the end of PGY-4 through 3 faculty-named grants. These are subject to the same rigorous criteria used by the external granting agencies.

Costs

Estimated annual cost of the program is $35,000 ($19,000 [0.3 full-time equivalent] for the academic research coordinator and $16,000 for resident travel to professional meetings). When the residents are on their research rotations, the Hospital for Special Surgery is responsible for their salaries through its operating budget. The estimated total faculty time commitment per year is 213 hours. Faculty who devote time to teaching and mentoring residents receive an annual stipend from the hospital for their academic efforts.

Evaluation and Analysis

To assess the impact of the new curriculum and the support structures, we measured the research output of residency graduates before and after 2005 by searching PubMed. The academic research coordinator conducted interviews with PGY-4 and PGY-5 residents to assess and improve the program, and the Resident Research Curriculum Committee discussed information from the interviews. Data and feedback from residents were reviewed by Resident Research Curriculum Committee members and the residents themselves.

Results

Funding

Grant funding increased substantially after the enhanced research curriculum was established. As shown in table 1, the total amount of grants awarded increased from $15,000 for eight 2007 graduates to $380,000 for nine 2010 graduates. The individual grants ranged from $2,000 to $150,000.

TABLE 1.

Amount of Grant Funding Since Changes in the Research Curriculum^a

In addition, residents began to submit more than 1 research proposal, peaking in 2010 with 9 graduates submitting 17 applications (table 2).

TABLE 2.

Number of Grants Submitted Before and After Changes in the Research Curriculum^a

Publications

To determine the number of publications, we searched PubMed for articles in which the resident was listed as an author. The number of publications increased substantially after the new curriculum was established (table 3). The twelve 2005 graduates had a total of 16 publications from 2000 through 2006, whereas the nine 2010 graduates published 84 papers from 2005 through 2011.

TABLE 3.

Number of Publications Per Resident Before and After Changes in the Research Curriculum^a

Discussion

Our study demonstrated a significant increase in the number of grants and publications after implementation of the new residency research curriculum. In addition, an environment more conducive to scholarly activity was established. Studies in internal medicine have defined the useful elements of a research program as the existence of a curriculum, designated funding, a research director, protected time, a research committee, mandatory research requirement, mentorship, training in research methods (eg, through classroom teaching), support from the program director, chances to present the research, and a journal club.^9–11 Our revised curriculum included all of these elements, and we added 2 vital components: (1) expanded “mentorship” by scientists from our research division, who are actively engaged as mentors, advisors, and reviewers, and commit not only their time but also their passion for science; and (2) giving residents priority access to the BSEL and its expert staff, including the ability to schedule time in the laboratory that fits around their clinical obligations.

Programs often integrate research longitudinally,⁹ but the use of block schedules has been shown to correlate positively with the number of grants and publications produced.¹⁰ Those who had block time received more grants and had more publications than those who did not (29% published versus 8%, respectively).¹⁰ We felt that a continuum would be more productive, and we took the longitudinal approach, beginning the research curriculum in PGY-1 and continuing through PGY-5. A hypothesis-driven project may require a longer time commitment than a rotation can provide. Rotations may work for programs that are fulfilling the research requirement by topic review, case report, and/or literature review.

Many programs do not have the resources to institute all the changes described here, and they may not have an emphasis on developing clinician-scientists. We feel that certain elements of our revised curriculum could be useful, the most important of which is given as the first item in box 2.

Box 2 Lessons Learned From Our Experience

The program leadership, the faculty, and the residents themselves deemed the following issues important based on formal and informal evaluations:

• Critical components of our program were protected time for research, dedicated administrative and research expertise/support, and mentorship

• Institutional Review Board complexities were solved by help from more experienced senior scientists and from the academic research coordinator, with a key element being to learn the basics of a good research proposal; need for increased funding was solved by creating internally funded named grants from clinical revenues

• Difficulty in finding time for grant-writing workshops was eventually solved by setting up a self-study course followed by meetings with 2 senior scientists with extensive grant experience to discuss and refine the specific aims section of proposals

• Need for greater expertise in writing grant proposals, possibly through participating in external grant writing programs

• Need for the program director to continually safeguard the protected research time

A limitation of our study is that we did not have a comparison group of residents at a similar institution. Also, as our intervention was multi-factorial, it was difficult to prove which critical elements led to our outcomes.

Conclusion

Our orthopedic resident research curriculum was restructured to allow dedicated research time as well as improved mentoring and infrastructure. The number of grants and peer-reviewed publications increased considerably.