Abstract
Background
The primary aims of this study were to: 1) assess barriers and facilitators of completing scholarly projects from residents and faculty mentor perspectives, 2) determine the perceived value of new initiatives designed to support resident scholarly projects and 3) determine if these initiatives led to changes in resident publications.
Design and Methods
Between June and September 2014, we surveyed 18 paediatric residents and 41 faculty mentors regarding barriers to resident scholarly project completion and the value of new initiatives to support scholarly activity. We also tracked scientific publications by residents before and after implementation of these initiatives.
Results
The primary perceived barriers to research for residents and faculty were lack of protected time (64.3% versus 68.6%, respectively), lack of resident interest in scholarly activity (50.0% versus 60.0%, respectively) and lack of mentor motivation. Mentors and residents did not agree that lack of proper training in research (29% versus 54%, respectively) and faculty motivation (29% versus 17%, respectively) were barriers to completing a project. A dedicated research coordinator (71.4% versus 70.6%, respectively), a revised research curriculum (71.4% versus 41.2%, respectively) and works in progress sessions (50.0% versus 61.8%, respectively) were perceived as valuable initiatives to the program. These initiatives were not associated with changes in annual resident publication rates.
Conclusions
Lack of time and competing clinical training are primary barriers to scholarly project completion for residents in addition to a lack of motivation on the part of faculty members. Improving program support was perceived as positive changes to address these barriers but did not increase resident publication rates. The information provided here could be used to tailor future resident research programs and highlight the value of gathering input from resident and faculty when designing initiatives to enhance resident research productivity.
Keywords: Resident research, Medical education, Mentorship, Scholarly activity
The CanMEDS Physician Competency Framework encompasses seven core roles that are developed throughout residency training including the role of scholar. The role of scholar encompasses core concepts of lifelong learning, teacher, evidence-informed decision-making and research (1–3). These concepts are mirrored by the ‘Objectives of Training in Paediatrics’ to develop key competencies within all Canadian paediatric resident training programs (2,3). These aim to foster interest and skills in research but also to provide the ability to evaluate and integrate new research into practice. This is, in part, accomplished through the completion of a scholarly project (1) and applying knowledge in the form of testing a research question. Unfortunately, accomplishing a scholarly project is viewed as a challenge for residents within paediatric programs (4–6). A better understanding of the barriers and determinants to completing a scholarly project may help guide the process of harmonizing expectations and developing programs to support completing a scholarly project within paediatric resident programs in Canada.
The barriers and facilitators to completing a resident scholarly project have been described in other medical disciplines (7,8) and paediatric programs in the USA (5). The primary perceived barriers to completing scholarly projects include: lack of time, lack of resident motivation or interest, inadequate knowledge and skills related to research (study design and methodology), lack of resources and inadequate mentors. Perceived facilitators to completing a scholarly project include: provision of protected research time (4), a structured curriculum (5,6,9) and research support provided by an active research director (7). To date, there are no published surveys of the barriers to completing a scholarly project among Canadian paediatric residents and, to the best of our knowledge, none of the surveys of residents in any discipline have ever attempted to triangulate responses with faculty mentors.
Several programs have published their experiences revising curriculum or initiating innovations to enhance resident scholarly productivity (4,9–13). These include the formation of a research team/committee, changes to curriculum design, resident-led initiatives, reward systems and increasing protected time in longitudinal research blocks. In some cases, these innovations increase resident research output as defined by oral project presentations or publications (10,11,13). Despite these observations, there are several gaps in our understanding of the factors that influence the completion of a scholarly project for paediatric residents. These include, the perspectives of faculty members, whether the perceived barriers and facilitators of faculty members are similar to those of paediatric residents and the benefits of novel initiatives to enhance timely project.
The primary aims of this study were to address these limitations and to identify the barriers and facilitators to successful completion of a paediatric resident scholarly project from both the resident and mentor perspective, to determine the perceived value of a series of initiatives designed to enhance resident scholarly project output within our program and to describe rates of resident publications in the 10 years before and 4 years following implementation of initiatives to enhance scholarly project activity.
METHODS
Study design and population
Residents and faculty members that have supervised residents in the Department of Paediatrics at the University of Manitoba were surveyed in the summer of 2014. All postgraduate year 3 and 4 (PGY3 and PGY4) residents, recent graduates from 2012 and 2013 and all faculty members who had mentored a resident within the past 10 years were invited to complete the survey. Residents and mentors were recruited via e-mails available from the Manitoba Paediatrics Society and the Department of Paediatrics and Child Health. This study was approved by the Health Research Ethics Board at the University of Manitoba.
Survey development and content
Two surveys were developed (Appendix A) specific to residents or research mentors with questions adapted from previous studies (14). The survey covered four of the following general topics: 1) participant characteristics; 2) barriers and facilitators to research completion as divided into resident, mentor, program and institutional factors; 3) the usefulness of current program initiatives and 4) self-identified barriers and facilitators to successful research project completion (free text fields). The survey consisted of 36 questions for residents across the four topics and 24 questions across two general topics, ‘perceptions about research’ and ‘factors that influence research during residency’. All answers for both surveys were provided on a 5-point likert scale. The likert responses were grouped into three categories to represent ‘agreement with the statement’, ‘neutral with the statement’ and ‘disagreement with the statement’ to simplify the interpretation of the survey results. The surveys were reviewed by two residents for their relevance and clarity prior to data collection.
Survey administration
E-mails were distributed via the Department of Paediatrics and Child Health and Post-Graduate Medical Education offices. The e-mails contained an explanation of and consent for the study and a link to the resident or mentor specific online survey (SurveyMonkey, Palo Alto, CA). Responders received a $25 gift certificate for their participation. Two subsequent reminder e-mails were sent 1 week apart, with survey collection spanning a period of 4 months.
Changes to support resident research
Within the paediatric resident program at our institution, several innovations were initiated in 2011 to increase scholarly productivity, including: 1) a revised structured research curriculum to provide research skills; 2) research dinner workshops where faculty researchers were invited to speak, motivate and mentor residents; 3) scheduled works in progress sessions presented by residents to the resident body and members of the research committee to stimulate discussion and problem-solve; 4) a dedicated research coordinator—an institution-nominated dedicated faculty member with protected time and resources to assist with research projects and 5) the mandatory submission of a single research summary page at the end of the PGY1 year by residents and their mentors to evaluate the feasibility and quality of projects.
Publication records
Resident productivity was defined as manuscripts published in academic journals that are indexed in Pubmed and Scopus for which the resident was an author, regardless of level of authorship. Publications in these indexes were identified by a librarian in the Faculty of Health Sciences at the University of Manitoba and double checked by one of the authors. Annual indexed scientific publications in Pubmed or Scopus per resident were tabulated for each separate cohort between January 2001 and January 2017. Records were tabulated and reported as publication rates per resident for each academic program year.
RESULTS
The online surveys were sent to 18 paediatric residents and 41 faculty members, and 89% of paediatric residents and 93% of mentors responded (Table 1). Faculty mentors were predominantly male (55%), practicing in a paediatric subspecialty (84.2%) and had practiced for an average of 10 years. The majority of faculty mentors (60.5%) had supervised less than five paediatric residents through their time on faculty.
Table 1.
Resident and faculty characteristics
| Residents, N=16 (%) | Mentors, N=38 (%) | |
|---|---|---|
| Gender | ||
| Male | 2 (12.5) | 17 (44.7) |
| Female | 14 (87.5) | 21 (55.3) |
| Year of Residency | ||
| PGY3 | 10 (62.5) | -- |
| PGY4 | 3 (18.8) | -- |
| Completed | 3 (18.8) | -- |
| Area of (Anticipated) Practice | ||
| General | 6 (37.5) | 6 (15.8) |
| Subspecialty | 7 (43.8) | 32 (84.2) |
| NR/Undetermined | 3 (18.8) | -- |
| Educational Background | ||
| MD | 13 (81.3) | 17 (44.7) |
| MD + Masters/PhD | 3 (18.8) | 21 (55.3) |
| Years on Faculty (average) | -- | 9.5 ± 2.8* |
| Number of residents supervised | ||
| <5 | -- | 23 (60.5) |
| 5–10 | -- | 9 (23.7) |
| >10 | -- | 6 (15.8) |
NR Not responded
*95% confidence interval.
The resident respondents were predominantly female (87.5%), in their PGY3 year (62.5%), and without an advanced research degree (81.3%). Half of the residents (53.3%) had no academic output at the time of completing the survey, while 26.7% had published their study or were pending publication, 13.3% had presented their study either as a poster or oral presentation and only 6% of residents had completed both a publication and a presentation.
Faculty mentor experiences
Faculty mentors described having a positive experience mentoring paediatric residents through a scholarly project. Of 35 faculty mentor respondents, 45.7% rated the experience as very good and would be willing to mentor additional residents, while 31.4% rated the experience enjoyable but challenging. A small minority of faculty mentors (5.7%) had very poor experiences and would not mentor again, while 17.1% felt that the experience required a lot of work and effort with very little output.
Barriers to completing a research project
Barriers to completing a research project are presented in Table 2. Fourteen residents (87.5%) and 35 faculty mentors (92.1%) responded to all questions. Residents and faculty mentors both felt strongly that residents lacked sufficient protected time for research (64.3% and 68.6%) and they both felt strongly that the majority of resident working hours were spent learning clinical skills (78.6% and 80.0%). Residents and faculty mentors also agreed that residents lacked interest and motivation to complete a research project (50.0% and 60.0%, respectively), though this was a less frequently cited limitation.
Table 2.
Barriers and facilitators to scholarly activity in paediatrics
| Residents (n=14) | Mentors (n=35) | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| SA/A* (%) | N* (%) | SD/D* (%) | N/A* (%) | SA/A* (%) | N* (%) | SD/D* (%) | N/A* (%) | ||
| Resident factors | |||||||||
| Lack of interest/motivation | 7 (50.0) | 3 (21.4) | 4 (28.6) | -- | 21 (60.0) | 9 (25.7) | 5 (14.3) | -- | |
| Lack of protected time | 9 (64.3) | 2 (14.3) | 3 (21.4) | -- | 24 (68.6) | 7 (20.0) | 4 (11.4) | -- | |
| Learning clinical skills took all time | 11 (78.6) | 1 (7.1) | 2 (14.3) | -- | 28 (80.0) | 4 (11.4) | 3 (8.6) | -- | |
| Inadequate training/skills | 4 (28.6) | 4 (28.6) | 6 (42.9) | -- | 19 (54.3) | 6 (17.1) | 10 (28.6) | -- | |
| Mentor factors | |||||||||
| Adequate number of mentors | 8 (61.5) | -- | 5 (38.5) | 1 | 11 (31.4) | 15 (42.9) | 9 (25.7) | -- | |
| Mentors are Interested/ motivated | 5 (45.4) | 2 (18.2) | 4 (36.4) | 3 | 17 (48.6) | 12 (34.3) | 6 (17.1) | -- | |
| Program factors | |||||||||
| Dept of Paediatrics is supportive | 5 (35.7) | 5 (35.7) | 4 (38.6) | -- | 16 (45.7) | 12 (34.3) | 7 (20.0) | -- | |
| CHRIM** is supportive | 10 (71.4) | 2 (14.3) | 2 (14.3) | -- | 29 (82.9) | 5 (14.3) | 1 (2.9) | -- | |
| Lack of research committee support | 1 (7.7) | 2 (15.4) | 10 (76.9) | 1 | 5 (14.3) | 26 (74.3) | 4 (11.4) | -- | |
| Lack of resident monitoring | 2 (14.3) | 1 (7.1) | 11 (78.6) | -- | 22 (62.9) | 9 (25.7) | 4 (11.4) | -- | |
| Lack of strong research curriculum | 3 (21.4) | 2 (14.3) | 9 (64.3) | -- | 18 (51.4) | 9 (25.7) | 8 (22.9) | -- | |
| Institution factors | |||||||||
| Lack of patients to enroll | -- | 2 (25.0) | 6 (75.0) | 6 | 2 (5.7) | 5 (14.3) | 28 (80.0) | -- | |
| Lack of resources | 1 (8.3) | 4 (33.3) | 7 (58.4) | 2 | 13 (37.1) | 2 (5.7) | 20 (57.1) | -- | |
| Lack of funding | -- | -- | 14 (100) | -- | 13 (37.1) | 11 (31.4) | 11 (31.4) | -- | |
| Research awards are an incentive | -- | 2 (14.3) | 12 (85.7) | -- | 17 (48.6) | 13 (37.1) | 5 (14.3) | -- | |
*SA/A = Strongly agree/agree, N = Neutral, SD/D = Strongly disagree/disagree, N/A = Not applicable.
**CHRIM = the Children’s Hospital Research Institute of Manitoba, the research division of the Children’s Hospital Foundation of Manitoba.
Nearly 50% of mentors and only 34% of residents believed that faculty members were motivated to support resident scholarly projects. More than one in four residents (28.6%) felt that faculty mentors lacked interest or motivation in guiding a resident through a scholarly project. Finally, the majority of faculty mentors (54.3%) but a minority of residents (28.6%) believed residents lacked the training and skills required to conduct research.
Institutional and program barriers.
Patient enrolment, research resources (hardware, software, supplies, office space, statistical support) and funding were not considered barriers to a successful project by both residents and faculty mentors (Table 2). At the program level, both residents and faculty mentors felt very strongly that there was support from the research institute (71.4% of residents and 82.9% of mentors) and the local research committee (71.4%).
Residents and faculty mentors did not agree on the perceived institutional support regarding training and monitoring of progress for scholarly projects. A large majority (78.6%) of residents felt that there was adequate monitoring of scholarly projects, however, 64.3% felt there was a sufficient research curriculum. However, faculty mentors reported that there were deficiencies in both monitoring of progress (62.9%) and curriculum (51.4%).
Perceived benefits of recent innovations.
Perceived benefits to recent innovations made to the resident research program are summarized in Figure 1. The most useful innovation perceived by both residents (71.4%) and mentors (70.6%) was having a dedicated research coordinator assigned to track and facilitate resident research progress. This was followed by the inclusion of works in progress sessions to ensure resident projects were on pace for completion and to receive feedback from faculty (residents 50.0%, mentors 61.8%). The revised academic curriculum was viewed by residents as a positive change to the program (71.4%), however only 41% of mentors felt the revised curriculum was a valuable initiative. The submission of a project summary at the end of the PGY1 year (residents 35.7%, mentors 41.2%) to be reviewed by the resident scholarly activity committee and the resident dinner workshops (residents 42.9%, mentors 23.5%) were found to be the least beneficial to support scholarly activity.
Figure 1.
Perceived benefits of various program changes to support scholarly activity. Data are presented as the positive response to a binary response (agree/disagree) regarding the perceived benefits for several changes to the paediatric resident scholarly activity program.
Changes in productivity.
Resident publication of manuscripts in Pubmed and Scopus for each resident cohorts between 2001 and 2015 were quantified. Publication rates ranged from a high of 1.7/resident to 0.16/resident over the 15-year time frame. In the three years prior to program changes publication records were between 0.25 and 1.1/resident and in the 4 years following program implementation were between 0.21 and 0.86/resident. No clear impact of the novel initiatives to support scholarly activity was observed on publication rates.
DISCUSSION
The current survey of paediatric residents and faculty mentors provides novel information that could help guide other paediatric resident programs to facility completion of the CanMeds scholar role. First, the majority of paediatric resident mentors described the experience of completing a scholarly project as positive and enjoyable. While faculty members enjoyed the experience most residents and over half of the faculty felt that faculty mentors lacked the motivation to supervise resident scholarly projects. Second, residents and faculty mentors felt that the key barriers to completing a scholarly project were dedicated time and competing clinical demands. Disparate perceptions of barriers centred on the degree of previous and on-going training for scholarly projects, which the faculty mentors felt was lacking but the residents did not. The most useful of the initiatives for supporting paediatric research projects included the addition of dedicated research coordinator with protected time, the creation of works in progress sessions, and a specific research curriculum delivered in conjunction with the timing of their scholarly project. Finally, these initiatives however did not enhance publication rates in subsequent years.
Surveys of residents in the USA and Canada consistently find that lack of protected time and interest in scholarly activity are the primary barriers to completing a scholarly project (4–7,9). The data presented here support this concept and extend it in a few ways. For the first time in Canada, we confirm that both residents and faculty mentors in a paediatric resident training program collectively find that lack of time and interest impede resident scholarly project completion. Additionally, despite strong financial and infrastructural support from the department and the affiliated research institute, both resident and faculty motivation for supporting scholarly projects remained low. Additionally, we found that while residents feel they have sufficient training to complete a scholarly project, faculty mentors felt the opposite, that most residents were inadequately trained in research. A second disparity in perceived barriers was the role of training, where residents were happy with the scholarly project curriculum provided, but faculty mentors felt it was inadequate. Collectively, these observations could guide paediatric resident scholarly activity programs, by providing more protected time for scholarly activity training, and engaging the faculty mentors actively supervising residents to guide training sessions and help develop relevant curriculum. Several programs in Canada are developing longitudinal blocks and providing 1–2 selective months to address these barriers. These initiatives have enhanced resident scholarly project output in the form of increased abstracts and publications (4,6). Formal evaluations of these emerging Canadian models are needed to determine perceived value and impact on academic output.
Several studies have examined the impact of changes to resident scholarly project training on the research productivity in the form of scientific publication of manuscripts (4,9–13). Some of these initiatives (15) led to increased resident scholarly productivity in the form of manuscript publication rates from 0.03 to 0.67 publications per resident. Others revealed increased participation at the local resident research day (13). More recently, Pound and colleagues (9) at the Children’s Hospital of Eastern Ontario described the implementation of an innovative Scholarly Activity Guidance and Evaluation (SAGE) program consisting of a research curriculum and resources, research rounds, a research committee, structured timelines and milestones, and on-going resident assessments and evaluation. The preliminary results from this program found that all residents over the past 2 years met the requirement of completing a scholarly project by presenting their work at a conference or submitting a manuscript to a peer-reviewed journal. They also found that the SAGE program led to a twofold increase in the number of residents presenting their scholarly project results at their annual resident research day. The program initiatives described here did not yield similar results. Despite increased support, curriculum and dedicated staff supporting residents, the rates of publications in manuscripts indexed in major scientific repositories did not change in the 4 years following implementation. It is possible that the impact of these changes may take years to observe or could be observed if conference abstract presentations were included as an outcome. Large scale tracking of productivity and centre–centre comparisons may provide insight into optimal models of scholar support for paediatric residents in Canada.
While the current study expands on previous work related to resident scholarly activity, there are several limitations that need to be addressed. The results presented here are restricted to a single centre and a relatively small cohort of residents, therefore, these observations cannot be generalized to all programs in Canada. Furthermore, while the addition of perspectives from faculty mentors was a novel addition, they had individually mentored a small number of residents. Therefore, their perspectives may be different from faculty mentors with more experience supervising residents. Additionally, regarding the evaluation of program elements to support scholarly activity, follow-up data were only available for 4 years following the implementation of the program. It is possible that the effects of program changes may take longer to detect. Additionally, while we asked mentors about the number of residents they supervised and their tenure on faculty, we did not specifically address their experience supervising residents prior to and immediately following program changes. This may have provided greater insight into the perceived value of the program initiatives. Finally, we assessed academic output as published manuscripts available in major search engines. It is possible that changes other forms of scholarly output such as conference presentations may have occurred that were not captured by our search.
CONCLUSIONS
The results from this survey reveal that barriers to resident scholarly project completion is related to low motivation from faculty members for supervising resident scholarly projects, lack of time and competing clinical responsibilities for residents. Additionally, resident and faculty mentors share different perceptions of resident capacities and training to complete a scholarly project. Residents and faculty mentors believe that a dedicated faculty member and structured curriculum are facilitate the completion of a scholarly project. Finally, despite changes in support, tracking and structured curriculum, paediatric resident scholarly activity in the form of published manuscripts was not enhanced. More robust examinations of paediatric resident scholarly activity and programs across Canada are warranted to understand the factors that support completion of scholarly projects.
Acknowledgements
The authors thank Dr Celia Rodd and Dr Atul Sharma for their participation in the technical editing of the manuscript. The study was funded through the Robert Wallace Cameron Chair in evidence-based medicine and a CIHR Applied Public Health Chair, both awarded to Dr McGavock.
Appendix A
Resident and mentor survey.
References
- 1. Frank J. The CanMEDS 2005 Physician Competency Framework. Better Standards. Better Physicians. Better Care. Ottawa: The Royal College of Physicians and Surgeons of Canada, 2005. [Google Scholar]
- 2. Frank JR, Snell L, Sherbino J, eds. CanMEDS 2015 Physician Competency Framework. Ottawa: Royal College of Physicians and Surgeons of Canada, 2015. [Google Scholar]
- 3. Richardson D OA, Chan M-K, Lang ES, Harvey BJ. Scholar. In: Frank JR SL, Sherbino J, eds. The CanMEDS 2015 Physician Competency Framework, Ottawa: The Royal College of Physicians and Surgeons of Canada, 2015. [Google Scholar]
- 4. Vinci RJ, Bauchner H, Finkelstein J, Newby PK, Muret-Wagstaff S, Lovejoy FH Jr. Research during pediatric residency training: Outcome of a senior resident block rotation. Pediatrics 2009;124(4):1126–34. [DOI] [PubMed] [Google Scholar]
- 5. Ullrich N, Botelho CA, Hibberd P, Bernstein HH. Research during pediatric residency: Predictors and resident-determined influences. Acad Med 2003;78(12):1253–8. [DOI] [PubMed] [Google Scholar]
- 6. Roth DE, Chan MK, Vohra S. Initial successes and challenges in the development of a pediatric resident research curriculum. J Pediatr 2006;149(2):149–50. [DOI] [PubMed] [Google Scholar]
- 7. Clancy AA, Posner G. Attitudes toward research during residency: A survey of Canadian residents in obstetrics and gynecology. J Surg Educ 2015;72(5):836–43. [DOI] [PubMed] [Google Scholar]
- 8. Gill S, Levin A, Djurdjev O, Yoshida EM. Obstacles to residents’ conducting research and predictors of publication. Acad Med 2001;76(5):477. [DOI] [PubMed] [Google Scholar]
- 9. Ahmad S, De Oliveira GS Jr, McCarthy RJ. Status of anesthesiology resident research education in the United States: Structured education programs increase resident research productivity. Anesth Analg 2013;116(1):205–10. [DOI] [PubMed] [Google Scholar]
- 10. Kichler K, Kozol R, Buicko J, Lesnikoski B, Tamariz L, Palacio A. A structured step-by-step program to increase scholarly activity. J Surg Educ 2014;71(6):e19–21. [DOI] [PubMed] [Google Scholar]
- 11. Lennon RP, Oberhofer AL, McNair V, Keck JW. Curriculum changes to increase research in a family medicine residency program. Fam Med 2014;46(4):294–8. [PubMed] [Google Scholar]
- 12. Chang CW, Mills JC. Effects of a reward system on resident research productivity. JAMA Otolaryngol Head Neck Surg 2013;139(12):1285–90. [DOI] [PubMed] [Google Scholar]
- 13. Pound CM, Moreau KA, Ward N, Eady K, Writer H. Enhancing pediatric residents’ scholar role: The development of a scholarly activity guidance and evaluation program. Med Educ Online 2015;20:27452. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Silcox LC, Ashbury TL, VanDenKerkhof EG, Milne B. Residents’ and program directors’ attitudes toward research during anesthesiology training: A Canadian perspective. Anesth Analg 2006;102(3):859–64. [DOI] [PubMed] [Google Scholar]
- 15. Rothberg MB, Kleppel R, Friderici JL, Hinchey K. Implementing a resident research program to overcome barriers to resident research. Acad Med 2014;89(8):1133–9. [DOI] [PubMed] [Google Scholar]