Critical Care PGY-2 Graduate Perceptions and Practices Regarding Residency Project Publication

Abstract

Objective. To characterize the practices and perceptions of recent post-graduate year 2 (PGY2) critical care pharmacy (CCP) residents surrounding the completion and publication of their primary research project.

Methods. Potential factors and perceptions influencing primary research project publication success were identified and incorporated in a validated electronic survey distributed to 2011 and 2012 PGY2 CCP residency program graduates.

Results. Among the 94/124 (76%) respondents, 26% had published their research project (67% were first authors; 50% were successful on first submission), while 36% still planned to pursue publication, and 38% had no plans for their manuscript. Factors more commonly reported by publishing (vs. non-publishing) PGY2 graduates included: publication of their PGY1 research project, any publication during the PGY2 year, and national presentation of the PGY2 research project. Perceptions associated with research project publication success were a higher degree of self-motivation to publish, post-PGY2 mentor support, project’s publication deemed important to post-PGY2 employer, adequate training for manuscript submission, and adequate time during residency for manuscript completion. Two factors, ≥1 publication during PGY2 [odds ratio (OR)=3.7; 95% confidence interval (CI) 1.3, 10.2] and research project presentation at a national conference (OR=4.5; 95% CI 1.2, 16.9); and two perceptions, self-motivation to publish (OR=8.1; 95% CI 1.7, 37.7) and post-PGY2 mentor support (OR=3.3; 95% CI 1.1, 9.8) were independently associated with publication success.

Conclusion. Only one-quarter of PGY2 CCP residents published their primary research project. PGY2 programs should consider those factors associated with research project publication success when formulating strategies to increase resident publication rates.

INTRODUCTION

Pharmacists are an essential part of the multi-disciplinary, critical care team. The efforts critical care pharmacists devote to research activities play a vital role in advancing the profession and improving patient care.¹ The Society of Critical Care Medicine (SCCM) and the American College of Clinical Pharmacy have listed involvement in research-related activities as an important part of pharmacists' scope of practice.² Participation in research and scholarly activities is an essential component of all post-graduate pharmacy residency-training programs.³ Research activities strengthen residents’ time management skills, enhance written and verbal communication aptitude, and boost problem-solving abilities. The American Society of Health-System Pharmacists (ASHP) lists conducting a critical care practice research project as a key component of residency learning goals and objectives. The residency learning goals and objectives (applicable at the time of training for surveyed residents) specify that PGY2 residents should be able to “successfully employ an accepted manuscript style to prepare a final report” of their primary (practice-based) research project. The updated standards highlight that an “accepted manuscript style suitable for publication in the professional literature” as a requirement for primary research project.⁴ Publication of this project in the peer-reviewed literature, however, is not a stipulation for graduation from any PGY2 programs.^3,4

During the PGY2 year, residents are expected to build on their PGY1 scholarly experiences by conducting a research project that is both more rigorous and substantive. Among PGY1 pharmacy resident graduates, primary research project publication rates are less than 10%; among PGY2 graduates, they are unknown.^5-9 Barriers to pharmacy resident primary research project publication remain unclear. Pharmacy residency program directors (RPD) identify the successful completion of the resident research project as one of the most challenging aspect of the training they provide.^10,11

Publication of the primary research project is potentially an objective measure of the quality of any residency program’s research activities and has also been shown to be a predictor for future scholarly success among graduates.¹² A better understanding of the barriers surrounding publication of the research project among PGY2 graduates is critical for identifying strategies that will enable PGY2 residents to publish their project after graduation. This study sought to measure the success of recent PGY2 critical care pharmacy graduates in publishing their research project and to characterize their practices and perceptions associated with its publication. Understanding barriers and perceptions associated with publication may be beneficial for other PGY2 programs that involve direct patient-care, specifically ones aiming to improve their publication rates.

METHODS

This study was an IRB-approved cross-sectional survey of 2011 and 2012 graduates from the American Society of Health-System Pharmacists’ (ASHP)-accredited PGY2 critical care pharmacy residency programs in the United States. The authors of this study deemed that residents might require up to two years after graduation to publish their research project in a peer-reviewed journal and thus, PGY2 graduates from 2013 and onward were excluded from this study.¹³

The survey instrument was developed through a deliberate, stepwise process that included item generation and construction and then pilot testing and clarification (survey tool available upon request). Survey components, initially identified through a literature review and two investigator conference calls, included questions on residency program and research mentor demographics, prior research training and publications among both residents and preceptors, and the perceptions of the PGY2 critical care residents regarding their experiences conducing their PGY2 research project. Perceptions were evaluated using a 4-point Likert scale (1=strongly disagree; 4=strongly agree). Pharmacy faculty members (n=8), directors of non-critical care PGY2 residency programs (n=4), pharmacy students (n=8) and recent non-critical care PGY2 pharmacy residency graduates (n=6), who were not directly involved with the study, reviewed the survey tool for content validity and question clarity. The survey was revised based on all written and verbal feedback received.

Extensive efforts were made to obtain the names and current email addresses of all 2011 and 2012 PGY2 critical care pharmacy graduates and included communication with current (or prior) RPDs (as listed in the ASHP residency directory), current residents, and other pharmacists affiliated with the institution. The residency program directors were emailed to obtain contact information for graduating residents. Starting in mid-January 2015, the survey was distributed weekly for 3 weeks directly to the residents (graduating in the year 2011 and 2012). The email included the project background and rationale and an anonymous web link for the survey. Survey participation was voluntary. Residents completed the survey using an online survey software and insight platform (Qualtrics, Provo, UT). Data were aggregated from all participants such that all personal information linking specific responses to participants was removed.

Frequency and descriptive statistics were used to characterize resident graduates’ demographics, research training prior to and during PGY2 training, other publications during PGY2 training, characteristics of the research project (focus, study design, funding, timing of IRB approval, presence of a review committee, structured timeline for completion, conference presentation, and type of coauthors), as well as the mentor’s own publication record. The following characteristics were re-categorized for analysis: age ≥25, any research training before or during the PGY2 program, any publication before or during the PGY2, and the scope of practice for research collaborators (eg, statisticians, non-pharmacist clinicians). Residents were able to select multiple answers for the type of research training they had received during their PGY2 year. This response was re-organized hierarchically [ie, informal (self-study of research references provided) vs formal training (didactic lectures and interactive workshops)]. Responses were compared between residency graduates’ who reported publishing their PGY2 research project and those who had not (at the time they completed the survey) using a Chi-square test or Fisher’s exact test, as appropriate.¹⁴ The study authors hypothesized that many of the factors and perceptions influencing PGY2 project publication success were inter-related. Therefore, logistic regression using a stepwise selection procedure was performed to generate OR and 95% CI to facilitate the prioritization of those factors and perceptions associated with greater publication success. SAS version 9.4 (Cary, NC) was used for all analyses. A p value of ≤.05 was deemed statistically significant.

RESULTS

Ninety-four (76%) of the 124 surveyed PGY2 graduates responded. At the time of the January/February 2015 response window, 24 respondents (26%) reported their research project manuscript was “in press or published” in peer-reviewed literature. Among those who published, 16 (67%) were first authors and 12 (50%) successfully published in the journal to which they first submitted their manuscript. One-third of respondents (n=34, 36%) stated they still planned on working to publish their research project. Among the remaining 36 (38%) not planning to publish, most (n=20, 56%) perceived their research project as not publishable because of low quality and/or low impact.

Resident characteristics comparing those who published their research project and those who did not are summarized in Tables 1 and 2. Residents who published their PGY2 research project were more likely to have also published their PGY1 research project, publish other research manuscripts or non-research manuscripts during their PGY2 year. Residents who were able to publish their research projects were more likely to have presented their project as a poster or platform presentation at a national or international meeting. Among those presented, the Society of Critical Care Medicine’s (SCCM’s) Annual Congress was the leading venue.

Table 1.

Comparison of PGY2 Pharmacy Resident Characteristics Based on Published and Non-published PRP

Table 2.

Comparison of PRP Characteristics Between Published and Non-published Residents

Respondents’ agreement with perceived factors that may have influenced research publication is presented in Figure 1. Perceptions that significantly differed based on research project publication included the importance of publishing the research project in peer-reviewed journal to the post-PGY2 employer, provision of adequate time during the PGY2 year for project completion, resident’s self-motivation to publish his or her research project, perceived influence the research project results had on clinical practice at the resident’s institution, post-PGY2 project mentor support/communication provided to the resident, and provision of adequate training for manuscript writing during the PGY2 year.

Figure 1.

Pharmacy Residents’ Level of Agreement with Perceptions Associated with Publication Success.

Among all resident characteristics examined, the stepwise selection procedure identified any publication during the PGY2 year [OR=3.7; 95% CI 1.3, -10.2] and research project presentation at a national or international conference [OR 4.5; 95% CI 1.2, -16.9] as factors independently associated with publication success. In addition, perceptions about self-motivation to publish [OR 8.1; 95% CI 1.7, -37.7] and post-PGY2 project mentor support [OR 3.3; 95% CI 1.1, -9.8] were identified as the perceptions that were independently associated with publication success among all perceptions examined.

DISCUSSION

This study is the first to evaluate PGY2 graduate factors and perceptions associated with primary research project publication success. It demonstrates that only one-quarter of critical care PGY2 graduates publish their research project in the peer-reviewed literature within 2-3 years after residency completion. It also highlights key elements that RPDs, policymakers, and pharmacy colleges should consider for boosting research project publication success among the PGY2 residents they train. For example, when PGY2 residents are provided with adequate research training and protected time to publish this research, they are significantly more likely to publish their PGY2 research project.

Graduating residents from PGY2 programs often pursue faculty positions; 40% of newly hired faculty report completing a PGY2 program.¹⁵ While the Accreditation Council for Pharmacy Education 2016 standards (specifically 18 and 19) list “research and scholarship activities” as a key element for faculty and staff at pharmacy colleges, it does not specify the research training or experience a faculty member should have to reach this goal.¹⁶ Inadequate research training, including that provided during PGY2 training, may contribute to faculty burnout.¹⁷ A PGY2 graduate resident not confident in conducting scholarly activities may not pursue a faculty position.

The use of formal research-training programs during physician residency training enhances the comfort of residents in conducting research and boosts the rate by which these scholarly efforts are published.^18-20 Pharmacy residents completing a mentored research program that covers topics such as research plan development, data analysis and manuscript development are more likely to publish their project.^21-25 These programs should incorporate both didactic and experiential pedagogical strategies. Results suggest most critical care PGY2 residency programs do not have such formalized training programs in place. National standards for the research training that PGY2 residents should receive are needed. RPDs and health-systems should consider establishing affiliations with pharmacy colleges to develop and administer formalized research training programs for graduates. Pharmacy colleges may be able to develop and administer formal research training programs for PGY2 residents as they currently do for teaching certificate programs.²⁶

While research mentorship, particularly one that involves regular communication and follow up, is an important contributor toward project publication success, many research project mentors have no prior peer-reviewed publication experience.¹¹ Health-systems and pharmacy colleges have a responsibility to provide the training and resources (eg, biostatistical support) that PGY2 preceptors need for them to better fulfil this important research mentorship role. The authors suggest developing high-functioning mentor-protégé relationships that extend well beyond the PGY2 graduation date, the period when the research project is most likely to be submitted for publication. RPDs who mentor research project may need to balance their mentorship commitments between incoming PGY2 residents and their recent graduates who are likely to be still finishing up data analysis and starting the manuscript development process.

It is noteworthy that one-third of PGY2 graduates were still motivated to publish their research project despite nearly three years having elapsed since program completion. If these motivated PGY2 graduates are provided with the skills and mentorship to see their project through to publication, the rate by which they will enjoy publication success will increase. Pragmatic considerations exist during the transition period between PGY2 training and a new job that may hinder publication success. Only 5% of PGY2 graduates were provided with dedicated time to publish their research in their new position. The employers of PGY2 graduates should be encouraged to provide their new staff/faculty with more time for PGY2 project publication given that research is a well-recognized core activity for critical care pharmacists. Moreover, data from this survey and others show that publication of the PGY2 research project is associated with a greater likelihood to engage in future research activities.^12,27,28

An important disconnect exists between PGY2 graduates motivated to publish their research project but who feel that it lacks the degree of innovation and/or methodological rigor to be published. When the topic of a research project is a priority to the health-system, but not necessarily novel, efforts should still be made so that the design of the project may still be novel enough to meet the interest of journal editors. Project feasibility is a critical consideration with any PGY2 research project; data collection should be finalized before the PGY2 year ends. A flipped model for residency research projects may be one solution, in which residents complete data collection and analysis for an ongoing project during the first 6 months of the residency year and then develop and obtain IRB approval for a new research project during the second half of their residency year.²⁹

A successful research culture for PGY2 residents that incorporates scholarly training surrounding research and writing and makes project publication an important metric for PGY2 program success is critical. Residents working in a strong research culture may be more likely to persist and overcome the many obstacles to publication that often exist.³⁰ Research exchanges and journal clubs that stress research methods, analysis and presentation, experience with peer evaluation, regular writing exercises (eg, newsletters), clinical faculty visits, and the provision of regular feedback from a residency research committee are important strategies to jumpstart such a culture.³¹ Residents should be provided with dedicated time to explore non-research publications (eg, case reports, review articles) and to prepare and publish their PGY1 research project with the guidance of experienced researchers. Presentation of the project at a national conference (specific to sub-specialty) should be encouraged, as it will serve as an important impetus for the resident to analyze data, organize the presentation of results, and consider the messaging that should be promulgated in the discussion. The updated residency learning objectives emphasize presenting the residency research project at a regional or national conference.⁴ Moreover, residents will receive valuable feedback during poster and platform sessions on how their results can be better displayed and/or interpreted and important limitations that should be considered.

This study has some important limitations. It only evaluated critical care PGY2 residents and only considered graduates from two years (2011 and 2012). Barriers to research project publication may vary between critical care and non-critical care PGY2 residents. Results may not be as applicable for residencies that include less direct patient care activities such as: informatics, administrative and executive residency training programs. Challenges to publication may be different among more recent PGY2 graduates. While the respondent cohort was large and the response rate high, differences may exist between published and non-published residents that were too small to detect. Factors and barriers to research project publication may be different among residents who did not respond. The publication rate may be lower than the reported 26%, as the authors did not verify actual publication of research projects and non-published residents may have been less motivated to complete the survey. Respondents may not have remembered all factors that occurred in their PGY2 given that close to three years had elapsed since graduation. The wide confidence intervals pertaining to specific perceptions such as self-motivation associated with publication limits the applicability of the study findings. While definable steps were taken to improve survey instrument quality, formal survey validation methods were not used. Future research would benefit from these methods. There is a potential for reporting socially desirable responses pertaining to perceptions associated with publication. There may be important perceptions, barriers, and factors that influence project publication success that were not included in the survey.

CONCLUSION

Only a quarter of the responding PGY2 critical care residents published their primary research project nearly three years after graduation. Several important factors and resident beliefs influence the likelihood that a PGY2 resident will publish his or her research project. Programs aiming to increase their publication rates should consider establishing a culture of scholarship during the residency training year, support scholarly pursuits during the residency year (including submission of PGY1 research project), encourage productive mentor-protégé relationships after the completion of residency training, and setting clear expectations for presenting project results at national conferences and publishing them.