Outcomes of Student-Driven, Faculty-Mentored Research and Impact on Postgraduate Training and Career Selection

Abstract

Objective. To evaluate scholarly deliverables from student-driven research and explore the impact on postgraduate training placement rates, pharmacy faculty appointments and lifetime publications.

Methods. A retrospective analysis of Doctor of Pharmacy graduates who conducted student research between the academic years of 2002 and 2015 was performed. Data were collected on research participation, abstracts, presentations, postgraduate training, full-time faculty appointments, and publications.

Results. Of 1229 graduates, 300 participated in research during pharmacy school. Fifty-six percent (n=167) submitted at least one abstract and 68 students (23%) published their research. Research participation was associated with a significantly higher likelihood of postgraduate training and specialty training. Research participation positively affected the likelihood of faculty appointment and lifetime publication rate.

Conclusion. Students who engaged in elective research had significant scholarly deliverables, including peer-reviewed publications, and were more likely to successfully match in a postgraduate position and achieve full-time academic appointments.

INTRODUCTION

Pharmacy organizations strongly support the value of scholarship in the profession and emphasize the importance of research training in Doctor of Pharmacy curricula and postgraduate residency programs.^1-4 The 2016 Accreditation Council for Pharmacy Education (ACPE) standards state that research design should be incorporated into the core curriculum and all graduates should be able to evaluate and incorporate evidence-based practice.³ Participation in research strengthens communication, collaboration, and critical thinking abilities and aids in acquiring new skills such as protocol development, data collection and analysis and scientific writing.^2,5 Many of these skills are highly valued in the job market, including among residency and fellowship program directors.^6-8

Larger class sizes and growing demands among students and faculty members may prove challenging to the incorporation of research and scholarly activities into curricula. According to a 2007 survey, 25% of 79 responding schools required a research project with an additional 57% offering elective opportunities; however, only 15% of colleges included student participation in data collection, analysis, and write-up.⁹ Few studies have examined scholarly output among pharmacy students engaged in research programs. In a 2016 systematic review, only 12 research programs were identified that reported outcomes of student involvement in research programs.¹⁰ Only three of the programs were elective in nature, each of which was small in number (7 to 25 students) and reported very limited scholarly output.¹⁰ A follow-up report from one of those three elective programs demonstrated limited scholarly outcomes among approximately 145 students over six years enrolled in at least one semester of elective research in a non-research intensive institution.¹¹

Student involvement in research has been used to enhance interest in academic careers, increase enthusiasm in postgraduate training, and improve confidence in the scholarly process.^10,12,13 To date, very few programs have demonstrated scholarly outcomes of student research and impact on career direction.¹⁰ This study seeks to describe and quantify the scholarly output of elective, faculty-mentored, student-driven research and its impact on career pursuits of graduates over a 12-year study period at a single college of pharmacy.

METHODS

This was a retrospective cohort study that received exempt approval from the governing institutional review board. The study included all graduates of the Doctor of Pharmacy program from the College of Pharmacy at the University of South Carolina between the academic years of 2002 and 2015 (n=1229). Of note, the college underwent a significant change in curriculum and structure following the 2009 graduating class. Changes in the curriculum relevant to this study included a sequential delivery of integrated material beginning in the second professional year, the addition of formal pre-residency track, and the organization of formal faculty mentoring program for students.

At the study institution, students who engage in a research experience serve as co-principal investigators or co-investigators on faculty-mentored projects in order to gain a full research experience (eg, protocol development, institutional review board approval process, data collection and scholarly deliverables). The topic of these projects may be chosen by either the student or faculty member. These projects are led by the participating student with the faculty member serving as an advisor. The ideal outcome of these projects is presentation at local, state, or national professional meetings and subsequent peer-reviewed publication. The institution offers course credit for completing student research electives (up to 2 credit hours per semester, maximum of two semesters), but each year, students may opt to pursue research as a co-curricular, non-elective opportunity without course credit.

The objectives of this study were to determine the scholarly outcomes of student-driven, faculty-mentored research projects including successful abstract submissions, platform presentations, research grants, and peer-reviewed publications and to evaluate postgraduate training rates and academic positions among graduates. The secondary objectives were to assess the impact of student-driven research studies, abstracts, receipt of grants, and student publications on successful placement in postgraduate residency training programs, pursuit of academic careers, and postgraduate publication rates.

Students were divided into two groups: those who were participating in student-driven research (n=300) and those who were not (n=929). Participating students were identified by two methods: registrar records of all pharmacy students enrolled in research electives, and electronic survey responses from faculty members regarding students who participated in research and verified through assessment of faculty curriculum vitae (CV). Faculty members who were retired or no longer employed were also contacted. Forty-two of the 44 faculty research mentors (95%) completed the requested survey.

To outline outcomes for this study, completion of the research was defined as successful submission of an abstract resulting in a poster presentation. Identification of student participants and collection of research outcomes data were facilitated through college of pharmacy, university, faculty data sources, and public records. Online databases including affiliated journals and conference proceedings from the American Association of Colleges of Pharmacy (AACP), American Society of Health-System Pharmacists (ASHP), American College of Clinical Pharmacy (ACCP), American Pharmacists Association (APhA), and additional state and local, university-based research forums were also queried. Data from these sources included abstract title, mentor, abstract category, when and where the project was presented, any awarded honors such as a platform presentation, if the project received grant funding, if the project was submitted for publication, and if the project was published in a peer-reviewed journal. Data were collected and stored in a de-identified, password-protected Excel (Microsoft, Redmond, WA) spreadsheet. An abstract was considered unique only if it included new methodology or results from a unique data set not presented elsewhere. Abstracts presented in encore fashion at a national and local or regional conference were not counted as unique, but included in total abstract numbers.

Participation in a postgraduate training program, defined as a PGY1 or PGY2 residency or fellowship, was evaluated. Internal records of postgraduate training pursued by graduates have been maintained during the study period by the pre-residency track coordinator. When available, national match service records were used to confirm results. The year, location, and specialty training area, if applicable, were collected. Data on graduates between the years of 2002 and 2014 who entered careers in pharmacy academia were collected from online databases and the AACP faculty registry. Rates of graduates after 2014 entering academia were not collected due to anticipated placement of these graduates in academic positions following the study period, and because of prolonged training and specialization often required for such positions.

Publications were defined as “student publications” or “lifetime publications” if they were published pre- or post-graduation, respectively. To maintain reliability in the data retrieval and reporting, this part of the study was conducted by two data collectors who used a standardized search strategy of PubMed and Google Scholar databases to determine if included graduates were listed as an author on a peer-reviewed publication.⁶ Publications were also identified using faculty CVs. In PubMed, investigators used the following search strategy to locate publications: using the advanced search function, the last name of the graduate with and without a first initial was searched using the “author” field and a custom range for a publication date of five years prior to the student’s graduation year to present. If this search resulted in more than 150 results, three keywords from their student abstract title were grouped together and added to the previous search strategy.

Google Scholar was explored using the advanced search function by entering the last name of the graduate with or without their first initial in the “return articles authored by” field. The “return articles dated between” field included five years prior to the student's graduation year to present. If this search resulted in more than 150 results, three keywords from their student abstract title were grouped together and added to the previous search strategy in the “find articles with all of the words” field. The “where my words occur” field was changed to “in the title of the article.” For all databases, publications were counted when a graduate was listed as an author on the publication.

Descriptive statistics were used for the primary objectives, whereas statistical analyses for secondary objectives were conducted using the SAS for Windows software version 9.4 (SAS Institute Inc., Cary, NC) Statistical Analysis System software. Comparison of 2002-2008 and 2009-2015 periods is displayed for several outcomes due to the significant change in college structure and curriculum in 2008-2009. Secondary analysis was conducted to evaluate the relationships between student research and a variety of endpoints related to postgraduate training and academic career placement. Logistic regression analysis was used to assess for statistically significant associations between a variety of independent variables and the dependent variables of student publication, postgraduate training, specialty training, academic appointment, and lifetime publication. To assess for multicollinearity within the logistic regression model, a correlation matrix was produced that included every binary endpoint. A conservative correlation coefficient of greater than 0.7 was used to identify variables with potential for collinearity within the model. Due to the correlation of several factors, including participating in research and being enrolled to receive course credit; abstract submission and presenting an abstract at a national meeting; and student publications and lifetime publications, the only variables included in the final multiple logistic regression model were participating in student-driven research, abstract submission, and lifetime publication. Three variables were excluded from the multiple logistic regression model including enrolling in a student research elective for credit, national/international abstract presentation, and student publication due to their potential for overlap with the other included variables. Alpha was set at .05 for statistical significance.

RESULTS

All 1229 students who graduated between the academic years of 2002 and 2015 were included in the analysis. The average class size was 60 students from 2002 to 2007, increasing to 110 students from 2008 to 2015. Of the 1229 students, 24.4% (n=300) participated in student-driven research and the majority (72.3%, n=217) were enrolled for course credit (Figure 1). There were 167 (55.6%) research participants who had at least one abstract accepted for presentation, and 22.7% (n=68) who had a peer-reviewed publication. Because several students participated in multiple research projects, these 167 students produced 206 unique abstracts and 52 unique publications (as defined above). Many of these abstracts were presented at multiple meetings (eg, presented at a national conference and repeated at a local or regional conference) resulting in 339 total abstracts (includes repeated presentations) presented. Most students (80.8%; n=135) presented at least one abstract at a national or international meeting (Figure 2). Infectious diseases, teaching and learning, and critical care topical areas composed 61% of all abstracts (Table 1). There was no statistical difference in percentage of students with accepted abstracts between those who obtained course credit as compared to those who did not (54.4% vs 59%, p=.52).

Figure 1.

Research Participation by Graduation Year.

Figure 2.

Research Outcomes by Graduation Year.

Table 1.

Topical Areas of Research for Student Abstracts

Student participation in research and associated scholarly deliverables, including abstracts and publications, was significantly greater from 2009 to 2015 compared to 2002 to 2008 (Figure 3). In addition to the results shown, 12 students presented a platform presentation, and 32 students received grant funding for their research project during the study period. Grant funding for research projects was primarily awarded through an internal research funding pool and rarely exceeded $3000.

Figure 3.

Change in Growth Between Two Time Periods.

Secondary analyses were conducted to evaluate the relationships between student research and a variety of endpoints related to postgraduate training and academic career placement. During the study period, 18.1% of graduates (n=223) completed some form of postgraduate training, while 7.5% (n=92) specialized with postgraduate year two pharmacy residency (PGY-2) and/or fellowship training (Figure 4). Of the 223 students who completed any form of postgraduate training, 67.3% (n=150) participated in research as a student. Based on a multivariate regression model, participation in student-driven research and abstract submission significantly increased the odds of postgraduate training (p<.0001) (Table 2). When comparing early and late study periods, 14.7% (n=70) of graduates between 2002 and 2008 completed some form of postgraduate training compared to 20.3% (n=153) between 2009 and 2015.

Figure 4.

Postgraduate training rates, faculty appointments, and publication rates by graduation year.

Table 2.

Results of Logistic Regression Analysis

Of the 92 graduates who pursued PGY2 or fellowship training, 71.7% (n=66), engaged in student-driven research. Rates of specialization increased slightly from 5.9% (n=28) to 8.5% (n=64) between 2002 and 2008 compared to 2009 and 2015, respectively. Specialty training was found to be influenced by participation in student-driven research (p=.0002), abstract submission (p=.0066), and grant funding (p=.032). The most common areas of PGY-2 specialization included infectious diseases (19%), hematology/oncology (18%), critical care (17%), and pediatrics (14%).

From 2002 to 2014, 3% of graduates (n=34) have been appointed to full-time academic positions. Of these 34 graduates, 76.5% (n=26) engaged in student-driven research. Participation in research (p=.049), any postgraduate training (p<.0001), and postgraduate specialty training (p=.0359) all increased the likelihood of a future faculty appointment. Postgraduate training and specialty training each significantly increased the odds of a graduate publishing in their lifetime (p=.0013) (Table 2). Approximately 10% (n=117) of graduates have at least one lifetime (eg, student or postgraduation) peer-reviewed publication, resulting in 417 total publications. The majority of these graduates (85.5%, n=100) participated in student-driven research while enrolled at the college of pharmacy.

DISCUSSION

This study demonstrates the successful outcomes that can be achieved from faculty-mentored, student-driven research and the positive link to the successful attainment of postgraduate training and a future academic career. It represents the largest, longitudinal assessment of student research impact on postgraduate careers. Similar to several other colleges nationwide, research has been integrated as a required part of a track program at the study institution, specifically a pre-residency track designed to provide students a roadmap to postgraduate training.^14,15 Short-term research experiences are offered through various methods, including targeted seminar series, elective courses, and other mentored research opportunities.^9,10 A follow-up evaluation of pharmacy college websites revealed research tracks present in only 14 (10.6%) of the 132 evaluable programs.¹⁶

Motivation to include such research experiences in the pharmacy curriculum is multifactorial. It is not only used to improve critical thinking, analytical and writing skills, but also as a method to increase postgraduate residency success and foster an interest in academic careers.^7,11,12,17 In this study, students completing research were nearly 5 times as likely to obtain a residency training position and almost 4 times as likely to obtain a specialized residency training position when compared to students who did not participate in research. This could be for several reasons. Residency program directors consistently rank candidate characteristics such as time management, critical thinking and self-motivation highly in desired incoming residents.¹⁸ Successful execution of the research process allows one to demonstrate these skills in a tangible manner. While it is plausible that some very motivated students would elect to pursue research, a research skillset is uncommon among incoming professional students and the value gained from such an experience only adds to the strong, baseline character traits in the professional program student. Others have failed to show a significant difference in successful residency matching between those with research experience and those without, although the authors acknowledge the homogeneity of the sample as a limitation to establishing a relationship between the two.¹⁹

Professional networking and relationships built during the research process are also likely to play a pivotal role providing the student with additional mentorship for postgraduate training and a potentially strong reference.^6,20 At this study institution, students participating in research, including those outside of the pre-residency track, are typically highly motivated and mentored by faculty members who have a strong commitment to furthering the profession through research. Projects are also designed to be unique and achievable by ambitious students. These factors culminate in a strong local research culture and effective student/faculty relationships that are largely responsible for the scholarly success of the students, including residency match rates that are higher than the national average.¹⁵ Although not directly analyzed, the focus of student research and faculty areas of training may likely influence a specialized area of training or practice as 35% of our students engaged in infectious diseases-related research and nearly 20% pursuing residency training ultimately specialized in infectious diseases.

Dissemination of scholarly results as a poster or platform presentation following abstract submission or as a peer-reviewed publication is an important factor in the research process. In this study, 339 total abstracts were presented representing 167 students or 56% of all those participating in research studies. Similar to other colleges, these abstracts were accepted at both regional and national meetings, primarily including the ACCP and ASHP Midyear Clinical Meeting, which offer excellent opportunities for student research platforms. Presentation at a meeting provides an additional avenue for networking, professional growth and the opportunity to interact with prospective future residency directors and/or preceptors.⁶ In this study, 23% of students involved in research achieved a peer-reviewed publication of their student research, which accounted for over 8% of the entire graduating class regardless of research from 2009 to 2015. Additionally, as a comparison, the 2002 graduating class did not publish any student research, while the class of 2013 published 44.7% (n=17) of their student projects. The required time and quality of research needed to produce publishable results is a barrier to publication.^2,21,22 To overcome these barriers, Hasegawa and colleagues described the need to intend to publish from the beginning of the project.²³ Over a 10-year period at the University of California at San Francisco’s College of Pharmacy, Assemi and colleagues found, on average, that one in three required research projects resulted in a poster and approximately 13% were published, increasing since 2008.²¹ A similar low yield has been described among pharmacy residents as less than 10% of these projects result in publication.²⁴ The high publication rate in this study exceeds those described by others, especially among elective research offerings, this is primarily attributed to students beginning feasible projects early in the professional program, dedicated mentorship of motivated, primarily non-tenure track practice faculty, and opportunity for students to work on group projects. Despite these benefits, somewhat ironically, the perceived value of research participation remains debatable among pharmacists, including RPDs, residents, and students.^2,6,21,25,26

Fostering a future career with a focus on scholarship, such as academia, is a common goal of engaging students in research.^13,17 In this study, engaging in student research resulted in 3 times the likelihood of a faculty appointment. While the majority of students pursue community pharmacy roles, this study saw 34 graduates over the study period (slightly greater than 3%) accept a position as a full-time faculty member. The majority (77%) of these graduates completed a student-driven research study. A 2017 survey revealed 44% and 23% of 92 colleges surveyed offer some type of a short-term research experience or longitudinal research track, respectively, to enhance interest in academia.¹⁷ Despite these findings, interest in research and/or research-oriented careers has not been consistently affected among students involved in research, especially in non-elective programs. Many residency programs have established teaching certificates or academic preparation programs which likely play a significant role in career direction for postgraduate trainees.^27-29

There are limitations to this study. While the large sample of students improves strength and validity, some endpoints had fewer positive outcomes, such as grant funding, not allowing for impact assessment of all variables. Additionally, the close relation of some endpoints could result in some collinearity or overlapping sample bias that could not be completely removed by the correlation matrix. The length of time included in this study enabled the study investigators to display the natural growth in research and residency training in pharmacy but also posed challenges in appropriately attributing the influence of the growth and correlating the endpoints. Beginning the study in 2002 enabled the study investigators to include a large sample size with consistent faculty records; however, students from the earlier years may not accurately represent students currently enrolled. Despite meticulous record-checking, due to length and retrospective design, the study is potentially weakened by recall bias, errors of omission in faculty records, and changes in name (eg, marriage). Certain postgraduation achievements may also be underreported due to publication lag time and the experience needed to obtain a faculty position. This study was not designed to measure intrinsic motivating factors behind student engagement in research and/or in career ambitions. It is feasible that these inherent student traits, alone or in combination with scholarship experience, may contribute to the study outcomes.

While the results indicated a positive association between research and postgraduate residency training, other variables known to affect successful residency match (eg, GPA, leadership, extracurricular activities) were not examined in this study. Institution-specific factors related to the program and the types of students it recruits must also be taken into consideration when interpreting and applying the results. Additional investigation around this topic could explore the specific demographic and motivating factors, such as desire for residency or academic positions, competitive and ambitious spirit of individual, or level of involvement or interaction with a faculty mentor that influence the success of student research projects. Such research would provide a framework for improving student scholarship. Further analysis on the impact of research will allow pharmacy colleges to reflect on what enables their continued success, and what ingredients yield a positive culture of research within a pharmacy college.

CONCLUSION

Students who engaged in research during pharmacy school produced significant scholarly deliverables as evidenced by abstracts and peer-reviewed publications. Benefits may have been incurred through research participation as evidenced by their propensity for postgraduate training and an academic career path. Participation in student-driven research was found to significantly increase the likelihood of postgraduate training, specialty training and a faculty appointment, while positively affecting rates of lifetime publication. Faculty mentorship of research is likely to play a significant role in fostering the careers of these motivated students beyond pharmacy school.