Assessment of a Scholars Program Designed to Enhance Pharmacy Students’ Competitiveness for Postgraduate Residency Training

Abstract

Objective. To assess pharmacy residency match/placement rates and student perceptions of a program designed to enhance Doctor of Pharmacy (PharmD) student competitiveness for postgraduate residency positions.

Methods. The Scholars Program was developed to provide advanced training to select PharmD students who had an interest in postgraduate residency training and was completed during the third and fourth professional years. The program consisted of mentoring; elective coursework encompassing clinical practice, teaching, and leadership; modified experiential education; journal club meetings; teaching assistant duties; conducting research and/or scholarship; and delivering professional presentations. Residency match/placement rates of students who had completed the program were compared to national data and to students in the school who were not enrolled in the program. Perceptions of the program were assessed using an online survey.

Results. Sixty-four students enrolled in and completed the Scholars Program from 2013 to 2019. Of these, 58 (91%) pursued postgraduate residency training. Students enrolled in the program had a higher combined phase 1/phase 2 match rate (91.4% vs 67.4%) than students in other PharmD programs across the United States. Similarly, students enrolled in the Scholars Program had a higher combined phase 1/phase 2 match rate (91.4% vs 62.9%) and overall residency placement rate (96.6% vs 67.0%) compared to students in the school who were not enrolled in the program. More than 85% of students enrolled in the Scholars Program who pursued residency training agreed that the program prepared them for and helped them attain a postgraduate residency.

Conclusion. Pharmacy students enrolled in the Scholars Program experienced high residency match/placement rates and viewed the program as valuable preparation for postgraduate training.

INTRODUCTION

Schools and colleges of pharmacy are expected to prepare students to directly contribute to patient care in collaboration with other health care providers.¹ Nationally, approximately half of graduating students choose to further their education through postgraduate residency training.^2,3 The American Society of Health-System Pharmacists (ASHP) and the American College of Clinical Pharmacy (ACCP) support the position that, as of the year 2020, completion of a postgraduate residency should be required for all new pharmacy graduates who will be providing direct patient care.^4,5 The number of pharmacy students who pursue postgraduate residency training and the number of available residency positions continue to increase each year; however, a considerable gap exists between the number of applicants and the number of available positions.³ This gap has fostered a competitive environment wherein Doctor of Pharmacy (PharmD) students need to be highly qualified and well-prepared before embarking on their search for postgraduate residency training.

The Accreditation Council for Pharmacy Education (ACPE) advises that pharmacy schools dedicate resources to preparing students for their career following graduation.¹ The increasingly competitive nature of postgraduate residency training has led many schools to implement programming to better prepare students for postgraduate training and to provide guidance to them during the residency application process.^6-8 A 2010 survey by Dunn and colleagues reported that 16 schools and colleges (22% of respondents) offer a structured residency preparation program with a clinical practice focus.⁶ The qualities desired in an ideal residency candidate, however, extend beyond clinical experience alone and include knowledge of and experience in scholarship, teaching, and leadership.^9-12 To date, there have been no published reports describing a formalized program that provides students with mentorship and advanced training in the area most valued by residency programs (ie, clinical practice, teaching, research/scholarship, and leadership).

In an attempt to better prepare and qualify students for postgraduate residency training, the University at Buffalo School of Pharmacy and Pharmaceutical Sciences designed and launched a Scholars Program in 2010. Students attend required all-school conferences in their first and second professional years (P1 and P2) of the PharmD program to learn more about postgraduate residency training and how to prepare themselves to be competitive residency candidates. The Scholars Program is presented to students as one option for gaining the experiences and developing the qualities desirable in a residency candidate. The Scholars Program pairs students with a faculty mentor and provides access to an intensified didactic and experiential curriculum that includes opportunities to develop their clinical, teaching, research/scholarship, and leadership skills (Table 1). Students with an interest in postgraduate residency training apply to the program during the fall semester of their P2 year. The recommended GPA for application to the program is 3.5 or higher; however, students with a GPA less than 3.5 are allowed to apply but must include a letter of support from their faculty advisor. Applications are reviewed by a committee composed of five pharmacy practice faculty members and one staff member. The number of applications received each year has ranged from 10-13 in the first three years of the program to 20-24 in the last four years. Evaluation criteria include GPA, pharmacy work experience, professional organization involvement and leadership, postgraduate training plans, and professional goals. Student applications are evaluated using a rubric, and 10-20 students are invited to a brief (10-15 minute) interview with the committee. Each year, 10-12 students out of a class size of approximately 120 are accepted into the program. Accepted P2 students participate in the process of identifying a faculty mentor by speaking with potential mentors and submitting a ranked list of their top three choices. Potential mentors are contacted by the chair of the Scholars Program committee, with consideration given to students’ clinical interests and to the number of students each faculty member is able to mentor. The mentor/protégé assignment is then formalized by the committee during the spring semester of the students’ P2 year.

Table 1.

Components of the Scholars Program at the University of Buffalo School of Pharmacy and Pharmaceutical Sciences

The primary objective of this study was to assess postgraduate pharmacy residency match rates among students enrolled in the program compared to that of students across the United States who participated in the ASHP Resident Matching Program. The secondary objectives of this study were to assess postgraduate pharmacy residency match and placement rates among students enrolled in the program compared to that for other students at our school who were not enrolled in the program and to assess enrolled students’ perceptions of the program.

METHODS

This study was a retrospective review of match rates (phase 1, phase 2 (2016-2019), and phase 1/phase 2 combined) for accredited (ie, pre-candidate, candidate, or accredited status) postgraduate year one (PGY1) residency training programs by students who were enrolled in the Scholars Program and graduated between 2013 and 2019. Match rates for students in the Scholars Program were compared to national match rates provided by ASHP from National Matching Services during the same timeframe.¹³ Match rates for students in the program were also compared to PGY1 residency match and placement rates (ie, overall residency placement, including the ASHP Match plus the post-Match Scramble) of students at our school who were not enrolled in the program but who did pursue postgraduate residency training during the same timeframe. Additional data collected included whether graduates attained a postgraduate year two (PGY2) pharmacy residency and the publication status of their scholarly project.

To assess student perceptions of the Scholars Program, a non-anonymous 25-item exit survey was administered online to graduating students who had participated in the program. The survey included an evaluation of the program and its individual components, an evaluation of the student’s mentor, and outcomes data (eg, research or scholarly project status). Students assessed the program using a seven-point Likert scale. The Scholars Program Committee reviewed these data annually for quality improvement purposes. A core component of the program was deemed valuable if at least 80% of the students agreed that it was (Likert score of 5 or higher). SurveyMonkey (SurveyMonkey, Inc., Portland, OR) was used to collect survey responses. This research was deemed exempt from review by the university’s institutional review board.

With 58 students from the program entering the match, and assuming a national comparison phase1/phase 2 combined match rate of 65%, a sample size of 158 individuals in the national comparison group was required to achieve 80% power to detect an absolute increase of 20% in phase 1/phase 2 combined match rate among students enrolled in the program. Program demographics, residency match rates, residency placement rates, and survey responses were analyzed using Excel and GraphPad (GraphPad Software, San Diego, CA). Chi-square or Fisher exact tests were used to compare PGY1 residency match rates during phase 1, phase 2 (2016-2019), and phase 1/phase 2 combined within our school and nationally; to compare PGY1 residency placement rates within our school; and to compare the proportion of students in the Scholars Program who attained a PGY2 residency to students in our school who were not enrolled in the program and to national data. The a priori level of significance was less than 0.05.

RESULTS

While admission to the Scholars Program was selective and based on an application and interview process, program students were demographically similar to those enrolled at the University at Buffalo School of Pharmacy and Pharmaceutical Sciences. There were no significant differences in program students versus non-program students with respect to undergraduate degree earned prior to admission into the PharmD program (37.9% vs 42.2%, respectively, p=.66).

Each student in the program successfully completed all requirements as listed in Table 1. Students achieved the research/scholarship aspect of the program by completing retrospective research (68.3%), prospective research (11.1%), survey-based research (9.5%), a review of therapeutics (7.9%), benchtop research (1.6%), or a case-series/case report (1.6%). All students presented their research as either a poster or platform, with 60 (95.2%) students presenting a poster at a state, national, or international professional meeting. Project data was available for all 64 students who completed the program. As of the end of the 2019 spring semester, 19 (29.7%) project manuscripts by Scholars Program students had been published or accepted for publication in a peer-reviewed journal.

Between 2013 and 2019, 58 (90.6%) students enrolled in the program pursued a PGY1 residency and participated in phase 1 of the ASHP Match. During the same timeframe, 233 students at our school who were not enrolled in the program pursued a PGY1 residency. Of these, 224 participated in phase 1 of the ASHP Match (nine students only participated in phase 2 and/or the Scramble). The PGY1 match rate of students at our school, whether or not they were enrolled in the Scholars Program, was similar to that seen nationally (minus our students) in phase 1 (66.0% vs 66.3%, p=.96) and phase 2 (28.3% vs 23.3%, p=.54). Of the six students enrolled in the program who did not pursue a residency, four directly entered community pharmacy practice and two accepted a research fellowship.

Students enrolled in the Scholars Program had a higher PGY1 match rate in phase 1, phase 2, and phase 1/phase 2 combined compared to students who pursued postgraduate residency training nationally (p<.05 for all comparisons) (Table 2). Similarly, students enrolled in the program who participated in the ASHP Match had a higher PGY1 match rate in phase 1, phase 2, and phase 1/phase 2 combined, and had a higher PGY1 residency placement rate than students in the school who were not enrolled in the program (p<.01 for all comparisons) (Table 2). There were no differences observed in either phase 1 (p=.16), phase 2 (p=.16), or phase 1/phase 2 combined (p=.69) PGY1 match rates between students at our school who were not enrolled in the program and students who pursued PGY1 residency training nationally. Of the students in the program who graduated between 2014 and 2018, 29 (67%) pursued and obtained a PGY2 residency following their PGY1 training compared with 45 (42%) students in our school who were not enrolled in the program (p<.005) and 3079 (20%) students nationally (p<.0001).

Table 2.

Postgraduate Residency Match and Placement Rates for Students Enrolled in a Scholars Program

Sixty-two students who completed the Scholars Program responded to the exit survey (96.9% response rate). Overall, students enrolled in the program viewed it as valuable as suggested by 85% agreement or higher on all survey items and a median Likert score of 7 (Table 3). When evaluating only students who pursued residency training, 51 of 57 (89.5%) respondents agreed that the program helped them attain their goal of being accepted into a postgraduate residency program. Furthermore, 53 of 57 (93.0%) agreed that the Scholars Program prepared them well for postgraduate residency training. Although participation in this program entailed a higher workload as compared to the school's standard core curriculum, nearly all students (60 of 62, 96.8%) indicated that the amount of work required to complete the program was "just about right" (median Likert score=4 on a seven-point scale). Of the eight items used to evaluate the individual core components of the program, seven (87.5%) had 85% agreement or higher and a Likert score of 5 or higher, and were therefore deemed valuable (Table 4). More than 90% of students agreed that the research or scholarly project provided them with experience in research and/or medical writing and familiarized them with aspects of the research and/or medical writing process. The only item rated as not being valuable was journal club, which received a median Likert score of 4 on a seven-point scale).

Table 3.

Students’ Perceptions of the Scholars Program (n=62)

Table 4.

Students’ Perceptions of Individual Components of the Scholars Program (n=62)

DISCUSSION

The Scholars Program was designed to increase the competitiveness of a subset of our PharmD graduates in the ASHP Match and to better equip them for postgraduate pharmacy residency training. This program paired students with a mentor, enrolled students in a tailored curriculum that covered valuable facets of postgraduate residency training (including advanced pharmacotherapy, leadership, and teaching), provided opportunities for assisting in the education of pharmacy students, and engaged students in research and/or scholarship. Enrollment was dictated by available resources at the school and was therefore limited to 10-12 students annually. However, students who were not enrolled in the program had the opportunity to engage in most aspects of the program informally, if they chose to do so.

A significantly higher PGY1 match rate in each phase of the Match was observed among students enrolled in the Scholars Program as compared to other PharmD students across the United States and as compared to students in our school who were not enrolled in the program. Furthermore, PGY1 residency placement (ASHP Match plus the post-Match Scramble) was significantly higher among students who completed the program than students in our school that were not enrolled in the program. Only students who obtained a PGY1 residency with ASHP accreditation status of pre-candidate, candidate, or accredited that participated in the ASHP Match were included in this analysis. An analysis of PGY1 residency placement rates between students at our school and national data could not be performed because data from the ASHP Scramble for other PharmD programs were not publicly available. Similarly, while acceptance into another postgraduate training program type (such as a research fellowship) was also considered a favorable program outcome, this was not evaluated because national data were not readily available for comparison.

Attributes desired in a residency candidate include leadership experience, teaching experience, meeting presentation experience, research and publication experience, research potential, and research goals.¹² Students enrolled in the Scholars Program engage in all of these areas. They conduct a scholarly project from start to finish and are required to present the results of their project at a state or national meeting. Students enrolled in the program are required to enroll in an elective course on teaching/learning theory and are provided opportunities to apply these principles through their teaching assistant duties. As teaching assistants in a pharmaceutical care course, students participate in the design of skills-based educational activities such as simulated patient cases for counseling, adherence activities, and simulated patient rounds. They also have the opportunity to teach and provide feedback to junior students one-on-one or in a small group setting. To enhance this experience, the teaching assistantship was increased from one semester to two semesters in 2018. While the Scholars Program cannot guarantee that students will obtain an official leadership position, many students enter the program already having leadership experience as it is a criterion evaluated for admission into the program. During the program, students enroll in a pharmacy leadership course wherein they learn important skills needed during residency training, including but not limited to professional communication, goal setting, prioritization, and time management. They are also encouraged by their mentors to grow and refine their leadership skills through active involvement in school and professional organizations. Strong letters of recommendation and strong letters of intent also appear to be highly valued by program directors.⁹ The mentor-protégé relationship that is built during the two-year program gives students the opportunity to seek a letter of recommendation from a faculty member with whom they have worked closely in both the clinical and research setting and who can therefore speak in detail with good familiarity to the candidate’s attributes as they relate to residency training. All of these factors make for a stronger application packet, helping these students to secure a residency interview.

Students enrolled in the Scholars Program gain experience in a variety of activities that they will engage in during their residency training. They also refine their time management and prioritization skills through the increased workload the program places on them and through management of their longitudinal research project. Career placement data for program students was not available. However, as compared to PGY1 residents who did not complete the program (at the school and nationally), a higher percentage of PGY1 residents who were graduates of the Scholars Program obtained a PGY2 residency. This suggests that students who complete the program are better equipped to excel in residency training, as PGY1 performance can influence a candidate’s ability to successfully match with a PGY2 residency program.¹²

Overall, the Scholars Program was well received by enrolled students. A majority of students agreed that the program helped them attain a postgraduate training position and that the program prepared them for postgraduate training, while also indicating that the added workload was reasonable. To address poor feedback received regarding the journal club exercise, this activity was integrated into the school’s pharmacy resident journal club in 2017, which is attended by approximately eight PGY1 and PGY2 residents and their program directors. In the two years following this change, the percentage of students who agreed that journal club was valuable to their professional development increased approximately 10% and the percentage of students disagreeing with said statement decreased by more than 20%. Thus, journal club remains the least valuable component of the program in the opinion of enrolled students. As of the 2018-2019 academic year, the resident/student journal club is held only once per semester and program students only participate during their third professional year. Two residents present at each meeting, and while program students are required to attend, write up a study summary and critique, and participate in the discussion, the fact that journal club is not held more frequently may diminish its perceived value. The Scholars Program Committee is actively exploring additional avenues to enhance the value of this program component, such as increasing the frequency of journal club and linking the resident/student journal club to those offered by other organizations within and outside of the school.

The need to increase involvement of pharmacy students and residents in scholarly activity has been emphasized by ACCP.¹⁴ However, they have identified several barriers to doing so, including a lack of curricular emphasis on scholarship and research skills, limited availability of experienced research mentors, and time constraints leading to students graduating prior to completion of a scholarly endeavor. By creating a curricular framework that carves out time for students to engage in research or scholarship with an experienced faculty member, the Scholars Program provides a novel framework to overcome these barriers. Furthermore, these benefits are not limited to students. The faculty-student collaboration fosters research/scholarship deliverables for faculty members serving as mentors in the program, most of whom are in the clinical track, balancing clinical, teaching, and service demands with research expectations. The publications resulting from program student projects also advance the research mission of our school. The pairing of motivated, engaged students with faculty members can result in a successful outcome for all parties involved.

The Scholars Program is chaired by a clinical faculty member who, in addition to coordinating the application process and interviews, facilitates the mentor/protégé assignments each year. Administration of the program itself does not require a substantial amount of committee member time: their primary duty is to review applications and interview candidates over a period of several weeks early in the spring semester. While mutually beneficial to both students and faculty mentors, expanding the program to accommodate more students would present logistical challenges. More students could be integrated into the elective coursework and teaching assistant assignments; however, the major factor limiting program expansion is saturation of available mentors. Students enter the program during their P2 year and spend more than two years working with their mentors. They each complete one IPPE and two APPE rotations with their mentor. Therefore, mentors often limit their availability to one or two program students every one or two years.

The major limitation of this study was the potential for selection bias. Enrolled students were required to have demonstrated academic success and to have made initial progress towards involvement and leadership in professional organizations. Therefore, it is inherently difficult to determine the extent to which the Scholars Program impacted residency match and placement rates because the program a priori attracts strong students who were likely to have been more competitive residency candidates at baseline. That said, phase 1, phase 2, and phase 1/phase 2 combined match rates in the cohort of graduates who were not enrolled in the program but who did pursue postgraduate residency training was similar to that of students across the United States, averaging 62%, 19%, and 63% over the seven-year timeframe, respectively. This suggests the higher match/placement rates observed among students enrolled in the program was, at least in part, the result of the experience they gained through enrollment in this program.

A second limitation of this study was that prior to 2015, ASHP did not provide schools with student-specific match data. Our residency match data during that timeframe (2013 and 2014) was collected via an internal survey sent by school faculty members to all students in the graduating class. Unmatched students may not have responded to the survey, which could have inflated our estimate of the school’s match rate during this timeframe. Assuming this occurred, the match rate in the cohort of students not enrolled in the program would have been lower than that reported here, resulting in a larger difference between the group of students enrolled in the Scholars Program and the group of students who were not.

A third limitation was the design of the survey. The survey was designed as a non-validated quality improvement tool, and in order to evaluate the effectiveness of the mentor, could not be anonymous. As a consequence, student perception data may have been inflated and therefore must be interpreted with caution. Our survey results indicate that the majority of students felt the program prepared them well for postgraduate residency training. However, these are perceptions of students surveyed at the end of their P4 year who had not yet begun residency training. To better evaluate the ability of the program to prepare students for residency, future consideration should be given to surveying graduates who completed the Scholars Program and/or their residency program directors at the end of their PGY1 residency.

CONCLUSION

Pharmacy students enrolled in the Scholars Program at the University at Buffalo School of Pharmacy and Pharmaceutical Sciences experienced higher than average residency match/placement rates and viewed the program as valuable preparation for postgraduate training. Other schools and colleges of pharmacy, particularly those with residency match rates below national averages, should consider implementing similar programming as a means to enhance student competitiveness in the ASHP Match and better prepare their students for postgraduate residency training.