Student and School-level Predictors of Pharmacy Residency Attainment

Abstract

Objective. To identify student and school level predictors of pharmacy residency attainment.

Methods. Data were collected from the American Association of Colleges of Pharmacy (AACP) and the Carnegie Classification of Institutions of Higher Education. Logistic multilevel modeling was used to examine the effects of select student and school level characteristics on pharmacy residency attainment, as indicated by students on the AACP Graduating Student Survey (GSS) from 2013 to 2015.

Results. The dataset included 24,351 graduating pharmacy students from 101 schools and colleges of pharmacy. Predictors of residency attainment included working in an institutional pharmacy, female gender, student age, school age, and Research I classification. Nonsignificant variables included curriculum type, class size, and institutional control.

Conclusion. Student and pharmacy school characteristics impact the likelihood of pharmacy residency attainment. Further research is needed to understand the mechanisms associated with these effects.

INTRODUCTION

Post-graduate residency training plays a critical role in preparing pharmacists for today’s medication complexity and can help develop pharmacists as leaders, researchers, clinicians, and members of health care teams. An increasing number of residency graduates work in advanced roles, such as primary care extenders, or specialize in areas of medicine, such as emergency medicine, oncology, or infectious diseases. Whereas faculty members recognize residency training as an opportunity to prepare students for evolving pharmacist roles, students may pursue residency to gain experience, improve confidence, and secure a job.¹ The American College of Clinical Pharmacy (ACCP) and the American Society of Health-System (ASHP) released a statement advocating that by 2020, all new graduates who wish to provide direct patient care should complete residency training.²

In 2015, 20% of doctor of pharmacy degree (PharmD) graduates attained a position as a postgraduate year one (PGY1) resident.^3,4 Due to a lack of available residency programs, the process is competitive and more than a third of applicants do not match with a residency program.³ According to research by Morton and colleagues, pharmacy schools varied widely in the overall proportion of students who accepted a residency from 2008 to 2011, with class match rates ranging from 3.9% to 49.4%.⁵ During that same period, the applicant match rate, which represents the percent of applicants from each pharmacy school that accepted residency, ranged from 28.2% to 82.5%.⁵

Though largely unexplored, a handful of studies provide insight into the institutional characteristics associated with variability in residency attainment. Surveys of pharmacy residency directors found pharmacy school reputation as a priority for granting applicants an interview.^6,7 In addition, Morton and colleagues found that publicly controlled institutions were associated with higher rates of residency attainment.⁵ Differences in residency attainment between institutions may be further elucidated by resource dependence theory, which argues that organizations depend on external resources that originate from the organization’s environment.⁸ In the case of pharmacy schools, external resources may originate from a variety of sources, such as an affiliated university, the state legislature, alumni, and grant agencies. A growing body of literature describes resource-based characteristics of US higher education institutions associated with student outcomes, including Carnegie Classification, size, wealth, and institutional control.^8,9

In addition to understanding institutional characteristics associated with residency attainment (ie, between schools), it is critical to examine variability in residency attainment within pharmacy schools. At the student level, the decision to pursue postgraduate residency can be described by human capital theory.¹⁰ This economic perspective assumes that individuals make decisions by weighing the benefits against the costs for all possible alternatives, and then selecting the best alternative.^11,12 Attaining additional training, such as a residency, can signify to the job market that one is capable of performing a specific job, principally improving his or her human capital.^13,14 This supposition is supported by findings from Gohlke and colleagues and Jellinek-Cohen and colleagues that suggested pharmacy residency directors considered student’s work experience as an important factor in evaluating residency applications.^6,7 Additional student characteristics known to be associated with postgraduate attainment include female gender and pharmacy school GPA.¹⁵

Various stakeholders, including students, educators, and residency directors, have a vested interest in understanding residency attainment. Given the importance of residency programs for preparing pharmacists for the complexities of 21^st century health care, it is critical and timely to examine the institutional and student factors associated with this outcome. The purpose of this study was to examine the extent to which these characteristics affected the likelihood that student pharmacists accepted a residency position.

METHODS

This study used logistic multilevel modeling (MLM) (also called hierarchial logistic modeling) to examine the relationship of student (level 1) and school (level 2) characteristics to residency attainment from 2013 to 2015 (Table 1). All data used to build the model were collected from the 2013, 2014, and 2015 American Association of Colleges of Pharmacy (AACP) Graduating Student Surveys (GSS), AACP Degrees Conferred Surveys, AACP Pharmacy School Locator, and the Carnegie Classification of Institutions of Higher Education website.¹⁶ The AACP GSS is one of four curriculum quality surveys developed by AACP and the Accreditation Council for Pharmacy Education (ACPE) and has evidence to support both validity and reliability. Data on class size and year of first graduating class were obtained from AACP’s annual Degrees Conferred Survey. Data on institutional control (public vs. private) were obtained from AACP’s Pharmacy School Locator, a geographic listing with institutional characteristics of all US pharmacy schools. Each school’s Carnegie Classification was obtained from the Carnegie Classification of Institutions of Higher Education’s website. Per AACP policy, consent to access and analyze the AACP GSS data was obtained from the CEO Deans of 101 out of 125 pharmacy schools that had graduating students in 2015 (response rate 80.8%). Data were provided to the research team from AACP in de-identified form so that each student and institution had a unique identifier (eg, School 093) but no student or institution could be identified.

Table 1.

Data Sources for Multilevel Model of Graduating Student Residency Attainment, as Measured on the American Association of Colleges of Pharmacy (AACP) Graduating Student Survey (GSS) from 2013-2015

The dependent variable in this study was residency attainment, which was coded as “yes” for students who indicated on the AACP GSS that they were continuing on to a postgraduate year 1 (PGY1) residency program or a dual pharmacy residency-masters program (n=6,543 total) and “no” for all other respondents. Residency attainment in this study does not account for the number of students who applied for a residency. Variables included as predictors in the model were: gender (female, male); student age (25 and younger, 26-30, 31-35, older than 35); prior work in institutional pharmacy (yes, no); Carnegie Classification (Research I (R1) highest research activity doctoral institution, special focus institution in medicine or health professions, other); institutional control (public, private); pharmacy curriculum (accelerated, traditional); school age (less than 5 years since first graduating class, 5 or more years); and class size (0-100 students, 101-200 students, more than 200 students). Carnegie Classification is commonly used in higher education research as a proxy measure for resource availability, social capital, and educational opportunities.^17,18

In the final dataset, missing data included one student report for age and three schools’ class size for one year. The student with missing age was removed from the dataset. For the three schools with missing class size data, the missing data were replaced with the value corresponding to the other two years of data for that school. There also appeared to be some anomalies in the variable representing curriculum type (ie, accelerated or traditional). This was originally intended to be a school-level variable, yet, at some institutions, students occasionally reported enrollment in a different curriculum type than their peers. Since the research group was blinded to institution, the group could not confirm which schools used an accelerated curriculum and which schools did not. As a result, it was decided to enter the variable into the model as a student-level variable, thereby allowing it to vary within schools.

MLM is an appropriate method for analyzing nested data structures (eg, students nested within schools). When students attend the same school, their experiences are likely to be dependent on each other, violating the assumption of independent observations in traditional regression analysis.^19,20 Other analytic approaches that do not account for nested data can produce misestimated standard errors, incorrect statistical inferences, and biased coefficients.²¹ Therefore, the MLM developed in this study statistically controlled for students nested within pharmacy schools.

Prior to building the model, univariate and bivariate aspects of the data were examined. Pearson chi-square was used to explore expected and observed frequencies of each independent variable with the dependent variable (residency attainment). Relationships between the independent variables were tested using Phi (for 2 x 2 tables) and Cramer’s V (for n x n tables) correlation coefficients. Phi and Cramer’s V are similar to chi-square but account for sample size and are reported on a scale of -1 to 1.²² If the correlation coefficient was above .50, the interaction was considered large, and if it was above .70, the interaction was considered collinear.²³ The residency attainment rate, which represented the percentage of students indicating postgraduate residency plans for each school, was also used to examine variability between schools prior to controlling for any variables.

To build the full MLM, an unconditional model (also called null model) containing no independent variables was used to determine MLM appropriateness. School-level variance (ie, random effect variance) indicated whether the intercept variance varied between schools, which confirmed the appropriateness of MLM for this dataset. A full model that included student- and school-level characteristics was created. All variables were entered into the models as fixed effects. Dummy variables were created for all categorical variables. The model specification for the final model is illustrated by the equations in Appendix 1. Model coefficients were calculated as odds ratio (OR), 95% confidence interval (CI). In general, the OR represents the likelihood of pursuing residency when controlling for all other variables in the model. The intercept for each model represents student likelihood to attain a residency when controlling for all variables in the model. Since all variables entered into the model are categorical, the “0” value for each variable represents the comparator group. This means that “controlling for a variable” can be interpreted as “when the variable is set to the comparator group.” The interclass correlation (ICC) was used to estimate the proportion of variance in residency attainment that lies between schools, according to the equation: ρ = σ²_Between/(σ²_Between + 3.29_Within).²³ The ICC for all models was calculated, however due to the rescaling of level 1 variance in logistic regression models, comparisons across successive models is not recommended. All analyses were conducted in SPSS version 23 (IBM Corp., Armonk, NY). The minimum significance level for all significance tests in this study was p<.05. Results at more stringent significance levels are noted. Continuous data are presented as mean (standard deviation). The study was deemed exempt by the University of North Carolina’s Institutional Review Board.

RESULTS

The study’s overall sample included 24,351 respondents from 101 pharmacy schools that administered the AACP GSS in 2013, 2014, or 2015. Most commonly, students were female (63%), aged 25 and younger (47%), who had not worked in an institutional pharmacy during school (82%) (Table 2). Forty-nine percent of students who worked in an institutional pharmacy during school attained residency, compared to 22% of students who did not work in an institutional pharmacy (p<.001); 30% of all females attained residency compared to 22% of males (p<.001); and 34% of all students from R1 universities attained residency compared with 23% of students from non-R1 universities (p<.001).

Table 2.

Descriptive Statistics for All Participants in the AACP GSS from 2013-2015

Most students reported their schools as a traditional curriculum (89%) with a class size between 101 and 200 students (50%). Most students were at schools that graduated their first class 5 or more years ago (58%), although there was a notable surge of students at pharmacy schools that graduated their first class less than 5 years ago (42%). The median residency attainment rate for all schools was 26.8% (Figure 1). School residency attainment rates ranged from 6.3% to 64.6%, with more than half of the students attaining residency for three institutions (53.0%, 62.8%, and 64.6%).

Figure 1.

Box and Whisker Plot Representing the Variation in Rate of Residency Attainment for All Schools (n=101)^a

All independent variables had a statistically significant chi-square with the dependent variable and thus, were included in the MLM analysis. Relationships between the independent variables were small except for a large interaction (Cramer’s V=.60) between institutional control (private, public) and Carnegie Classification (RI, special focus, other). Although this did not meet established criteria for being collinear, it was determined that the best method was to continue modeling without an interaction effect.²⁴

The intercept for the unconditional MLM was .35 (CI .31-.39, p<.001), which indicates that the odds for a student to attain residency was .35 to 1. This translates to about a 1 in 4 probability of attaining residency and suggests that students were about .35 times more likely to attain residency than not attain residency within the average school. The ICC suggests that 8.2% (CI .06-.11) of the total variability in residency attainment existed between schools. Since the standard deviation of the intercept (or random effect intercept) between schools was statistically significant (p<.001), it can be assumed that the intercept variance varied significantly between schools. This indicates significant variation between schools regarding residency attainment and provides justification for the use of MLM for this data set.

When controlling for student and school characteristics in the full MLM, the odds of attaining residency were 1.49 (CI 1.38-1.61) times higher for females than males (p<.001) (Table 3). Students who worked in institutional pharmacy settings during pharmacy school had 3.34 (CI 3.00-3.72) to 1 odds of attaining a residency over other students when controlling for other variables (p<.001). There was a decreasing trend of attaining residency as the age of the student increased (p<.001). The odds ratio for Carnegie Classification - Other was .66 (CI .52-.84) which signified a large positive effect for students from a school or college affiliated with an R1 university (p=.001) when compared to other institutions that are not R1 or special focus. Lastly, there was a small negative effect on residency attainment for students at a school established less than 5 years ago (p<.030). Statistically non-significant variables included class size, curriculum type, Carnegie Classification Special Focus and institutional control.

Table 3.

Model of Student- and School-level Predictors of Residency Attainment for Students Completing the AACP GSS in 2013, 2014, or 2015 [Comparison group denoted in brackets]

DISCUSSION

Residencies are an increasingly common aspect of professional pharmacy training.³ Schools and colleges of pharmacy vary widely in the proportion of graduating students reporting plans to enter a post-graduate residency program. Beyond this large descriptive difference, this study’s MLM revealed a statistically significant difference both within and between schools. The most differentiating student characteristics associated with residency attainment included institutional pharmacy experience, gender, and student age while the most differentiating school characteristics included Carnegie Classification and school age. While some of these findings support previous research, others provide new insight into the dynamics that may influence the likelihood that a student attains a pharmacy residency.

This study supports previous literature that highlights the relationship between student characteristics and residencies.^6,7 A recent survey of more than 500 graduating pharmacy students found that students who worked in an institutional pharmacy during pharmacy school were more likely to apply to residencies than their peers.¹⁵ The survey results also indicated that female students were more likely to apply to residency and more likely to match to a residency program. These gender and work-related effects persisted in this study, even when controlling for other student and school variables.

In addition, this study adds to a growing body of research concerning the relationship between institutional characteristics and student outcomes in pharmacy education. Morton and colleagues, for example, described more students attaining residency from well-established schools yet failed to find a significant difference.⁵ In contrast, the MLM in this study identified a significant effect for schools established less than five years compared to their peers. The difference in residency attainment based on school age may be a reflection of residency directors who prefer familiar pharmacy schools or a temporary byproduct resulting from fewer graduates in the marketplace representing and advocating for their alma mater.⁶ Also, Morton’s model found a statistically significant difference between public and private schools but this variable was insignificant in the full MLM model when controlling for all other variables.⁵ This may be explained by the addition of several more variables, in particular RI status, which may have moderated the institutional control effect.

The model built in this study extended previously published research by including novel school level characteristics such as curriculum type, class size, and Carnegie Classification. Surprisingly, curriculum length and class size did not predict residency attainment. One might hypothesize that students who enroll in accelerated curricula desire faster access to the job market, but it appears this is no different from four-year curricula when controlling for other variables in the model. Also, previous research has reported mixed results on whether class size matters in higher education, and in pharmacy education, it does not appear to influence residency attainment.²⁶ Although a known predictor of undergraduate student outcomes, a new finding for pharmacy was the relationship between Carnegie Classification and residency attainment. As a proxy measure, Carnegie Classification may have accounted for differences in institutional resources or other aspects of campus that influence student experiences.^8,17,18

As institutions strive to define, promote, and enhance student outcomes, consideration must be given to what success means and how it is measured within the context of rapidly evolving health care and education systems. The results of this study suggest that some institutions are differentially positioned to promote residency attainment, which may prompt discussions about the role and ability of pharmacy schools to prepare students for residency positions. For schools in which this is an outcome of interest, these results may provide a benchmark that highlights how characteristics of the school and experiences of the students relate to residency attainment. Some schools are developing programs and strategies to better prepare student pharmacists to “enter the race for postgraduate training.”²⁷ Ultimately, additional research concerning institutional effects could provide important insight for those striving to position students for success in postgraduate residencies.

This study provides a first step toward understanding residency attainment within a multilevel framework and raises new questions about patterns of postgraduate outcomes in pharmacy education. Clearly, additional research is needed to better understand the role of students in the residency process and the institutional environments that impact student outcomes. Future research, for example, should explore the mechanisms behind the identified work and gender effects and their relationship with residency attainment.¹⁵ Examination of other student characteristics, such as motivating factors and planned behavior, may also be useful.²⁸ At the school level, research should examine institutional characteristics and behaviors that impact student residency placement. Are these effects reflective of the type of students entering those schools, the influence of the school on students applying to residency, or the impact that certain institutional characteristics may have on residency directors’ decision to match candidates? Qualitative research, in particular, could provide rich understanding about the “how” and “why” questions behind these findings. Case studies of outlier schools may also reveal how and why these schools have a markedly different proportion of students attaining residency.

This study has several limitations in. First, compared to linear models, logistic models do not allow reliable comparisons across models. As a result, we were unable to reliably state how much variance the full model accounted for compared to the unconditional model. Second, although the full MLM is the most comprehensive pharmacy residency attainment model to date, it was limited to variables available for analysis. Future research should examine additional characteristics that may explain variability within and between schools, as this study has provided only a glimpse into the types of questions that could be answered if the profession committed to large-scale data collection, sharing, and analysis. Third, this study is limited to residency attainment and does not include data about residency application or residency matching. Lower rates of residency attainment may be due to fewer students from those schools applying to residency programs or could reflect lower matching rates. Use of American Society of Health-System Pharmacists’ residency data could complement this study with information at the point of application. Fourth, the AACP GSS is available for schools to administer from early March until the end of June. Students at schools that open the survey early in the administration window may not have received their match results at the time of survey completion. Lastly, the AACP GSS is a self-report survey, which means that respondents may have misreported information. However, the large sample size and the large proportion of pharmacy schools consenting to this study promote trustworthiness of the findings.

Despite these limitations, using MLM to examine pharmacy education is a new direction for the field and one that clearly advances our understanding of student outcomes in pharmacy education. This study is the first to model residency attainment from a nested data structure and its use is supported by a robust dataset. MLM is an increasingly common statistical approach in numerous fields, including sociology, education, and public health; however, its application in health profession education has been limited to date.^20,21 Creating opportunities to expand our research toolkit to include more sophisticated analytic methods in pharmacy education, such as MLM, will help educational scholars further elucidate critical aspects of educational practice and research.

CONCLUSION

Given the growing number of postgraduate positions in pharmacy, the lack of research on residency attainment is surprising. A range of stakeholders, including students, schools, and residency programs, have a vested interest in better understanding the mechanisms associated with variability within and between schools. This study suggests that a number of student and school characteristics are associated with residency attainment. This study’s investigators hope that this study promotes productive dialogue about residency attainment, aspects of pharmacy education related to this outcome, and additional research that can further explicate the mechanisms underlying these findings.

Appendix 1. Model Specification for Full MLM Examining Residency Attainment

For each individual i in school j, the effect of student predictors (X_(1-q)ij) and school predictors (W_(1-Sq)j) on residency attainment can be expressed as:^a

^a Where η_ij is the predicted log odds and β_qj is the intercept for the jth group. This model uses the logit link function, which is the natural logarithm of the odds that Y = 1 (residency attainment, as denoted by π_ij) versus Y = 0 (no residency attainment, as denoted by 1- π_ij) (Heck, Thomas, Tobata, 2012).