Examining the Association Between the NAPLEX, Pre-NAPLEX, and Pre- and Post-admission Factors

Abstract

Objective. To examine the relationship between the NAPLEX and Pre-NAPLEX among pharmacy graduates, as well as determine effects of pre-pharmacy, pharmacy school, and demographic variables on NAPLEX performance.

Methods. A retrospective review of pharmacy graduates’ NAPLEX scores, Pre-NAPLEX scores, demographics, pre-pharmacy academic performance factors, and pharmacy school academic performance factors was performed. Bivariate (eg, ANOVA, independent samples t-test) and correlational analyses were conducted, as was stepwise linear regression to examine the significance of Pre-NAPLEX score and other factors as related to NAPLEX score.

Results. One hundred fifty graduates were included, with the majority being female (60.7%) and white (72%). Mean NAPLEX score was 104.7. Mean Pre-NAPLEX score was 68.6. White students had significantly higher NAPLEX scores compared to Black/African American students. NAPLEX score was correlated to Pre-NAPLEX score, race/ethnicity, PCAT composite and section scores, undergraduate overall and science GPAs, pharmacy GPA, and on-time graduation. The regression model included pharmacy GPA and Pre-NAPLEX score.

Conclusion. The findings provide evidence that, although pharmacy GPA is the most critical determinant, the Pre-NAPLEX score is also a significant predictor of NAPLEX score.

INTRODUCTION

The North American Pharmacist Licensure Examination (NAPLEX) is used to assess pharmacy graduates’ knowledge of pharmacy practice, with a focus on the following three areas: “Identify[ing] practice standards for safe and effective pharmacotherapy and optimiz[ing] therapeutic outcomes; identify[ing] and determin[ing] safe and accurate methods to prepare and dispense medications; and provid[ing] and apply[ing] health care information to promote optimal health care.”¹ The National Association of Boards of Pharmacy (NABP), which administers the NAPLEX, offers the Pre-NAPLEX as a preparatory tool for the NAPLEX. The purpose of the Pre-NAPLEX is to simulate the experience of taking the NAPLEX, allowing students to better understand the NAPLEX testing process.² The NABP further explains that the Pre-NAPLEX is, “intended to provide you [the pharmacy student] with information on your performance in answering a subset of test questions similar to those you will encounter on the NAPLEX under pre-testing conditions.”² A beneficial aspect of the Pre-NAPLEX is that it incorporates questions that were previously included on the NAPLEX.² The Pre-NAPLEX also may be useful to colleges and schools of pharmacy in identifying students who need more rigorous preparation for the NAPLEX prior to graduation. Yet, little published research exists about the relationship between the Pre-NAPLEX and NAPLEX.

Possible determinants of NAPLEX performance have been explored in prior studies but have largely focused on pre-pharmacy and pharmacy school-related factors.^3-6 Among various pre-pharmacy variables, McCall and colleagues found that a higher composite Pharmacy College Admission Test (PCAT) score, higher pre-pharmacy GPA, and younger age were predictors of better NAPLEX score.⁴ Shaya and colleagues, in a study of pre-pharmacy and pharmacy school factors that may predict NAPLEX performance, found that cumulative pharmacy school GPA and PCAT section scores (specifically, chemistry, reading comprehension, and verbal ability) were associated with higher NAPLEX scores.⁶ In a similar study, Allen and Diaz noted that NAPLEX score was correlated with undergraduate GPA, undergraduate math-science GPA, cumulative pharmacy school GPA, on-time graduation, and the number of unsatisfactory grades in pre-pharmacy and PharmD courses.³ They also found that higher pharmacy school GPA and no unsatisfactory pre-pharmacy grades were better predictors of NAPLEX performance in regression models.³

None of the aforementioned studies addressed the possible effect of the Pre-NAPLEX on NAPLEX scores. Given its potential value in assisting colleges of pharmacy in targeting low-performing students for intervention prior to NAPLEX testing, a more thorough understanding is needed of the relationship between the NAPLEX and Pre-NAPLEX. Therefore, the purpose of the study was to address this gap in the literature and examine the relationship between NAPLEX and Pre-NAPLEX scores among pharmacy graduates, as well as determine the effect of pre-pharmacy, pharmacy school, and demographic factors on NAPLEX performance.

METHODS

A retrospective review of records of Class of 2015 graduates at the University of Tennessee College of Pharmacy (UTCOP) was performed, and all Class of 2015 graduates were eligible for inclusion. Prior to graduation, students were asked to sign a release allowing NABP to provide UTCOP with individual NAPLEX scores. Those students who returned signed release forms consented to participate in the study. The study was approved by the University of Tennessee Health Science Center Institutional Review Board.

The NAPLEX is a 185-item, computer-based exam that uses adaptive technology to select an individual test-taker’s questions.¹ Scores on the NAPLEX range from 0 to 150, with higher scores indicating better performance; 75 is considered the minimum passing score.^7-9 The majority of the Class of 2015 took the NAPLEX in the summer of 2015, and scores were reported to UTCOP in late January 2016. NAPLEX total scaled scores were used in the analysis.

The Class of 2015 took the Pre-NAPLEX near the end of the fall semester of their third year of pharmacy school (late November 2013), as that was the end of their didactic coursework. All students were required to take the Pre-NAPLEX. The college purchased vouchers from NABP that allowed students to each pay $45 for the $50 exam; the price has since increased to $65. Students were allowed to take the Pre-NAPLEX on a personal computer in the setting of their choosing by the college’s set deadline (for students who preferred to take the exam on campus, a 3-hour time slot was reserved in the college’s computer labs). As this was not considered a formal or official exam, no proctors were used to monitor administration. Scores were collected from students by the college’s assessment coordinator. At the time of the Class of 2015’s testing, the web-based Pre-NAPLEX was a 50-item, 70-minute exam. Since March 2014, it has been revised to include 100 items, with a 140-minute testing period.^2,10,11 Similar to the NAPLEX, Pre-NAPLEX scores range from 0 to 150, and higher scores reflect better performance.¹⁰

Other data were collected through a review of UTCOP student records in fall 2015. Based on previous NAPLEX studies, select pre-pharmacy and pharmacy school factors were included as independent variables.^3-6 The following data were collected: demographics including gender and race/ethnicity (categorized as non-Hispanic white, black/African-American, Asian, and other); pre-pharmacy academic performance factors including PCAT composite scores, PCAT section scores (biology, chemistry, quantitative, reading comprehension, verbal), undergraduate overall GPA, undergraduate science GPA, and achievement of an undergraduate degree; and pharmacy school academic performance factors including cumulative pharmacy school GPA (referred to as “pharmacy GPA”), on-time graduation, and number of unsatisfactory grades (none vs one or more D and/or F grades).

Statistical analyses were performed using IBM SPSS Statistics 22.0 (Armonk, NY). Participant characteristics were summarized by frequencies for categorical variables, and by means and standard deviations for continuous variables. Coding of categorical variables was as follows: gender, 0 = male and 1 = female; race/ethnicity, 0 = non-Hispanic white, 1 = black/African-American, 2 = Asian, and 3 = other; having an undergraduate degree, 0 = no and 1 = yes; on-time graduation, 0 = no and 1 = yes; and unsatisfactory grades, 0 = no (none) and 1 = yes (one or more). Analysis of variance (ANOVA) with Bonferroni post-hoc tests was used to assess differences in NAPLEX total scaled scores based on race/ethnicity, and an independent samples t-test was used to assess differences in NAPLEX scores based on gender, undergraduate degree, on-time graduation, and unsatisfactory grades. Pearson’s r correlation and point biserial correlation were used to assess the strength of the relationships between NAPLEX total scaled score, Pre-NAPLEX score, gender, race/ethnicity, undergraduate degree, PCAT composite score, PCAT section scores, undergraduate overall and science GPAs, pharmacy GPA, on-time graduation, and unsatisfactory grades. Stepwise linear regression was conducted to determine the significance of Pre-NAPLEX score, demographics, pre-pharmacy factors, and pharmacy school factors as predictors of NAPLEX total scaled score. To conduct the stepwise regression analysis, variables were entered into the model in one block.^12,13 “Stepwise” was selected as the method of linear regression analysis. Unlike the “Enter” method, which forces all variables into the model, the “Stepwise” method uses an automated process of entering and removing variables to find the “best fit” model.^13,14 In this manner, the stepwise regression analysis produces a final model that includes only significant independent variables. The a priori significance level was .05.

RESULTS

Of 170 eligible graduates, 150 (88.2%) were included. Fifteen were excluded because they did not sign NAPLEX release forms, and five were excluded because they had not yet taken the NAPLEX at the time of this study. There were no significant differences between participants and non-participants based on any of the independent variables (Pre-NAPLEX score, gender, race/ethnicity, pre-pharmacy factors, and pharmacy school factors; Table 1). As noted in Table 1, the majority of participants were female (60.7%) and non-Hispanic white (72.0%). Mean NAPLEX total scaled score was 104.7 (13.9), with scores ranging from 67 to 135. Mean Pre-NAPLEX score was 68.6 (15.0), with scores ranging from 31 to 110.

Table 1.

Characteristics of the Class of 2015 (n=170) and Comparison of Participants (n=150) and Non-participants (n=20)

In the Bonferroni post-hoc analysis of the ANOVA for race/ethnicity, non-Hispanic white graduates had significantly higher NAPLEX total scaled scores compared to black/African-American graduates, 107.0 (13.2) vs 94.4 (15.0), respectively; p<.01. There were no other statistically significant differences in NAPLEX total scaled scores based on race/ethnicity in the post-hoc analysis. In an independent samples t-test, participants who graduated on time had significantly higher NAPLEX total scaled scores compared to those who did not: 105.6 (13.4) vs 93.4 (15.0), respectively, p=.01. Independent samples t-tests did not find statistically significant differences in NAPLEX total scaled scores based on gender: males 104.9 (13.9) vs females 104.6 (13.9), p=.88; having an undergraduate degree: yes 104.7 (14.0) vs no 104.8 (13.4), p=.99; or having unsatisfactory grades: yes (one or more) 99.3 (15.9) vs no [none] 105.2 (13.6), p=.14.

NAPLEX total scaled score was significantly correlated with Pre-NAPLEX score, race/ethnicity, PCAT composite score, PCAT section scores, undergraduate GPA, undergraduate science GPA, pharmacy GPA, and on-time graduation (p<.05). The correlations are displayed in Table 2, which also presents the correlation matrix of the independent variables.

Table 2.

Correlations of NAPLEX Total Scaled Scores, Pre-NAPLEX Scores, Pre-Pharmacy Factors, Pharmacy School Factors, and Demographics of the Class of 2015 (n=150)

In the stepwise regression analysis, pharmacy GPA and Pre-NAPLEX score were significant and included in the final model (Table 3), accounting for approximately 44.4% (adjusted R² = .444; p<.01) of the variance. Pharmacy GPA accounted for the largest percentage of variance in the regression model (40.3%), and Pre-NAPLEX score accounted for the remaining 4.1%. The regression model was as follows:

Table 3.

Stepwise Regression Analysis Model Summary

NAPLEX total scaled score = 3.514 + 25.479 (pharmacy GPA) + .205 (Pre-NAPLEX score).

As tolerance was greater than .1 and Variance Inflation Factor (VIF) was less than 10,¹⁵ multicollinearity was not an issue in the final model (Table 3). Other assumptions of multiple linear regression also were met. Table 4 provides a summary of the variables that were excluded from the final regression model.

Table 4.

Summary of Variables Excluded in the Stepwise Regression Analysis

DISCUSSION

Although the Pre-NAPLEX is designed to resemble the NAPLEX testing experience,² a dearth of information exists concerning the relationship between the Pre-NAPLEX and NAPLEX. This study attempted to address the gap through examination of the association between NAPLEX and Pre-NAPLEX scores, as well as other factors that may affect the NAPLEX score, such as demographics and pre-pharmacy and pharmacy school academic performance factors. We found that NAPLEX total scaled score was correlated with multiple factors, including Pre-NAPLEX score, race/ethnicity, PCAT composite score, PCAT section scores (biology, chemistry, quantitative, reading comprehension, and verbal), undergraduate GPA, undergraduate science GPA, on-time graduation, and pharmacy GPA. However, in the final regression model, only pharmacy GPA and Pre-NAPLEX score were included as predictors of NAPLEX total scaled score. This finding is consistent with Allen and Diaz, who also found that pharmacy GPA was a predictor of success on the NAPLEX.³ In fact, pharmacy GPA was the strongest predictor of NAPLEX performance in the analysis, explaining approximately 40% of the variance in NAPLEX total scaled scores. This is unsurprising given that the purpose of the NAPLEX is to assess pharmacy practice knowledge acquired by an individual throughout pharmacy school.

Although explaining only 4% of the variance in NAPLEX total scaled scores, the inclusion of the Pre-NAPLEX in the regression model supports it has at least some utility in NAPLEX preparation. As noted by Peak and colleagues, pharmacy school graduates consider the Pre-NAPLEX one of the most useful licensure examination study tools available.¹⁶ Students may use the Pre-NAPLEX to gain a better understanding of the structure of the NAPLEX. As suggested by Chisholm-Burns and colleagues, colleges of pharmacy also may use the Pre-NAPLEX in combination with other performance indicators, such as pharmacy GPA, to identify low-performing students who may need academic intervention such as intensive content review in preparation for the NAPLEX.¹⁰

Recent changes in accreditation standards (ie, Standards 2016) require colleges of pharmacy to administer the Pharmacy Curriculum Outcomes Assessment (PCOA) to all students prior to beginning experiential education. This PCOA requirement may bring into question the need for the Pre-NAPLEX, although the two exams differ in intent, scope, delivery, and reporting. The PCOA is intended to provide evidence of student progress and growth throughout the curriculum, and is an indicator that students are ready to proceed to Advanced Pharmacy Practice Experiences (APPEs).¹⁷ PCOA post-test reports are extensive, providing data on four major content areas and allowing students to more precisely identify areas in need of improvement, whereas the primary purpose of the Pre-NAPLEX is to simulate the NAPLEX testing experience. In sum, while PCOA is a comprehensive exam that may benefit student learning overall, the Pre-NAPLEX is more aligned to the NAPLEX testing experience. Future studies of the NAPLEX should consider the effects of the both the Pre-NAPLEX and the PCOA.

Significant differences in NAPLEX total scaled scores were found based on on-time graduation in the independent samples t-test analysis, but this variable was not included in the final regression model. We consider a possible explanation as to why on-time graduation did not reach significance in the regression analysis. Participants who graduated on time had significantly higher NAPLEX total scaled scores than those who did not graduate on time. This is a logical association: Students who perform well academically (and therefore may be more likely to do well on the NAPLEX) generally graduate on time. On the other hand, students who have poorer academic performance (and therefore may be less likely to do well on the NAPLEX) may repeat pharmacy school coursework, which can result in not graduating on time. However, factors unrelated to academic performance such as a medical leave of absence also may play a role when a student does not graduate on time. Given this, it is reasonable that a more direct measure of academic performance, pharmacy GPA – which is calculated based on course grades throughout pharmacy school, including repeated courses – was included in the regression model, while on-time graduation was not.

Significant differences in NAPLEX total scaled scores based on race/ethnicity also were noted in the independent samples t-test analysis, indicating that non-Hispanic white students had higher scores than black/African-American students. This is consistent with Chisholm-Burns and colleagues, who found race-based differences in Pre-NAPLEX scores, with white students scoring significantly higher than minority students.¹⁰ However, the exclusion of race/ethnicity from the final regression model would seem to support that other factors, pharmacy GPA in particular, are more relevant to NAPLEX outcome. Previous research offers little clarity on this subject, as prior studies have not examined race/ethnicity-based differences in NAPLEX performance. Research has revealed that minority groups tend to score lower on a range of standardized tests (eg, Medical College Admission Test, Law School Admission Test) and licensure exams in fields such as nursing.^18-26 Yet, a synthesis of meta-analyses of graduate and professional school standardized entrance exams found no bias against minority groups.²⁷ Therefore, additional research is needed to better understand what role, if any, racial/ethnic bias plays in NAPLEX outcomes.

There are limitations to this study. As this was a retrospective study based on a student records review, causality cannot be established. Additionally, this study was conducted among students at only one college of pharmacy, which may limit the generalizability of the results. However, characteristics of the UTCOP Class of 2015 are similar to the national pharmacy graduate population for 2015, suggesting the results may be applicable on a larger scale. Further, the uniqueness of the study may serve as a catalyst for other colleges to examine factors associated with NAPLEX performance.^28,29

A notable caveat of the study is that the NAPLEX was undergoing revision in 2016. Although our findings demonstrate Pre-NAPLEX performance is a predictor of NAPLEX performance, this relationship may be affected by the revision. Future studies should assess the association between the Pre-NAPLEX and revised NAPLEX. Such ongoing assessment is critical, because a preparatory exam such as the Pre-NAPLEX is only of benefit so long as it remains a true reflection of the actual examination.

CONCLUSION

The findings of this study provide evidence that, although pharmacy GPA is the most critical determinant, the Pre-NAPLEX score is also a significant predictor of NAPLEX total scaled score. Future studies should continue to assess the relationship between the NAPLEX and Pre-NAPLEX, including exploring the utility of the Pre-NAPLEX in identifying students who need greater support in preparing for the NAPLEX.