Evaluation of Student Factors Associated with Pre-NAPLEX Scores

Abstract

Objective: To examine relationships among students’ Pre-NAPLEX scores and prepharmacy, pharmacy school, and demographic variables to better understand factors that may contribute to Pre-NAPLEX performance.

Methods: A retrospective review of pharmacy students’ Pre-NAPLEX scores, demographics, prepharmacy factors, and pharmacy school factors was performed. Bivariate (eg, ANOVA) and correlational analyses and stepwise linear regression were conducted to examine the significance of various factors and their relationship to Pre-NAPLEX score.

Results: 168 students were included, with the majority being female (60.7%) and White (72%). Mean Pre-NAPLEX score was 68.95 ± 14.5. Non-Hispanic White students had significantly higher Pre-NAPLEX scores compared to minority students (p<0.001). Pre-NAPLEX score was correlated (p<0.001) to race/ethnicity (r=-0.341), PCAT score (r=0.272), and pharmacy school GPA (r=0.346). The regression model (adjusted R²=0.216; p<0.001) included pharmacy school GPA, academic probation, academic remediation, and PCAT composite.

Conclusion: This study highlighted that select demographic, prepharmacy, and pharmacy school factors were associated with Pre-NAPLEX outcomes. Such factors may assist colleges/schools of pharmacy in identifying students who may be at risk for poorer NAPLEX performance and may need greater preparation.

INTRODUCTION

Because success on the North American Pharmacist Licensure Examination (NAPLEX) is essential to entry of pharmacy graduates into the profession, it is useful to develop a better understanding of factors that are predictive of success on the NAPLEX. Previous studies have explored the relationships between various prepharmacy and pharmacy school factors and NAPLEX outcomes.^1-3 For example, Allen and Diaz examined associations between NAPLEX scores, prepharmacy criteria such as undergraduate cumulative grade point average (GPA), undergraduate math/science GPA, and having an undergraduate degree (eg, bachelor’s degree), and pharmacy school criteria such as number of unsatisfactory grades (ie, “D” or “F” in the prepharmacy and PharmD program), on-time graduation, and cumulative pharmacy GPA.¹ They found that undergraduate GPA, undergraduate math-science GPA, pharmacy school GPA, on-time graduation, and number of unsatisfactory grades were significantly correlated to NAPLEX score. In regression analyses, significant predictors of better NAPLEX scores included higher pharmacy school GPA and fewer unsatisfactory grades.¹ In a similar study, McCall and colleagues assessed the relationships between NAPLEX scores and prepharmacy variables (eg, Pharmacy College Admission Test [PCAT] score, having an undergraduate degree, prepharmacy GPA based on required professional courses, among others) and found that composite PCAT score was the strongest predictor of NAPLEX score.² Madden et al studied the relationship between first-time NAPLEX performance and course remediation during pharmacy school and found that students who had remediation had lower first-time NAPLEX pass rates compared to students who progressed on time.³

None of the aforementioned studies considered the impact of NAPLEX preparatory tools on NAPLEX scores. Peak et al found that certain preparatory tools were more highly rated in utility than others.⁴ For example, the Pre-NAPLEX was ranked as the most representative of the actual NAPLEX and was the fourth most useful instrument of 17 preparatory tools.⁴ According to the National Association of Boards of Pharmacy (NAPB), the objective of the Pre-NAPLEX is to provide students with exposure to a simulated “NAPLEX testing experience.”⁵ Use of the Pre-NAPLEX in this manner may assist colleges and schools of pharmacy in identifying students who, based on their Pre-NAPLEX performance, are in need of more intensive preparation for the NAPLEX. Schools would then have the opportunity to provide such preparation prior to graduation. Factors that negatively influence Pre-NAPLEX score may also indicate the need for greater academic intervention in students who may be at risk for poor NAPLEX performance following graduation. No studies have investigated factors that may determine Pre-NAPLEX score. The purpose of the current study was to address this gap in the literature and examine relationships among Pre-NAPLEX scores and prepharmacy, pharmacy school, and demographic variables to better understand factors that may contribute to Pre-NAPLEX performance.

METHODS

The study was based on a retrospective review of records of students at the University of Tennessee College of Pharmacy (UTCOP), a public institution that offers a 4-year PharmD curriculum. The UTCOP PharmD program requires that students complete 90 hours of prerequisite undergraduate coursework, with more than 45% of those hours devoted to science courses. All UTCOP students are required to participate in didactic coursework for their first 5 semesters, while the final 3 semesters are devoted to experiential learning. UTCOP has adopted an integrated examination model, in which content from multiple courses is included in a single exam. Students undergo this computer-based testing every 2 weeks throughout the semester. Such an approach increases student familiarity with this particular testing format and provides a foundation for students as they prepare for the computer-based NAPLEX.

The Pre-NAPLEX, first implemented in 2003, is a web-based, 50-item, 70-minute examination made available by NAPB, which is also responsible for producing the NAPLEX.^5,6 As with the NAPLEX, scores on the Pre-NAPLEX range from 0 to 150, with higher scores indicating better performance.⁵ The Pre-NAPLEX includes items that have previously appeared on the NAPLEX; however, NAPB explicitly states that performance on the Pre-NAPLEX is not necessarily predictive of performance on the NAPLEX. Rather, as stated previously, the purpose of the Pre-NAPLEX is to have exposure to the experience of taking the NAPLEX.⁵ The Pre-NAPLEX was administered for the first time at the UTCOP to all third-year (P3) students on the same date. Students were notified within 1 month of the test date and were asked to not study so that the examination would provide an unbiased assessment of the students’ preparedness for the NAPLEX.

This study was approved by the University of Tennessee Health Science Center Institutional Review Board, and all P3 students were eligible for inclusion. Data were collected through a review of UTCOP student records in January 2014. Prepharmacy and pharmacy school factors selected for inclusion were based on variables identified in previous studies examining predictors of NAPLEX score.^1-3 The following data were collected: demographics (gender, race/ethnicity), prepharmacy factors (ie, PCAT composite percentage scores, undergraduate GPA, undergraduate science GPA, achievement of an undergraduate degree), and pharmacy school factors (ie, pharmacy school cumulative GPA, academic probation status, if the student was required to remediate a year due to poor academic performance). Because the Pre-NAPLEX was administered at the end of the fifth semester of pharmacy school, the pharmacy school cumulative GPA included semesters 1 through 5. Academic probation is imposed if the student’s GPA at the conclusion of the first semester of the first professional year, or the cumulative GPA at the conclusion of any semester thereafter, is less than 2.33; the grade point average earned for any one semester is less than 2.0; or if the student receives less than a C- in any course. Students may be required to remediate (repeat) an academic year if they have a cumulative GPA less than 2.33 or a semester GPA less than 2.0 for a second time, or if they receive less than a C- in 2 courses throughout the curriculum. Regarding race/ethnicity, students were categorized as non-Hispanic White, Black/African American, Asian and Other. The “Other” category was created to account for those students in groups with very low representation in the cohort under study (eg, Hispanic students, Native American students).

Statistical analyses were performed using SPSS Statistics 20.0 (IBM, Armonk, New York). Participant characteristics were summarized by frequencies for categorical variables, and by means and standard deviations for continuous variables. Coding of categorical variables was as follows: (1) gender, 0=male and 1=female; (2) race/ethnicity, 0=non-Hispanic White, 1=Black/African American, 2=Asian, and 3=Other; (3) having an undergraduate degree, 0=yes and 1=no; (4) academic probation, 0=no and 1=yes; and (5) academic remediation, 0=no and 1=yes. Analysis of variance (ANOVA) was used to assess differences in Pre-NAPLEX scores based on race/ethnicity, and independent samples t test was used to assess differences in Pre-NAPLEX scores based on gender. Pearson’s r correlation and point biserial correlation were used to assess the strength of the relationships between Pre-NAPLEX scores and demographic factors, prepharmacy factors, and pharmacy school factors. Stepwise linear regression was conducted to determine the significance of prepharmacy and pharmacy school factors as predictors of Pre-NAPLEX score. The a priori significance level was 0.05.

RESULTS

Of 174 eligible students, 168 (96.6%) were included. Six were excluded due to missing data. As noted in Table 1, the majority of students were female (60.7%) and non-Hispanic White (72.0%). Among the included students, 7.7% were subject to academic probation and 7.1% were subject to academic remediation (no students were subject to both). Mean PCAT composite percentage score was 69.3 ± 16.4, mean undergraduate GPA was 3.49 ± 0.33, and mean pharmacy school GPA was 3.19 ± 0.40. The mean Pre-NAPLEX score was 68.95 ± 14.5, with scores ranging from 31 to 110.

Table 1.

Characteristics of Third-Year Pharmacy Students

Non-Hispanic White students had significantly higher Pre-NAPLEX scores (p<0.001) compared to minority students. Specifically, non-Hispanic White students had significantly higher Pre-NAPLEX scores compared to Black/African American students (72.2 ± 12.9 vs 60.2 ± 18.6, respectively; p=0.005), Asian students (72.2 ± 12.9 vs 62.1 ± 14.7, respectively; p=0.012), and students in the Other category (72.2 ± 12.9 vs 57.2 ± 15.4, respectively; p=0.01). There were no statistical differences in Pre-NAPLEX scores among Black/African American, Asian, and Other students. There were also no differences in Pre-NAPLEX scores based on gender (males 70.67 ± 15.34 vs females 67.83 ± 13.94; p=0.218).

Correlations of Pre-NAPLEX scores with demographic factors, prepharmacy factors, and pharmacy school factors are displayed in Table 2. Pre-NAPLEX score was significantly correlated to: (1) race/ethnicity (r=-0.341, p<0.001), with non-Hispanic White students having significantly higher scores than minority students; (2) PCAT composite score (r=0.272, p<0.001), with Pre-NAPLEX scores increasing as PCAT scores increased; and (3) pharmacy school cumulative GPA (r=0.346, p<0.001), with Pre-NAPLEX scores increasing as GPA increased.

Table 2.

Correlations of Pre-NAPLEX Scores and Student Characteristics

In the stepwise regression analysis, the following variables were included in the final model, accounting for approximately 22% (adjusted R²=0.216) of the variance in the model (p<0.001): pharmacy school cumulative GPA, academic remediation, academic probation, and PCAT composite score. Pharmacy school cumulative GPA accounted for the largest percentage of variance in the regression model (11.4%). Among the other variables in the model, academic remediation accounted for 3.5% of the variance, academic probation accounted for 4.7% of the variance, and PCAT composite score accounted for 2% of the variance. The regression model was as follows: Pre-NAPLEX score = 2.699 + 17.086(pharmacy school cumulative GPA) + 13.650 (academic remediation) + 12.256(academic probation) + 0.143(PCAT composite score).

DISCUSSION

Factors that influence Pre-NAPLEX scores and its use as a tool for determining pharmacy student progression toward readiness to take the NAPLEX are largely unexplored in the academic pharmacy literature. The current study attempted to understand prepharmacy, pharmacy school, and demographic factors that may affect Pre-NAPLEX scores. Previous studies of factors that influenced NAPLEX scores found that prepharmacy variables, such as PCAT composite score and prepharmacy GPA, as well as pharmacy school variables, such as pharmacy school cumulative GPA, were significant predictors of NAPLEX score.^1,2 Similar to findings of prior studies, this study found that PCAT composite score and pharmacy school cumulative GPA were significantly, positively correlated to test scores.^1,2 The association between pharmacy school cumulative GPA and Pre-NAPLEX scores was logical considering the test content is pharmacy related. However, the correlations between Pre-NAPLEX scores, pharmacy school cumulative GPA, and PCAT composite scores were weak to moderate (according to guidelines suggested by Cohen, wherein a correlation coefficient of 0.1 is small, 0.3 is medium, and 0.5 is large⁷), indicating other factors or a combination of factors may have had greater influence on Pre-NAPLEX score in our population. This postulation is supported by the findings of the regression analysis, in which 3 pharmacy school factors (pharmacy school cumulative GPA, academic probation, and academic remediation), and PCAT composite score were combined to produce a final model that explained approximately 22% of the variance in Pre-NAPLEX scores. It is worth noting that only one of the pre-pharmacy factors, PCAT composite score, was included in the final model and accounted for only 2% of the variance in that model. This suggests that pharmacy school factors play a more influential role in determining Pre-NAPLEX outcome than prepharmacy factors.

An interesting finding of the regression analysis was the contribution of academic probation and remediation of an academic year to the model. Both factors, probation and remediation, had significant, positive coefficients in the regression model, suggesting a beneficial influence on Pre-NAPLEX scores. It is logical that pharmacy school performance would be predictive of Pre-NAPLEX performance, as the exam reflects pharmacy-related content. However, it seems counterintuitive that 2 factors resulting from poor pharmacy school performance would prove to be of benefit. In our examination, we found the mean Pre-NAPLEX score was 71.5 ± 10.2 for students who received academic probation and 68.7 ± 14.8 for students who were never on academic probation. Likewise, the mean Pre-NAPLEX score of students who received academic remediation was 72.0 ± 15.5 compared to 68.7 ± 14.5 for those who were not subjected to remediation. We speculate that academic probation and remediation were significant in the regression model for a few reasons. First, students who received academic remediation may have been better prepared for the Pre-NAPLEX as a result of being required to repeat coursework. Another possible explanation is that because these students repeated course material, the content may have been more reinforced in the minds of these students, facilitating greater recall. Additionally, students who face academic probation and remediation are at higher risk of the consequences of negative academic performance compared to those students who are not in academic trouble. In turn, these students are motivated to prevent further sequela, such as moving from probation or remediation to dismissal from pharmacy school. As a result, students subjected to probation or remediation may have taken preparation and performance on the Pre-NAPLEX more seriously than their classmates. Although informed the Pre-NAPLEX results would not be part of any course grade and they should not study for the examination, students with academic challenges may have felt a greater degree of pressure to perform on the Pre-NAPLEX since faculty members and/or administrators would be reviewing examination performance. In summary, this was an unexpected and intriguing finding that contradicted Madden et al’s findings where remediation was found to adversely impact NAPLEX scores.³ Further studies are needed to explore the influence of academic remediation and probation on Pre-NAPLEX and NAPLEX performance.

There were significant differences in Pre-NAPLEX score based on race/ethnicity, namely that non-Hispanic White students had higher scores than minority students. Prior research on race/ethnicity and standardized testing suggested minority groups score notably lower on a wide array of standardized examinations including the SAT, the Medical College Admission Test (MCAT), Graduate Management Admission Test (GMAT), and Law School Admission Test (LSAT).^8-11 Previous studies also indicated racial/ethnic disparities in pass rates on licensure examinations in other professions such as teaching and nursing.^12-16 For example, multiple studies have noted that Whites have significantly higher pass rates on the National Council Licensure Examination for Registered Nurses compared to minority groups.^14-16 However, prior published studies of the NAPLEX did not consider racial/ethnic disparities. Despite noted score disparities in standardized testing, Kuncel and Hezlett’s synthesis of meta-analyses of graduate and professional school standardized entrance examinations, including the PCAT, found there was no bias against racial/ethnic groups.¹⁷ Additional research is needed to determine whether any racial/ethnic bias exists in the Pre-NAPLEX, as well as the NAPLEX, as such studies are absent from the literature.

We administered the Pre-NAPLEX to the P3 class to provide students with an experience similar to that of the actual NAPLEX. Building such familiarity with NAPLEX testing procedures may assist students as they prepare for the NAPLEX during their remaining time in pharmacy school. Secondarily, we used the Pre-NAPLEX as an assessment tool to gain a sense of how students are progressing in terms of preparation for the NAPLEX. The availability of this assessment may help identify students in need of greater preparation. Future research may consider assessing the progression of the cohort and analyzing the value of the Pre-NAPLEX as a preparatory tool after the NAPLEX has been taken. Additional studies may want to consider other ways the Pre-NAPLEX could be integrated into the assessment repertoire of colleges and schools of pharmacy, such as using the Pre-NAPLEX as a high-stakes examination to determine academic progression and student readiness to begin advanced pharmacy practice experiences (APPEs).

There were limitations to this study. As it was a retrospective study based on a student records review, causality cannot be established. However, these studies tend to be retrospective in nature and, moreover, the findings were generally consistent with prior studies of the NAPLEX, suggesting indicators of prior academic performance (eg, pharmacy school cumulative GPA) are associated with level of achievement on the Pre-NAPLEX. Additionally, this study did not examine the relationship between the Pre-NAPLEX and NAPLEX. As this was the first year the Pre-NAPLEX was administered in our institution and the study group had not yet taken the NAPLEX, such an investigation was not possible. However, there is value in focusing on possible predictors of Pre-NAPLEX score as such indicators may assist colleges and schools in identifying those students who are in need of more intensive NAPLEX preparation.

CONCLUSION

Virtually no studies exist addressing the Pre-NAPLEX, a tool used to familiarize students with the experience of taking the NAPLEX. The current study sought to redress this gap in the literature and examine prepharmacy, pharmacy school, and demographic factors that might be associated with the Pre-NAPLEX scores. Findings indicated that PCAT composite score, pharmacy school cumulative GPA, and race/ethnicity were correlated with Pre-NAPLEX score. Additionally, the final significant regression model included pharmacy school cumulative GPA, PCAT composite score, academic probation, and remediation of an academic year as predictors of Pre-NAPLEX score. Thus, this study highlighted that select demographic, prepharmacy and pharmacy school factors were associated with Pre-NAPLEX performance. Such factors may assist to better identify students who are at risk for poorer NAPLEX performance and may require greater preparation.