Preadmission Predictors of PharmD Graduates' Performance on the NAPLEX

Abstract

Objectives

To identify prepharmacy variables that predicted graduate performance on the North American Pharmacist Licensure Examination (NAPLEX) test.

Methods

Undergraduate and pharmacy school transcripts for 424 students admitted to the Texas Tech University Health Sciences Center (TTUHSC) Doctor of Pharmacy program from the fall of 1996 to the fall of 2001 were reviewed. NAPLEX scores were provided by the Texas State Board of Pharmacy (TSBP). Statistical analyses were performed using SPSS, 11.5.

Results

Grade point average, composite Pharmacy College Admission Test (PCAT) score, and California Critical Thinking Skills Test (CCTST) score were each positively correlated with NAPLEX score (p < 0.001 for each). Type of school where organic chemistry was completed; age; advanced courses taken in chemistry, biology, and math; and attainment of a BS, BA, or MS degree were not significantly correlated with NAPLEX score (p > 0.05). When comparing students who successfully graduated and passed the NAPLEX with students who were academically suspended or who failed the NAPLEX, there were significant differences in mean PCAT scores, CCTST score, and age (p < 0.05 for all). Stepwise regression analysis applying all admission variables except PCAT subscores revealed a correlation between predictors (composite PCAT, prepharmacy GPA, and age) and NAPLEX score (p < 0.001).

Conclusion

Composite PCAT score was the strongest predictor of success and failure on the NAPLEX. However, the combined predictive ability of PCAT and CCTST scores, prepharmacy GPA, and age was relatively low. Thus, a full review of each candidate's application is justified.

INTRODUCTION

The National Association of Boards of Pharmacy Licensure Examination (NABPLEX) was first administered in 1976.¹ In 1986, the NABPLEX was significantly revised to a scenario-based format that required examinees to answer patient-centered questions rather than subject-centered questions. The NABPLEX examination changed from a pencil and paper format to a computer-adaptive test in 1997. In the same year, the name of the examination was altered to the North American Pharmacist Licensure Examination (NAPLEX) to reflect that the instrument was valid for both US and Canadian pharmacy practice. The most recent NAPLEX revisions, which included the addition of a specific competency addressing dietary supplements, were introduced in 2005. The blueprint for NAPLEX questions is kept current with the “knowledge and skills necessary to safely and effectively practice entry-level pharmacy.”² The scaled NAPLEX scores range from 0 to 150 with a minimally acceptable level of performance on the examination reflected by a score of 75.

Numerous studies have explored correlations between student preadmission characteristics and academic success during pharmacy school.^3-27 Yet, only 2 studies have evaluated student admission variables with NABPLEX scores.^12,14 Pharmacy College Admission Test (PCAT) scores were predictive of NABPLEX scores in both of these studies. However, these studies were conducted in the late 1970s and early 1980s; well before the patient profile-based format and the computer-adaptive test format. A likely reason why no recent studies evaluating individual licensure examination scores have been reported is that many schools and colleges of pharmacy have difficulty obtaining individual student NAPLEX results. One explanation for this difficulty is that many pharmacy graduates take board examinations in states other than where they attended pharmacy school. Another reason is that states may not have similar public records laws as Texas and many state boards will not release the NAPLEX results in a format needed to perform this type of analysis.

To our knowledge, the relationship between PCAT scores, the California Critical Thinking Skills Test (CCTST) score, admission age, advanced prepharmacy coursework, and the attainment of a BS, BA, or MS degree with success or failure on the NAPLEX have never been evaluated. Exploration of these relationships is important because the NAPLEX is a prerequisite to obtaining a license and the most recognized assessment of the minimum competency to practice pharmacy. Furthermore, NAPLEX scores reflect a standardized, external performance assessment and are required to be evaluated by schools and colleges of pharmacy for regional accreditation by the Southern Association of Colleges and Schools and by the Accreditation Council for Pharmacy Education.^28,29 Thus, licensing examination scores are an important component in the assessment system of a college or school. Therefore, evaluation of preadmission variables that predict success or failure on the NAPLEX should be considered by admission committees at schools and colleges of pharmacy.

METHODS

The Texas Tech University Health Sciences Center (TTUHSC) School of Pharmacy has offered the Doctor of Pharmacy (PharmD) degree as its only professional pharmacy degree since the first class was admitted in 1996. The TTUHSC PharmD degree program requires at least 2-years of specific undergraduate college study followed by 4 academic years of professional pharmacy study. The required science and math prepharmacy courses at Texas Tech include 8-10 semester hours of general chemistry with laboratory, 8-10 semester hours of organic chemistry with laboratory, 4 semester hours of general physics with laboratory, 8 semester hours of general biology with laboratory, 4 semester hours of microbiology with laboratory, 3-4 semester hours of calculus, and 3 semester hours of statistics.

Applicants to the TTUHSC School of Pharmacy are required to take the PCAT, submit transcripts from undergraduate institutions, complete a pharmacy experiential writing essay, have 3 letters of recommendation submitted, and submit a completed application (along with an application fee). Based on these requirements, students are invited for an on-campus interview. During the interviews, students are required to take the CCTST, complete a writing sample, participate in a group patient problem-solving session, and undergo interviews with faculty members and students. The Student Affairs Committee conducts a comprehensive review of each candidate who completed the on-campus interview and then selects applicants for admission into the PharmD program.

This study was approved by the TTUHSC Institutional Review Board for the Protection of Human Subjects (IRB number: A05 – 3284). Undergraduate and pharmacy school transcripts for 424 students admitted to the Texas Tech PharmD program between the fall semester of 1996 and the fall semester of 2001 were reviewed in August 2005. This timeframe was chosen because students admitted in fall 2001 could now be differentiated as having graduated on time in May 2005 or having been academically delayed or suspended. The prepharmacy variables included/considered in this study were: PCAT scores, CCTST score, age upon admission, prepharmacy GPA calculated only from required preprofessional courses, whether organic chemistry was taken at a 2-year or a 4-year institution, chemistry, biology, and/or math courses beyond required prerequisites, and attaining a BS, BA, or MS degree prior to admission. Interview evaluations were not included in this analysis because the weighting of subcomponents of the interview score has changed over the years. NAPLEX scores were made available by the TSBP in May 2006.

Student data were initially transcribed to an Excel spreadsheet and then converted to SPSS version 11.5 for analysis. Character data were encoded numerically when needed. For quality assurance, frequency distributions for all variables included in the research study were verified for reasonableness. Data were cleaned to provide quality assurance that there were no transcription errors in the data. Following data validation, basic descriptive statistics including means, standard deviations, and percentages were produced. Inferential statistical tests were also employed. The chi-square test was used with nominal data, and the independent-samples t test was used with continuous data. Bivariate correlations, including Pearson's and point biserial, were computed to test the magnitude and direction of relationships between variables.

To assess the impact of covariates, stepwise multiple regression was used to develop prediction models for NAPLEX scores. The multiple regression analysis utilized all independent variables except PCAT subscores. Dummy variables were created for categorical independent variables. To verify the appropriateness of the resultant regression models, the correlation matrix for the independent variables and the variance inflation factors were examined for multicollinearities; no problems were detected. For all analyses, the a priori level of significance was 0.05.

RESULTS

Of the 424 students admitted to the TTUHSC PharmD program from fall 1996 to fall 2001, 373 were included in the study analysis. Fifty-one students were excluded for the following reasons: 5 withdrew from the PharmD program for personal reasons, 6 withdrew from the PharmD program due to poor academic performance, 6 were academically delayed but not suspended, 18 were academically suspended, and 16 graduates took the board examination in a state other than Texas. Thus only 373 NAPLEX scores were made available by the TSBP. Composite PCAT scores were available for 418 students, PCAT subgroup scores for 399 students, CCTST scores for 415 students, age at admission for 424 students, and prepharmacy GPA for 424 students. Prepharmacy academic transcripts were available for all 424 students. The admission characteristics of the 373 individuals included in the primary analysis are presented in Table 1.

Table 1.

Characteristics of Students Admitted to a PharmD Program (N = 373)

PCAT = Pharmacy College Admission Test; CCTST = California Critical Thinking Skills Test; PGPA = prepharmacy grade point average; BS = bachelor of science; BA = bachelor of arts; MS = master of science

^*CCTST score based on a 34-point scale

^†PGPA based on a 4-point scale

Type of school where organic chemistry was completed (2-year or 4-year); age; advanced chemistry, biology, and math courses; and attainment of a BS, BA, or MS degree were not significantly correlated with NAPLEX score (Table 2). Composite PCAT score, prepharmacy GPA, and CCTST score were each positively correlated with NAPLEX score (p < 0.001 for each respectively). Furthermore, PCAT verbal, PCAT biology, PCAT reading, PCAT quantitative, and PCAT chemistry scores were each positively correlated with NAPLEX score (r = 0.262, 0.277, 0.251, 0.215, and 0.338 respectively; p < 0.001 for each). Of these admission variables, composite PCAT score had the highest correlation value (r = 0.400; p < 0.001) with NAPLEX score.

Table 2.

Correlation Between Admission Variables and NAPLEX scores (n = 373)

NAPLEX = North American Pharmacist Licensure Examination; PCAT = Pharmacy College Admission Test; GPA = grade point average; CCTST = California Critical Thinking Skills Test;

Stepwise regression analysis applying all admission variables except PCAT subscores revealed a correlation between 3 predictor variables (composite PCAT, PGPA, AGE) and NAPLEX score (p < 0.001; Table 3). Colinearity of independent variables was not a problem; the correlation matrix for independent variables revealed that all bivariate correlations were less than 0.561. Furthermore, none of the variation inflation factors were greater than 1.04. While age was not significantly correlated with NAPLEX scores on zero-order bivariate analysis; multivariate analysis revealed a significant correlation. When controlling for composite PCAT and prepharmacy GPA, the second-order partial correlation of NAPLEX with age was significant (r = −.189, p < 0.001). After controlling for composite PCAT, prepharmacy GPA, and age variables, CCTST scores did not significantly add to the regression model's ability to predict NAPLEX scores.

Table 3.

Stepwise Regression Analysis Between Admission Variables and NAPLEX Score

PCAT = Pharmacy College Admission Test, GPA = grade point average

Correlation values between PCAT scores and other admission variables are presented in Table 4. Not surprisingly, advanced chemistry coursework and organic chemistry taken at a 4-year institution were significantly associated with higher chemistry PCAT scores. Also as expected, advanced biology coursework and advanced mathematics coursework were significantly associated with higher biology PCAT and quantitative PCAT scores, respectively. The CCTST was significantly associated with higher scores in each PCAT section, especially verbal and reading PCAT scores.

Table 4.

Correlation Between PCAT Scores and Other Admission Variables Among All Admitted Students

PCAT = Pharmacy College Admission Test; GPA = grade point average; CCTST = California Critical Thinking Skills Test; Organic Chem 2 y vs. 4 y = organic chemistry course completed at a 2-year vs. a 4-year institution

^*p < 0.001

^†p < 0.01

Of the 424 admitted students, 24 prematurely exited the program (academically suspended or withdrew) due to poor academic performance. Of the NAPLEX scores made available by the TSBP, 11 of 373 failed on the first examination. Therefore, a total of 35 individuals were either academically suspended/withdrew or failed the NAPLEX on the first examination. Table 5 compares the mean PCAT scores, CCTST score, age, and prepharmacy GPA of these 35 students with students who graduated from the PharmD program and passed the NAPLEX. The mean score for each of these admission variables presented in Table 5 was significantly different between these 2 groups of students except for verbal PCAT score and prepharmacy GPA. Table 6 presents the number and percent of students who either graduated and passed the NAPLEX or who were academically suspended/withdrew or failed the NAPLEX, further divided by prepharmacy advanced college coursework and degree status. Of note, neither advanced college coursework in biology, chemistry, or math, nor earning a BS, BA, or MS degree prior to enrollment were significantly correlated with a higher likelihood of graduating and passing the NAPLEX versus academic suspension or NAPLEX failure.

Table 5.

Comparison of Variables for Students Who Successfully Graduated and Passed the NAPLEX vs. Students Who Were Academically Suspended, Withdrew, or Failed the NAPLEX

PCAT = Pharmacy College Admission Test, CCTST = California Critical Thinking Skills Test, PGPA = pre-pharmacy grade-point-average

Table 6.

PharmD Students Who Graduated and Passed the NAPLEX Compared With Students Who Were Academically Suspended or Withdrew, or Who Failed the NAPLEX

BS = bachelor of science; BA = bachelor of arts; MS = master of science

^*Chi-square test

DISCUSSION

The present study significantly updates and expands the known relationships between prepharmacy admission variables and performance on the NAPLEX. The results of this study confirm the findings of previous studies that have shown a correlation between PCAT scores and national licensure scores.^12,14 This correlation is not surprising as both the PCAT and NAPLEX are nationally standardized tests. However, several observations in our study were unexpected. Whereas we previously found a relationship between advanced college biology coursework beyond required prerequisites (eg, biochemistry) and/or a BS degree with academic success in pharmacy school, these admission variables were not significantly correlated with NAPLEX scores.³⁰ Furthermore, neither advanced college coursework nor earning a degree prior to enrollment were significantly correlated with a higher likelihood of passing the NAPLEX. Also, taking organic chemistry at a 2-year institution versus a 4-year institution did not correlate with NAPLEX scores. These findings may suggest that while courses such as biochemistry and/or a BS degree help students academically succeed in a school of pharmacy program, completion of a 4-year professional degree pharmacy program ultimately “levels the field.” Therefore, regardless of advanced college coursework prior to pharmacy school, our data indicate that pharmacy graduates are similarly if not equally capable of performing well on the NAPLEX. We have found no evidence in the limited scope of this study to support requiring a third year of college prerequisites or obtaining a baccalaureate degree prior to pharmacy school.

Another unexpected observation was the significant inverse association between admission age and NAPLEX score with stepwise regression analysis. This finding may be explained by unique psychosocial challenges and competing responsibilities that older students face while completing a professional degree program. Furthermore, the admission age of the students who graduated and passed the NAPLEX versus the students who did not complete pharmacy school due to low academic performance or who graduated but failed the NAPLEX was significantly younger. These results are contrary to the perception that older students will perform better than younger students in pharmacy school and on the NAPLEX because of greater maturity. Even comparing NAPLEX performance among the youngest students (those ≤20 years upon admission; n = 42) and those ≥21 years or older upon admission (n = 331), there was no significant difference in mean NAPLEX scores (105 ± 13 versus 104 ± 13; p = 0.796).

Another important finding was that the CCTST score did not add any additional value beyond the composite PCAT score for predicting NAPLEX scores. Additionally, CCTST scores had a strong positive correlation with PCAT scores in the present study as well as a previous study by Allen and Bond.²² This suggests that the PCAT and CCTST examinations overlap to some extent and that the PCAT may be a sufficient measure of critical thinking in and of itself. The ability of the PCAT to measure critical thinking needs to be evaluated at other schools and colleges of pharmacy.

Despite the significant finding on stepwise regression analysis that several factors (PCAT, prepharmacy GPA, and age) in this study correlate with NAPLEX score, the capability of these admission variables to predict performance on the NAPLEX was relatively low. An r² of 0.211 (the coefficient of determination) means that only 21% of the variance in NAPLEX performance is explained by these 3 factors. Thus, 79% of the variance in NAPLEX performance is explained by other factors. Accordingly, consideration of a candidate's PCAT scores and prepharmacy GPA as well as a full-file review of each candidate's application materials appears justified during the admission process.

Since the research design of this study was retrospective, random selection of subjects and random assignment of subjects to treatments could not be implemented. Accordingly, a true experimental design could not be achieved. Given this limitation, causality can not be established. In addition, it is possible that confounding variables existed that are not included in the analyses. The findings may not be readily generalized to other institutions or student populations that are not similar to TTUHSC with respect to student demographics, faculty characteristics, or program requirements. Further research, including replication using different student populations, different environments, and additional independent variables such as interview scores may serve to strengthen and expand the validity of the findings of this study.

CONCLUSIONS

Prepharmacy GPA, PCAT scores, and CCTST score were each positively correlated on bivariate analysis with NAPLEX score, whereas whether the students completed organic chemistry at a 2-year or 4-year institution, completed advanced chemistry, biology, and/or math courses, and/or attained a BS, BA, or MS degree were not. Of note, there was an inverse association between age and NAPLEX score on multivariate analysis. There was a low correlation between 3 predictor variables (composite PCAT, PGPA, AGE) and NAPLEX score. Because these variables only explained a small portion of the variance in NAPLEX performance, and because schools have outcomes in mind for their graduates that go beyond passing the NAPLEX, admission committees should continue to evaluate a variety of factors when admitting students into a PharmD program.