A Predictive Validity Study of the Pharmacy College Admission Test

Abstract

Objective

To examine the validity of Pharmacy College Admission Test (PCAT) scores for predicting grade point averages (GPAs) of students in years 1-4 of pharmacy programs.

Methods

Data were collected from 11 colleges and schools of pharmacy: entering cumulative and math/science GPAs, PCAT scaled scores, pharmacy program GPAs for years 1-4, student status after 4 years. Correlation, regression, discriminant, and diagnostic accuracy analyses were used to determine the validity of the PCAT for predicting subsequent GPAs.

Results

PCAT scaled scores and entering GPAs were positively correlated with subsequent GPAs. Regression analyses showed the predictive value of the PCAT scores, especially in combination with entering GPAs. Discriminant and diagnostic accuracy analyses supported these findings and provided practical suggestions regarding optimal PCAT scores for identifying students most likely to succeed.

Conclusion

Both PCAT scaled scores and entering cumulative GPAs showed moderate to strong predictive validity as indicators of candidates likely to succeed in pharmacy school.

INTRODUCTION

The Pharmacy College Admission Test (PCAT) consists of 5 multiple-choice subtests (verbal ability, biology, reading comprehension, quantitative ability, and chemistry), with scores reported for each subtest and as a composite score (an unweighted average of the 5 subtest scaled scores). According to the American Association of Colleges of Pharmacy (AACP), the PCAT is currently used by 65 (68%) of the 95 colleges and schools of pharmacy in the United States as part of their admission criteria.

Since the introduction of the PCAT in 1974 by The Psychological Corporation (now Harcourt Assessment, Inc), a number of studies have investigated the predictive validity of the PCAT and have found the test to have value as a predictor of subsequent performance in pharmacy programs. These studies generally have found the PCAT to be a moderate to strong predictor of both pharmacy-program grades and licensing examination scores.^1,2 Studies of the predictive validity of the PCAT typically have used correlation and/or multiple regression analyses to examine the value of using PCAT scores to make predictions about subsequent GPAs earned by students in pharmacy school and to make predictions about other such indicators of success.

Several early studies conducted at various universities during the 1970s emphasized how well PCAT scores related to the academic performance of students entering schools of pharmacy. Research comparing the performance of first-year professional pharmacy students to their PCAT scores found that PCAT scores correlated positively with subsequent performance in specific coursework and throughout the course of a program.^3,4 PCAT scores were also predictive not only of students' performance in pharmacy school but also of their scores on the National Association of Boards of Pharmacy Licensing Exam (NABPLEX).^4,5 One study by Lowenthal found the PCAT to be a better predictor of academic performance in pharmacy school than the SAT-Verbal, SAT-Math, and Otis Quick-Scoring Mental Ability Test scores.⁵

In a 1985 study by The Psychological Corporation, in cooperation with the AACP, data collected from several pharmacy programs showed multiple correlations between PCAT scores and GPAs of first-year pharmacy students ranging from 0.35 to 0.77. When first-year pharmacy students' PCAT scores were combined with their prepharmacy GPAs, the correlations with their first-year pharmacy GPAs ranged from 0.56 to 0.82.⁶

Other studies conducted in the 1980s and 1990s generally showed that the PCAT provided useful assistance to colleges and schools of pharmacy with their admission decisions.⁷ One study conducted from 1984 to 1989 by Kawahara and Ethington found a declining trend in PCAT scores and found the biology and chemistry scores to be particularly sensitive to previous coursework.⁸ These conclusions, however, supported a common misperception at the time that the academic quality of entering pharmacy students had been declining. Another study examining first-year pharmacy students' performance focused especially on the development of desirable non-academic qualities and found the PCAT Verbal Ability and Reading Comprehension scores to be useful, not only in predicting first-year academic success, but also as the best predictors of characteristics such as responsibility, thinking ability, and professionalism.⁹

One study conducted in the 1980s by Bandalos and Sedlacek found that in addition to prepharmacy GPAs, only the PCAT Reading Comprehension scores were factors in predicting the performance for both African-American and Asian students and that PCAT Biology and Reading Comprehension scores were the strongest predictors of first-year pharmacy students' GPAs for the total group.¹⁰ A study by Chisholm, Cobb, and Kotzen on factors useful in predicting first-year academic performance found PCAT verbal ability and composite scores to be stronger predictors of academic success for female students than for male students.¹¹ A study conducted by Wu-Pong and Windridge found the best predictors of pharmacy GPAs to be the PCAT chemistry scores, followed by entering GPAs and the PCAT composite scores for applicants whose primary language was not English.¹² Another study also found significant differences in mean PCAT scores among gender, racial, and native-language subgroups, but the researchers concluded that PCAT scores combined with GPAs were still useful in selecting applicants for admission to pharmacy school.¹³

More recent research on the ability of the PCAT to predict academic performance supports earlier findings regarding the general usefulness of the test. In one study, Allen and Bond evaluated the relationship between academic performance, PCAT scores, the California Critical Thinking Skills Test (CCTST) scores, interview scores, and GPAs.¹⁴ PCAT and CCTST scores were the strongest predictors of subsequent success in courses and concluded that the PCAT may predict critical thinking skills in addition to academic success. In a different study of changes in critical thinking over a 4-year doctor of pharmacy (PharmD) program, Miller found PCAT and CCTST scores to be highly correlated and concluded the PCAT score was a strong indicator of critical thinking.¹⁵ Another study by Grillo, Latif, and Stolte examined the relationship between several preadmission indicators of basic math skills in first-year pharmacy students and found students' scores on the quantitative ability section of the PCAT to be a significant predictor of whether the students would commit medication errors once they entered pharmacy practice.¹⁶

Thomas and Draugalis found the PCAT to be a significant predictor of student performance in the first year of pharmacy school, with PCAT chemistry scores correlating highly with first-year pharmacy students' GPAs (0.58), followed by entering math/science GPAs (0.48) and PCAT composite scores (0.50).¹⁷ In combination with previous GPA and college degree earned, PCAT scores account for over 40% of the variance in first-year pharmacy students' GPAs when multiple correlations were adjusted to account for chance error and sample size. In another recent study, Kidd and Latif investigated the relationship between 7 predictors and examined how these predictors contribute to pharmacy students cumulative GPAs, GPAs for professional years 1-3 in pharmacy school, and GPAs of advanced pharmacy practice experiences (APPEs) in the fourth year of pharmacy school.¹⁸ The researchers concluded that the PCAT is a significant predictor of success in pharmacy classroom courses and in pharmacy school overall. In a study of 1982-2002 student performance at one pharmacy school, Granberry and Steigler found that PCAT scores rose over the study period, but that correlations between PCAT scores and professional pharmacy school GPAs dropped.¹⁹ However, the researcher speculated that the greater increase in pharmacy grades than in PCAT scores may have been due more to grade inflation than to improved student performance.

A recent meta-analysis of the research related to the PCAT conducted by Kuncel and colleagues found that both prepharmacy GPAs and PCAT scores are valid predictors of performance in pharmacy school and on licensing examinations.¹ According to these researchers, evidence from many research studies conducted over the years suggests that the PCAT continues to be a valid predictor of student success during the first 3 years of pharmacy school, especially for first-year GPA, and is a strong predictor of performance on the NABPLEX (now known as the NAPLEX, the North American Pharmacy Licensure Exam). They concluded that any skepticism regarding the predictive validity of the PCAT is unnecessary and that together with prepharmacy GPAs, PCAT scores can be used in admissions decisions to substantially increase the likelihood of identifying students who will perform successfully in pharmacy programs.

In recent years, nearly all colleges and schools of pharmacy have changed from offering only a bachelor of science (BS) degree in pharmacy, or the traditional postgraduate PharmD, to a first-professional PharmD degree, most of which require 2 or more years of prepharmacy coursework and 4 years of pharmacy coursework.²⁰ The PCAT continues to play an important role in selecting students for admission for the majority of these first-professional PharmD degree programs.

The purpose of this study was to examine data collected from 11 colleges and schools of pharmacy and to determine the value of PCAT scaled scores and entering GPAs in predicting subsequent GPAs during 4-year pharmacy programs. The methods of analyses employed in this study included correlation and regression analyses, the 2 methods most commonly used in previous studies, as well as discriminant analysis and diagnostic accuracy analysis, 2 methods not used in previous studies of the PCAT. The results of these analyses support earlier findings on the predictive validity of the PCAT.

METHODS

To determine the predictive validity of the PCAT in terms of GPA over a 4-year professional pharmacy program, the following data were collected from 11 colleges and schools of pharmacy for students entering in the fall of 2000: entering cumulative GPAs, entering math/science GPAs, PCAT scaled scores, GPAs for years 1-4 in the pharmacy programs (for each year separately, not cumulative), and student status after 4 years (at the end of the 2003-2004 academic year). These data were analyzed to determine the validity of the PCAT in predicting subsequent GPAs using correlation analyses, regression analyses, discriminant analyses, and diagnostic accuracy analyses.

While nearly all similar previous studies of the PCAT used correlation coefficients as indicators of predictive validity and many used regression analyses, none employed discriminant analysis or diagnostic accuracy analysis. These 2 analyses support and extend indications of predictive validity for the PCAT in both theoretical and practical ways that will be discussed in this report.

In early 2004, the AACP provided the researchers with a list of 47 colleges and schools of pharmacy in the United States that used the PCAT for admission decisions at that time. All of these schools were invited to participate in the study, and 11 agreed to participate. Of the 11 schools that participated, 8 were public universities and 3 were private universities. Using the US Census Bureau state regions and divisions to categorize the participating schools, 7 are located in the South (3 in the South Atlantic Division, 2 in the East South Central Division, and 2 in the West South Central Division), 2 schools are located in the Midwest (one in East North Central Division and one in West North Central Division), and 2 schools are located in the West (both in the Mountain Division).

A spreadsheet was sent to each participating school with the request for admission criteria and data on all candidates admitted for fall 2000, including PCAT scaled scores, entering cumulative GPAs, entering math/science GPAs, other standardized test scores, prepharmacy coursework completed, GPAs at the end of each of the 4 years of pharmacy study, and status at the end of the fourth year (graduated/still enrolled/discontinued enrollment). Information from the AACP web site and the colleges' and schools' of pharmacy web sites was analyzed to determine prerequisite course requirements and pharmacy school curricula. The researchers categorized the prerequisite courses and the courses required in the professional pharmacy programs for comparative purposes.

All data analyses were done using Statistical Analysis System (SAS, version 9.1). The Pearson product-moment correlation coefficient was used to determine the degree to which PCAT scaled scores and entering GPAs were related to subsequent GPAs in the pharmacy programs. In this study, PCAT scaled scores and entering GPAs were used as independent predictor variables, and student performance in years 1-4 of the professional pharmacy programs was used as dependent outcome variables. The outcome variables include the GPA in each of the 4 years of professional pharmacy study for students who graduated with a PharmD after 4 years, students who graduated with a BS within 4 years, students who were still enrolled after 4 years, and students who had discontinued enrollment prior to the fourth year.

The data analyses methods used for this study included correlation analyses, regression analyses, discriminant analyses, and diagnostic accuracy analyses. For correlation analyses, correlations were determined between PCAT scaled scores (for each subtest and Composite) and GPAs in years 1-4 of the professional pharmacy programs, and between entering GPAs (cumulative and math/science) and GPAs in years 1-4 of the professional pharmacy programs. For multiple regression analyses, 2 regression models were used. In one model, PCAT subtest scaled scores were combined with entering cumulative GPAs as predictor variables to predict subsequent GPAs in years 1-4. In a second model, PCAT subtest scaled scores were used alone to predict subsequent GPAs in years 1-4.

Discriminant analyses were done by weighting each variable according to its relative predictive contribution to group member performance and status using the optimal linear combination of independent variables. In this study, 2 discriminant analysis models were used, one with the 5 PCAT subtest scaled scores combined with entering cumulative GPAs as the predictor variables, and one with the 5 PCAT subtest scaled scores alone as the predictor variables. In both models, the predictor variables represent optimal values in predicting percentages of students from among those earning the lowest 5% (sensitivity, the percentage of low performers correctly predicted) and the highest 95% (specificity, the percentage of high performers correctly predicted) of GPAs in the professional pharmacy programs. In this study, 5% was selected as the low-performing group because approximately 5% of the students who enrolled in 2000 left the program prior to the fourth year and earned the lowest GPAs compared to students who graduated or were still enrolled after 4 years. Likewise, for purposes of this study, 95% represents the high-performing group because around 95% of the entering students either completed a degree or were still enrolled after 4 years of pharmacy school.

Though similar to discriminant analysis, diagnostic accuracy (or signal detection) analysis was also used to list all predictor variables, along with the corresponding sensitivity and specificity percentages for each. In this analysis, the same 2 categories of student performance were used to determine diagnostic accuracy as with discriminant analysis (percentages of the lowest 5% and the highest 95% of performance for specific predictor variables). However, for the diagnostic accuracy analysis, PCAT composite scaled scores were used as the predictor variables to determine the best range of PCAT composite scaled scores from the sample that could be used to identify both students unlikely to complete the program successfully (sensitivity) and those likely to perform in the program successfully (specificity). For this analysis, index values were calculated that were added to the mean PCAT composite scaled score for the study sample to determine probabilities of success for specific PCAT composite scaled scores.

For all analyses, the GPA data collected from the participating colleges and schools of pharmacy were equated on a common scale to accommodate for differences in grading scales (ie, 4.0 or 100), and all PCAT scaled scores were linked to adjust for the differences in sample means. Multiple imputation (MI) was used to fill in any missing PCAT scaled scores and GPAs with plausible values. MI was used only in cases where a student's PCAT subtest score or GPA was missing from the data submitted by a pharmacy school or seemed implausible (possibly the result of a data entry error).

RESULTS

General Characteristics of Participating Pharmacy Programs

Tables 1 and 2 summarize the prerequisite course credit requirements and professional pharmacy curricula for the 11 pharmacy programs that participated in this study. The information in Table 1 shows that the prepharmacy requirements at the participating schools emphasized credits in the biological sciences, chemistry, and the social sciences and humanities, with fewer required in English and speech, math, and physics. The information in Table 2 shows that the curricula in the pharmacy programs varied in the types of credits required each year, with increasing concentrations on credits in professional pharmacy courses, internships, and other profession-related courses.

Table 1.

Prerequisite Requirements for the 11 Participating Pharmacy Programs

Table 2.

Pharmacy Program Academic and Professional Course Requirements for the 11 Participating Pharmacy Programs

Table 3 shows the characteristics of the students in the study sample in terms of mean PCAT scaled scores, mean entering GPAs, and mean GPAs in the professional pharmacy programs for the 11 schools. The scaled scores for students in this study were based on the 1992 norms in effect in 2000 (used from 1992-2003), which ranged from 100-300 with a mean of 200. The pharmacy GPAs listed in Table 2 only include data for the 10 schools that reported GPAs on a 4-point scale. Sample sizes are also listed for all students entering a pharmacy program in the fall of 2000 and for each of the 4 years in the pharmacy programs.

Table 3.

Characteristics of Students Entering in the Fall 2000 for 11 Participating Pharmacy Programs: Mean Scaled Scores and Grade Point Averages

* Mean scaled scores are listed for entering students. GPA = mean grade point average; Entering GPA = earned prior to admission to pharmacy program; Pharmacy GPA = earned during the 4 years of the professional pharmacy program. The PCAT, GPA, and n-count data listed in this table only include cases where the information provided was complete or clear; PCAT scaled scores and GPAs have been adjusted for outliers and missing values. Entering cumulative and math/science GPAs were not reported for all entering students. The following standard deviations (SDs) are for all categories of students: PCAT subtest SS SDs range 21.5-38.0; PCAT Composite SS SDs range 19.3-24.0; PCAT PR SDs range from 25-35; entering cumulative GPA SDs range 0.36-0.42; entering math/science GPA SDs range 0.43-0.49; GPA in program SDs range 0.35-0.71

From the data listed in Table 3, at the end of 4 years, the vast majority of students had either graduated with a PharmD (84.4%) or a BS (5.2%) degree or were still successfully enrolled in a program (5.8%), and that only 4.5% (41) of the students had discontinued enrollment prior to the fourth year. Table 2 also shows that students who received a PharmD degree averaged the highest GPA for all 4 years (3.23, 3.21, 3.25, and 3.50, respectively). The mean GPAs earned by those who received a BS degree (2.88, 2.87, and 3.09, respectively) and those who still were enrolled after 4 years (2.62, 2.58, 2.76, and 2.88, respectively) were considerably lower than for students who received a PharmD in 4 years. Students who discontinued enrollment prior to the fourth year showed the lowest mean GPAs (2.57, 2.27, and 2.47, respectively).

Correlation Analysis

Table 4 shows that for the study sample (n = 899), correlations between GPAs in years 1-4 of the pharmacy programs and the predictor variables ranged as follows: PCAT subtest scaled scores, from 0.19 to 0.41; PCAT composite scaled scores, from 0.30 to 0.43; entering cumulative GPAs, from 0.38 to 0.49; and entering math/science GPAs, from 0.35 to 0.47. These data show that correlations were a little higher for entering GPAs than for PCAT scaled scores. For all independent variables, the correlation coefficients (r) generally decreased for each subsequent program year. While the correlation coefficients were somewhat larger for entering GPAs than for PCAT scaled scores, the proportion of variance in subsequent GPAs was not much more than for the PCAT composite scaled scores.

Table 4.

Correlations Between Predictors and GPAs in Years 1-4 of Professional Pharmacy Program

* Scaled score was used for this test. All correlations are statistically significant at 0.01. Entering cumulative and math/science GPAs were not reported for all students

GPAs = grade point averages; PCAT = Pharmacy College Admission Test

Regression Analysis

Tables 5 and 6 represent the results of multiple regression analyses for the study sample and show the degree to which combined independent variables predict specified outcome variables (R²) and the degree of contribution made by each independent variable (parameters). In Model 1 (Table 5), when PCAT subtest scaled scores and entering cumulative GPAs were considered together, they accounted for 37%, 34%, 30%, and 21% of the variance in first, second, third, and fourth-year GPAs, respectively. In Model 2 (Table 6), when PCAT subtest scaled scores were considered alone, they accounted for 24%, 21%, 19%, and 12% of the variance in first-, second-, third-, and fourth-year GPAs, respectively.

Table 5.

Multiple Regression Analyses for PCAT Subtest Scaled Scores and Entering Cumulative GPA (Model 1)

* No significant contribution by a predictor variable. All coefficients of determination are statistically significant at 0.01; all indicated parameters are significant at level 0.01

GPA=grade point average; SS= scaled score; quant.=quantatative; chem.= chemistry; cum.=cumulative

Table 6.

Multiple Regression Analyses for PCAT Subtest Scaled Scores (Model 2)

* No significant contribution by a predictor variable. All coefficients of determination are statistically significant at 0.01; all indicated parameters are significant at level 0.01

GPAs = grade point averages; PCAT = Pharmacy College Admission Test; SS = scaled score

Discriminant Analysis

The discriminant analyses conducted for this study involved weighting each variable according to its predictive contribution. For one analysis (Model 1), PCAT subtest scaled scores plus entering cumulative GPAs were used as predictor variables. For a second analysis (Model 2), only PCAT subtest scaled scores were used as predictor variables. Each analysis selected the best combination of sensitivity and specificity, which were defined as percentages of students correctly identified from among the lowest 5% (sensitivity) and highest 95% (specificity) of GPA earners in a pharmacy program. Tables 7 and 8 show the results of these analyses.

Table 7.

Discriminant Analyses for Weighted PCAT Subtest Scales Scores and Entering Cumulative GPA (Model 1)

GPAs = grade point averages; PCAT = Pharmacy College Admission Test

Table 8.

Discriminant Analyses for Weighted PCAT Subtest Scales Scores (Model 2)

Lowest 5% = percent receiving the lowest 5% GPAs in program (sensitivity); Highest 95% = percent receiving the highest 95% GPAs in program (specificity)

GPAs = grade point averages; PCAT = Pharmacy College Admission Test; SS = scaled score

The data for Model 1 (Table 7) show that when combined with entering cumulative GPAs, PCAT subtest scaled scores have predictive value in identifying between 83% and 65% of the lowest performing students and 75% to 62% of the highest performing students in years 1-4 of the professional pharmacy programs. The data for Model 2 show that when considered alone, PCAT subtest scaled scores have predictive value in identifying between 71% and 73% of the lowest performing students and 69% to 60% of the highest performing students in years 1-4. While in Model 1, the sensitivity predictions were higher for years 1-3 than in Model 2 (Table 8), and the specificity predictions were higher for all 4 years, Model 2 shows less variation from year to year when PCAT subtest scaled scores were considered alone. In Model 2, the sensitivity predictions were relatively consistent for each of the 4 years, and the specificity predictions decreased slightly for each year, but still remain relatively high.

Diagnostic Accuracy Analysis

As with the discriminant analyses, the diagnostic accuracy analysis conducted for this study also indicates sensitivity and specificity outcomes as percentages of the lowest 5% and highest 95% of performers, respectively. However, rather than using weighted subtest scores as predictor variables, the diagnostic accuracy analysis used PCAT composite scaled scores. As shown in Table 9, the diagnostic accuracy analysis lists a range of PCAT composite scaled scores and the corresponding percentages of sample examinees who represented the lowest and highest GPAs during each of the 4 years of the professional pharmacy programs that participated in the study. Table 9 lists both composite scaled scores based on the 1992 norms (the 2 columns on the far left) and the approximate equivalents based on the 2003 norms in effect since March 2004 (the 2 columns on the far right), which range from 200-600 with a mean of 400.²¹

Table 9.

Diagnostic Accuracy for Predicting GPAs in Professional Pharmacy Programs from Entering PCAT Composite Scaled Scores

Comp. SS Index = values added to the mean PCAT Composite scaled score for the study sample to determine the relationship between composite scaled scores and probabilities; SS = scaled score; PR = percentile rank; % of lowest 5% = percent of students receiving the lowest 5% GPAs in program (sensitivity); % Highest 95% = percent of students receiving the highest 95% GPAs in program (specificity). Scaled scores and percentile ranks include both those based on the 1992 norms in effect at the time the scores were earned (columns on left) and on the approximate equivalents based on the 2003 norms in effect since March 2004 (columns on far right)

DISCUSSION

The overall results of this study support previous research studies that have shown moderate predictive validity of the PCAT in terms of correlation analyses and multiple regression analyses. This study expands previous examinations of the predictive value of the PCAT by including discriminant and diagnostic accuracy analyses. The analysis with the most practical value is the diagnostic accuracy analysis because it suggests how well specific PCAT scaled scores can identify percentages of students who are likely to succeed in a professional pharmacy program.

From the data listed in Table 3, it is clear that the group of students earning the highest GPAs were those who went on to earn a PharmD degree in 4 years, followed by those who earned a BS and those who still were enrolled after 4 years. If success is defined as students who either received a degree or were still enrolled after 4 years and expected to eventually complete the degree, then 95% of the students who entered a professional pharmacy program in fall 2000 met this description.

However, if GPA in the pharmacy program is considered, there are clearly different degrees of success. Students who received a PharmD degree averaged the highest GPAs for all 4 years. The GPAs earned by those who received a BS degree and those who were still enrolled after 4 years were considerably lower than the PharmD students. Not surprisingly, students who discontinued enrollment prior to the fourth year had the lowest mean GPA. For this last group, it is notable that the largest proportion of students discontinued enrollment before the second year (nearly 60%). Apparently, the first year of pharmacy school is especially efficient at selecting those students who eventually will succeed from those who will not. This interpretation is significant when explaining each of the 4 predictive validity analyses conducted for this study.

The results of the correlation analysis shown in Table 4 are consistent with findings by other researchers, particularly as summarized in the meta-analysis by Kuncel and colleagues.¹ From these data, it seems clear that correlations are slightly higher for entering GPAs (both cumulative and math/science) than for PCAT scaled scores, and that for all independent variables, the correlation coefficients generally decrease for each subsequent program year.

Correlations may be higher for both entering GPAs than for PCAT scaled scores because the contents of a typical prepharmacy curriculum are more similar to a pharmacy curriculum than the contents of the PCAT subtests. Prepharmacy GPA is also earned over an extended period of time, while PCAT scaled scores are earned in a single testing session that is undoubtedly quite stressful for many individuals. In any case, the PCAT subtests are similar enough in content to prepharmacy curriculum that the correlations are not much less than those for both entering GPAs, especially as indicated by the PCAT composite scaled scores.

The decreasing correlations for each subsequent year of the pharmacy program may be influenced by some restriction of GPA range operative after the first year. Most students who discontinued enrollment did so before the second year (nearly 60%), resulting in narrower GPA ranges for students in years 2-4 than in the first year. The lower correlations in years 2-4 may reflect less variation in students' GPAs for those years. Nevertheless, the total number of students who discontinued enrollment is relatively small compared to the number who graduated. Moreover, the fourth year does not include any students who graduated with a BS degree, again a relatively small number (see Table 3). A more likely reason for the decreasing correlations between PCAT scaled scores and GPAs in each subsequent year may be due to the increasingly clinical nature of the program. In each subsequent year, the pharmacy programs required fewer credits in content areas assessed by the PCAT (see Table 2). Additionally, advanced pharmacy courses rely more heavily on knowledge acquired in prerequisite courses found earlier in the pharmacy curriculum than in courses taken prior to admission to a pharmacy program. From the multiple regression analyses shown in Tables 5 and 6, it is apparent that in Model 1 (Table 5), entering cumulative GPAs contribute more to subsequent GPAs than the PCAT chemistry, biology, and reading comprehension scaled scores, and that neither PCAT verbal ability scores nor quantitative ability scores contribute any significant predictive value. In Model 2 (Table 6), the same PCAT scaled scores make similar predictive contributions as in Model 1. However, the somewhat lower coefficients of determination (R2) in Model 2 suggest the value of considering entering cumulative GPAs in combination with PCAT scaled scores as seen in Model 1. The results of these analyses are consistent with findings by other researchers.^4-5,11,13

An explanation for why the PCAT verbal ability and quantitative ability subtests do not contribute significantly to the predictive validity in these 2 models may include the same causes as for the decreasing correlation coefficients over the 4 years in pharmacy programs. As seen in Tables 1 and 2, relatively few credits are required in the professional pharmacy curricula (or prepharmacy prerequisites) that seem to draw specifically upon the verbal and quantitative skills included in these 2 PCAT subtests. Furthermore, both multiple regression models suggest that PCAT scaled scores and entering cumulative GPAs, while valuable in helping predict subsequent performance, must always be considered along with other information about candidates when making admission decisions.

The data in Tables 7 and 8 show that when considered alone (Model 2, Table 8), PCAT scaled scores have predictive value in being able to identify between 71% and 73% of the lowest performing students and 69% to 60% of the highest performing students in years 1-4 of the professional pharmacy programs. Even though the specificity prediction decreases slightly for each year in the pharmacy program, the percentages of students earning the highest 95% of GPAs remain relatively high. Sensitivity predictions above 70% and specificity predictions above 60% suggest that the PCAT scaled scores alone have predictive value. When PCAT scaled scores are combined with entering cumulative GPAs (Model 1, Table 7), the predictive values are generally higher, consistent with the regression analyses. However, when considered alone, PCAT scaled scores show more consistent predictive values over years 1-4 than when combined with entering cumulative GPAs. These data support the correlation and multiple regression findings that PCAT scaled scores are generally valid predictors of subsequent performance.

While the discriminant analyses suggest predictive value in a theoretical sense, the diagnostic accuracy data shown in Table 9 have more practical value in suggesting specific PCAT composite scaled scores that can identify the students most likely to succeed in a pharmacy program. For example, using a PCAT composite scaled score of 198 (the 45th percentile) as a predictor variable would identify only 53% of the entering students among those earning the lowest 5% of GPAs during the first year in the program, but would identify 81% of those earning the highest 95% of GPAs during the same year. The opposite extreme is observed for a composite scaled score of 227 (the 90th percentile), which would predict 100% of the entering students among those earning the lowest 5% of GPAs during the first year in the program, but only 18% in the highest 95% GPA group during the same year.

The ideal use of these diagnostic accuracy data is to determine the optimal balance between sensitivity and specificity predictions based on a PCAT composite scaled score, where both outcome measures are similar in value and relatively high. For example, the data in Table 9 suggest that during the first year of a professional pharmacy program with a mean entering PCAT composite scaled score of 213 (75th percentile) as seen in the study sample, PCAT composite scaled scores in the 201-204 (52-58 PR) range correctly identify 69%-73% of candidates from among the lowest 5% of performers, and 75%-68% from among the highest 95% of performers. The similarity of these scaled scores to the optimal scores determined by the discriminant analysis as shown in Model 2 of Table 5 suggests a strong predictive value for the PCAT composite scaled scores.

The results of all 4 validity analyses suggest that PCAT scores are valid predictors of pharmacy student performance. While the results are most substantial for performance in the first year of study, the PCAT also is seen to have validity, though to a lesser degree, in predicting GPAs in professional years 2-4. Thus, this study suggests that the PCAT continues to have value in predicting the success of students in professional pharmacy programs.

Limitations

The results of this study should be interpreted with consideration for several necessary limitations. One limitation relates to the 11 colleges and schools of pharmacy that chose to participate in this study. Of the 11 colleges and schools of pharmacy that participated, 7 are located in the South, 2 are in the Midwest, and 2 are in the West. Furthermore, 8 of the participating colleges and schools are public institutions. These factors may limit the generalizability of the findings, especially for private schools and schools in regions outside the South. Another limitation relates to a study design that could not control for the differences among program admission and progression policies and curricula. For example, some programs allow students to repeat a course and allow the higher grade to count, while others require a student to repeat a course, but count only the lower grade or the average of the 2 grades together. In addition, the nature of the fourth-year GPA varies across schools. A typical fourth-year college or school of pharmacy curriculum consists of experiential courses (rotations, clerkships, and internships), with grades that are often pass/fail or credit/no credit. Letter grades assigned for these courses tend to be high. For these reasons, fourth-year GPAs may appear inflated. A final limitation relates to restriction of range. Because the recent trend among colleges and schools of pharmacy to tighten admission standards, there may be more variance in the sample used for this study (the fall 2000 entering class) than for a sample used in a study conducted at the present time. Furthermore, the cohort of students admitted in the fall of 2000 completed versions of the PCAT that were constructed on a test blueprint that has since been revised with consideration for more stringent prepharmacy requirements.

To ensure greater generalizability, future studies should attempt to enlist more schools , especially private schools and schools from the Northeast, Midwest, and West, and should attempt to control for the differences in program grading policies and curricula. Furthermore, future studies should utilize a cohort of students who completed a version of the PCAT that met the current test blueprint and who were admitted under more recent pharmacy school policies.

Regardless of the limitations of this study, the findings make a contribution to the existing literature on the predictive validity of the PCAT. This study not only supports the findings of previous studies that examined the predictive validity of the PCAT through correlation and regression analyses, but also includes 2 analyses not previously done in such studies: discriminant analysis and diagnostic accuracy analysis.

CONCLUSIONS

The results of this study suggest that the PCAT continues to have moderate to strong value in predicting GPAs during 4-year professional pharmacy programs, especially for the first year of study, and particularly when considered with a student's entering GPA. This study also found that specific PCAT scaled scores can be used to identify candidates for admission who have a high likelihood of success. When used in combination with other pertinent information about the candidates, the PCAT can be a valuable tool in selecting students likely to perform successfully in a professional pharmacy program.