Test Scores, Class Rank and College Performance: Lessons for Broadening Access and Promoting Success

Abstract

Using administrative data for five Texas universities that differ in selectivity, this study evaluates the relative influence of two key indicators for college success—high school class rank and standardized tests. Empirical results show that class rank is the superior predictor of college performance and that test score advantages do not insulate lower ranked students from academic underperformance. Using the UT-Austin campus as a test case, we conduct a simulation to evaluate the consequences of capping students admitted automatically using both achievement metrics. We find that using class rank to cap the number of students eligible for automatic admission would have roughly uniform impacts across high schools, but imposing a minimum test score threshold on all students would have highly unequal consequences by greatly reduce the admission eligibility of the highest performing students who attend poor high schools while not jeopardizing admissibility of students who attend affluent high schools. We discuss the implications of the Texas admissions experiment for higher education in Europe.

Introduction

During the last decade of the 20^th century, the baby boom echo created a “college squeeze” for American higher education because the demand for admission to selective institutions outpaced the expansion of the postsecondary system (Alon and Tienda, 2007). Rapid diversification of the college-age population, especially in high-immigration states like California, Texas and Florida, added a layer of complexity to the college squeeze because intensified competition for slots reignited debates about the legality of considering race and national origin of applicants in making admission decisions. Notwithstanding consensus that academic merit and talent should be the major criteria for allocating slots at selective institutions, there is considerable disagreement about how to define merit for purposes of admission (Alon and Tienda, 2007), particularly in the context of a highly stratified primary and secondary education system.

The college admission debate largely revolves around use of standardized test scores as the preferred measure of merit because minority and low-income students average lower scores than their white and affluent counterparts (Bowen and Bok, 1998; Clarke and Shore, 2001; Alon and Tienda, 2007). Advocates of standardized tests consider them a rigorous measure of academic preparedness that does not suffer from variation in grading standards across schools (Camara and Michaelides, 2005) and interpret the movement away from the SATs as the demise of meritocracy (Barro, 2001).

Nevertheless, several influential studies show that high school grade point average (GPA) is the best predictor of freshman grades and four-year collegiate GPA and college graduation (Crouse and Trusheim, 1988; Bowen and Bok, 1998; Rothstein, 2004; Geiser and Studley, 2002). Many studies also find that high school GPA has less adverse impact on admission prospects of economically disadvantaged students compared with the SAT because of its lower correlation with student socioeconomic characteristics compared with standardized tests. In fact, Rothstein (2004) argues that much of the SAT’s predictive power derives from its correlation with socioeconomic background and high school attributes.

Building on several recent insights showing that high school attributes influence college-going behavior, we use administrative data from several Texas universities to evaluate links between high school economic status quality, class rank and college performance. Specifically we address three questions that undergird the controversy about postsecondary admissions in the United States. First, is the predictive power of high school class rank and standardized test scores associated with high school economic status, and if so, in what ways? Second, how does the collegiate performance of high achieving students from high schools with low socioeconomic status compare with that of lower achieving students who attend high schools that largely serve high-income students? Third, what are the consequences of imposing a minimum admission test threshold for high-achieving students who attend high schools that serve low-income students? Combined, these questions engage the twin challenges of broadening access and maintaining high standards during a period of rising demand for postsecondary education.

Although race-sensitive admissions practices are considered an American response to issues of equity and access in higher education, the growth of international migration in Europe, including to countries that until recently were sending rather than host nations, has generated interest in the US experience with affirmative action policies in education as well as employment (see van Zanten, 2010; Trow, 1999). In countries with long immigration traditions, such as Germany and France, the large second generation has already begun to diversity enrollment in higher education, but in new immigration nations, such as Spain and Italy, implications of immigration for the diversification of higher education will depend on the contours of educational inequality at the lower grades. We return to these implications in the conclusion.

Texas Case Study

Texas is particularly well suited for evaluating the significance of high school socioeconomic status for collegiate academic performance because in 1997 the state legislature passed the uniform admission law (HB 588), which guarantees admission to any public post-secondary institution to all high school seniors who graduate in the top 10 percent of their class.¹ Because test scores are disregarded for rank-qualified graduates, the law eliminates the SAT filter that has served as an admission barrier to selective institutions to hundreds of qualified poor and minority students (Gerald and Haycock, 2006). This policy change provides a natural experiment that allows researchers to examine the performance of students whose admission does not require achieving a minimum threshold on standardized tests.

Important to the law’s success in broadening access is the provision that high schools, not colleges, determine how to compute class rank in order to qualify for the admission guarantee. This provision implies that students compete with their same-school classmates rather than students from other schools to qualify for automatic admission. As such, the Texas admission law levels the playing field between rich and poor high schools by using the same criterion to define the aristocracy of talent, namely graduating in the top decile of their class.

Initially the law was applauded as a race neutral alternative to affirmative action, but as demand for access to the selective institutions surged, opposition to the uniform admission law surged. One major criticism is that the law unfairly privileges high achieving students who attend underperforming high schools at the expense of allegedly better-qualified students from competitive high schools who graduate slightly below the cut point (Flores, 2003; Nissimov, 2000; Glater, 2004). Opponents also claim that the law eroded academic standards by pointing out that average test scores of top decile enrollees at UT-Austin fell below those of lower ranked students after the law went into effect (The University of Texas at Austin, 2008).

Critics of the uniform admission law assume that top 10% graduates from low performing high schools will underperform in college because they average low scores on their entrance exams. These criticisms are reminiscent of the arguments against the use of race-sensitive admission practices. In both instances, low standardized test scores are invoked to substantiate claims that students are ill prepared for college work. A key difference is that opponents of race-sensitive admission practices object to the use of ascribed individual attributes (i.e., race and national origin) in college admissions decisions; critics of the Texas uniform admission regime focus on schools, claiming that the law privileges top-performing graduates from low-performing schools over lower-ranked students who allegedly complete demanding academic curricula at competitive high schools.

Concerns about the erosion of standardized test scores among freshmen admitted to the public flagships were incorporated in the 2009 revision to the top 10% law. After two failed attempts to rescind or revise the legislation, the Texas legislature agreed to cap the number of students automatically admitted to the University of Texas at Austin (UT) at no more than 70 percent of the entering class.² In addition, the legislature imposed a minimum SAT threshold of 1,000 points for students who did not rank in the top 10% of their class, but fell short of requiring that students qualified for automatic admission also achieve the SAT minimum. Therefore, in the final section we simulate the implications of re-instating the SAT filter by applying the new minimum to top 10% enrollees. This exercise provides a stark lesson to college administrators who seek to maximize socioeconomic, geographic and ethno-racial diversity while also ensuring that enrollees succeed academically. We do not argue that standardized test scores are irrelevant for college admissions, but rather caution that their usefulness is more limited than that of high school class rank.

Data and Analytical Strategies

Our analyses are based on administrative data for five universities that were assembled as part of Texas Higher Education Opportunity Project. Table 1 profiles the five universities examined, which include UT-Austin (UT), Texas A&M (TAMU), Texas Tech (TECH), UT-San Antonio (UTSA) and Southern Methodist University (SMU). Collectively the institutions represent a considerable range in selectivity of admissions, public/private status, enrollment, and tuition sticker price. UT is the most selective among the universities compared, but according to Barron’s (2002), both SMU and TAMU have very competitive admissions. Six-year graduation rates, which range from 29 to 77 percent, vary directly with institutional selectivity.

Table 1.

Insitutional Chararcteristics (Fall 2002)

	UT-Austin	Texas A&M	Texas Tech	UT-San Antonio	SMU
Barron’s Selectivity	Highly Competitive	Very Competitive	Competitive	Non Competitive	Very Competitive
Public/Private Status	public	public	public	public	private
Freshman Enrollment	7,918	6,949	4,533	3,141	1,380
Total Enrollment	52,261	45,083	27,569	22,016	10,955
In-State Full Time Tuition	$4,527	$4,602	$4,001	$3,702	$19,466
6 Year Graduation Rate (2001 Cohort)	77%	77%	56%	29%	71%

Sources: Texas Higher Edcuation Coordinating Board, Institutional Reports.

http://www.thecb.state.tx.us/Reports

Southern Methodist University Common Data Set 2002–2003.

http://smu.edu/ir/CDS/Archive/cds2002.pdf

Barron’s College Profile for 2002.

Each university’s administrative data consists of an applicant file and term-specific transcript records for all enrollees. The applicant file contains basic demographic information, high school class rank, standardized test scores, admission and enrollment status, and graduation dates. Transcript files record several academic performance measures, including term-specific GPA and cumulative GPA for each semester enrolled. The analysis sample for each university is restricted to fall semester enrollees who graduated from a regular Texas public high school with at least 10 seniors.³ Using a database maintained by the Texas Education Agency (TEA), we append to each applicant record the percent of students ever economically disadvantaged at their high school, which we use to classify Texas public high schools into economic strata.

Outcome and Explanatory Variables

We examine four achievement outcomes available in the transcript files to portray college performance: freshman year cumulative GPA; four-year cumulative GPA; freshman year attrition; and four-year graduation rate. Freshman year attrition includes students who do not enroll for one or more semesters following fall matriculation. Four-year cumulative GPA and four-year graduation rates are considered only for those cohorts observed for four years after their first matriculation (that is, enrollees who are not right censored). Our theoretical discussion identified three key explanatory variables that we hypothesize are associated with collegiate performance: high school economic status; student class rank; and student standardized test scores. Each is described below.

High school economic strata

We classify regular Texas public high schools into three broad socioeconomic strata based on the share of students who ever received free or reduced lunch.⁴ High schools in the lowest quartile are designated affluent; those in the highest quartile are designated poor; and high schools in the middle quartiles are classified as average (or typical). Because the statewide share of economically disadvantaged students rose over time, we calculated the quartile cut-points for each year. Affluent schools are further sorted into two subgroups designating a subset of “feeder schools” with strong traditions of sending students to the public flagship institutions and others; similarly, poor schools are sorted into those designated “Longhorn or Century schools” versus other poor schools.

Operationally, feeder high schools are the top 20 high schools based on the absolute number of students admitted to UT and Texas A&M University (TAMU) as of 2000. Because of the considerable overlap between the two sets, the combined list of feeder schools represent only 28 high schools out of a possible 1,644 public high schools in 2000 (TEA, 2001).⁵ Most of the feeder high schools qualify as affluent based on criteria defined above, and none is poor.

An admission guarantee does not guarantee matriculation, particularly for low-income students. In order to raise the odds that high-achieving students would enroll at the flagships, both UT and TAMU targeted a subset of low income schools with low college-going traditions for aggressive outreach programs, offering “Longhorn” and “Century” scholarships to a few of their highest ranked graduates (Domina, 2007). UT’s Longhorn Opportunity Scholarship program began in 1999 with approximately 40 high schools and expanded to 60 during the early years of the top 10% admission regime. TAMU launched the Century program in 1999 with 20 participating high schools and added new schools in 2000 and 2001, reaching about 50 in 2003. The Longhorn and Century high schools are mostly non-overlapping sets, but 28 high schools participate in both programs. In this paper, schools ever designated for the Longhorn/Century program are coded consistently throughout the observation period.⁶

Table 2, which provides a snapshot of the five high school strata, shows the pervasiveness of ethnic and racial school segregation and its close correspondence with poverty.⁷ About three-in-four students from feeder and affluent high schools are white, compared with only 10 to 15 percent of students from poor high schools. Blacks represent less than 10 percent of the student body at affluent and feeder high schools and Hispanics comprise around 12 percent. At poor and Longhorn/Century high schools black and Hispanic students are dominant majorities—84 and 88 percent, respectively. Over two-thirds of students from these schools are economically disadvantaged.

Table 2.

Selected Characteristics of Texas Public High Schools in 2002

	Feeder	Affluent	Average	Poor	Longhorn/Century
N	27	240	514	206	89
Total seniorsa	603	242	151	135	278
(s.d.)	(174)	(201)	(152)	(139)	(125)
Race/Ethnicity Compositionb
% Black	8	8	14	9	30
% Hispanic	11	13	27	75	58
%White	72	74	56	15	10
% Asian	10	4	3	1	2
% Students ever economically disadvantagedb	9	12	33	70	63
% taking college entry examb	83	71	61	53	51
Average SATc	1094	1007	980	894	842
Average ACTd	23	21	20	18	17

Source: Texas Education Agency (TEA).

Note:

^aInclude only regular public high schools with at least 10 seniors.

^bResults weighted by class size.

^cMissing for 16% of average high schools and 33% of poor high schools.

^dMissing for 16% of poor high schools.

The high school strata also represent considerable variation in college-orientation, as evident by the share of students who take college entrance exams. Over 80 percent of students from feeder schools take entrance exams, and average a score of 1100 (out of a possible 1600 points); by comparison, only half of students attending Longhorn/Century high schools take a college entrance exam and their average score is only 850. This large disparity in test scores fuels criticism about the admissibility of students from poor high schools.

High school class rank

Under the provisions of the uniform admission law, high school administrators have great latitude in determining how to calculate grade point averages for purposes of generating a rank distribution.⁸ High schools report the size of their senior class and exact class standing. We sort students into three categories based on their rank: top decile, second decile, and third decile or below.⁹

Test scores

Although standardized test scores are not considered in the admission decisions of students who qualify for automatic admission, all applicants must submit results of college entrance exams, either SAT or ACT, in order for an application to be considered complete. ACT scores are converted to SAT scores based on a conversion table published by College Board, and SAT scores are re-centered for years prior to 1996.

Summary Statistics

Table 3 reports summary statistics for enrollees at each of the five Texas universities and indicates the period for which data are available. With the exception of SMU, our data span the period before and after the uniform admission law was in force. SMU, a very selective private institution, is not bound by the admission guarantee and considers test scores of all applicants, irrespective of class rank, in admission decisions.

Table 3.

Summary Statistics: Enrollees from Texas Public High Schools

	UT-Austin	Texas A&M	Texas Tech	UT-San Antonio	SMU
Years Avaliable	1990–2003	1992–2002	1991–2003	1990–2003	1998–2004
N	75,541	58,341	28,029	25,091	3,620
High School Class Rank (Col. %)
Top Decile	50	52	23	15	37
Second Decile	24	26	21	19	22
Third Decile or Lower	26	22	56	66	41
Test Score Means	1189	1152	1087 a	977 a	1162
(S.D.)	(147)	(139)	(139)	(145)	(152)
High School Economic Strata (Col. %)
Feeder	27	19	14	8	20
Affluent	34	37	40	34	43
Average	26	31	36	27	24
Poor	7	8	6	18	2
Longhorn/Century	4	3	2	9	8
Missing	2	2	2	2	2
College Performance
Freshman Year CGPA	2.94	2.78	2.92	2.19	3.04
4th Year CGPA	2.99	2.98	3.04	2.48	3.13
Freshman Year Attrition	11%	9%	13%	34% b	11% c
Graduated in 4 Years	33%	33%	25%	4%	52%

Source: Texas Higher Education Project (THEOP) administrative data.

Note:

^a1996+ only.

^b1990–2002 only, sophomore year fall semester data not available for 2003 cohort.

^c1998–2003 only, sophomore year fall semester data not available for 2004 cohort.

The institutional enrollee pools correspond well to Barron’s selectivity rankings in that the highest average test scores correspond to UT (nearly 1200), SMU and TAMU, and the lowest to UTSA, TECH falls between the high and low values. About half of first-time freshman at the two public flagships graduated in the top decile of their class, compared with less than a quarter of TECH students, about one-in-seven UTSA students, and over one-third of SMU students. Both public flagships and SMU draw at least one-fifth of their first-time freshmen from feeder high schools, but only 11 percent from poor high schools. By comparison, over one-quarter of UTSA enrollees graduate from high schools that serve large numbers of economically disadvantaged students, but only eight percent are feeder high school graduates.

Academic performance of enrollees at the two flagships, Texas Tech and SMU is more or less at par, especially after the freshman year, but SMU enrollees enjoy much higher four-year graduation rates—52 percent versus 33 percent for both UT and TAMU. UTSA has the most dismal record based on all outcome measures. About one-third of first-time freshmen that enroll at UTSA withdraw during or following their freshman year and a meager four percent graduates in four years. Differences in the composition of the enrollee pools have direct implications for students’ academic performance, and in particular for claims about the value of testing and ranking for predicting collegiate academic success, which we address next.

Analytical Strategies

First, to evaluate claims that high school economic status mediates the influence of test scores and class rank on collegiate achievement, we use OLS and probit regression techniques to predict the four college performance measures as a function of the three covariates of interest: high school economic strata, students’ high school class rank and students’ standardized test scores. Based on R-Squares and pseudo R-Squares from three baseline specifications and three nested interaction specifications, we decompose the components of variance due to each of the three predictors. Sequentially, the empirical specifications include:

1. School economic strata (five discrete categories);

2. School class rank (three discrete categories);

3. Individual standardized test scores;

4. School economic strata + class rank + (economic strata * class rank);

5. School economic strata + test scores + (economic strata * test scores);

6. School economic strata + class rank + test scores + (economic strata * class rank) + (economic strata * test scores).

All models include year dummies to monitor changes in covariates that may vary systematically over time. The R-Square and pseudo R-Square statistics for first three specifications reveal the overall predictive power of high school economic status, high school class rank and individual test scores; the 4^th and 5^th specifications indicate whether and to what extent high school economic status is conflated with the two high school achievement metrics, and the final specification considers the joint explanatory power of the three key predictors. Institution-specific analyses reveal whether the strength of the associations depends on the admissions selectivity and public/private status of the universities.

Second, to address claims that highly-ranked students with low test scores underperform academically relative to low rank students who scored high on standardized tests, we compare the four college performance measures for non-top decile students from feeder, affluent and average high schools with those of graduates from Longhorn/Century high schools who qualified for automatic admission, and also examine whether performance gaps depend on institutional selectivity. Whether the conditional association between test scores and class rank also depends on high school economic status or institutional selectivity is an empirical question for which there is no prior evidence.

Third, to simulate the consequences of extending the minimum SAT threshold to students qualified for automatic admission based on their class rank, we focus on the most selective public institution, the University of Texas at Austin because it has become saturated with students qualified for the admission guarantee; because it is the target of criticisms about declining student quality based on the erosion of standardized test scores; and because the recent cap on the share of students admitted automatically only affects UT. Operationally we compare four college performance measures by restricting the sample of non-top 10% students from feeder, affluent, average, and poor high schools who achieved a minimum SAT equivalent score of 1,000 relative to top decile Longhorn/Century high school students whose test scores fell below the threshold. Finally, to further buttress our findings that test score advantages do not insulate low ranked students from academic underperformance, we use Kernel density estimation to examine the entire distribution of college performance.

Predictive Power of High School Class Rank and Test Scores

Table 4 reports the gross predictive power of each covariate (Models 1–3) for each of the four college performance measures based on the R-Square and pseudo R-Square statistics. Three main findings emerge. First, consistent with other studies, high school class rank is an equivalent or better predictor of college performance than standardized tests. The second and the third columns show that the percent of variance in four college performance measures accounted for by high school class rank is comparable to or higher than that attributable to test scores, even using an aggregated, categorical metric for high school class rank. The only exception to this generalization is the model predicting 4^th year cumulative GPA for SMU enrollees. If percentile class rank is modeled as a continuous measure, the corresponding statistics increase slightly, rendering our estimates for class rank conservative.

Table 4. College Perfomance Variation Explained by High School Outcomes.

(R-sq and Pseudo R-sq)

Models	(1)	(2)	(3)	(4)	(5)	(6)
Predictors	High School Economic Strata	High School Class Rank	Test Score	High school Economic Strata and Class Rank	High School Economic Strata and Test Score	High School Economic Strata, Class Rank and Test Score
UT-Austin
Freshman Year CGPA	0.07	0.16	0.17	0.24	0.17	0.29
4th Year CGPA	0.04	0.14	0.13	0.21	0.14	0.25
Freshman Year Attrition	0.014	0.026	0.019	0.045	0.022	0.047
Graduated in 4 Years	0.021	0.034	0.028	0.060	0.033	0.063
Texas A&M
Freshman Year CGPA	0.03	0.14	0.15	0.20	0.16	0.26
4th Year CGPA	0.02	0.14	0.13	0.19	0.14	0.24
Freshman Year Attrition	0.007	0.019	0.012	0.032	0.015	0.035
Graduated in 4 Years	0.014	0.034	0.017	0.045	0.020	0.046
Texas Tech
Freshman Year CGPA	0.01	0.19	0.10	0.22	0.11	0.24
4th Year CGPA	0.01	0.20	0.13	0.22	0.14	0.25
Freshman Year Attrition	0.003	0.015	0.008	0.021	0.011	0.022
Graduated in 4 Years	0.006	0.030	0.012	0.041	0.014	0.042
UT-San Antonio
Freshman Year CGPA	0.04	0.11	0.10	0.18	0.11	0.21
4th Year CGPA	0.01	0.11	0.08	0.16	0.08	0.18
Freshman Year Attrition	0.007	0.016	0.002	0.021	0.004	0.017
Graduated in 4 Years	0.022	0.047	0.034	0.071	0.040	0.076
SMU
Freshman Year CGPA	0.03	0.21	0.21	0.28	0.21	0.33
4th Year CGPA	0.04	0.18	0.23	0.30	0.23	0.36
Freshman Year Attrition	0.008	0.020	0.010	0.030	0.014	0.032
Graduated in 4 Years	0.023	0.028	0.026	0.051	0.033	0.052

Source: Texas Higher Education Project (THEOP) administrative data.

Note: model specifications: 4. School economic strata + class rank + (economic strata * class rank); 5. School economic strata + test scores + (economic strata * test scores); 6. School economic strata + class rank + test scores + (economic strata * class rank) + (economic strata * test scores).

Second, results support Rothstein’s (2004) claim that much of the predictive power of standardized test scores derives from their correlation with socioeconomic background and high school attributes. The influence of test scores on college performance is conflated with high school economic status, but this is not the case for high school class rank because all high schools—rich or poor—have the same rank distribution. Test scores are highly correlated with the economic status of high schools, therefore the R-Square statistics from jointly modeling high school economic status with students’ test scores are virtually identical to those based on test scores alone (cols 3 and 5). By contrast, when high school economic status and high school class rank are modeled jointly, the associated R-Square statistics are substantially higher than those from the specifications that only include high school class rank (cols. 2 and 4). Substantively this indicates that high school economic status has explanatory power that is independent of students’ class rank. Inclusion of standardized test scores yields modest improvements in predicting college achievement beyond that attributable to high school economic status and high school class rank (see cols 4 and 6). Adding high school class rank to a model that includes both high school economic strata and test scores significantly improves predictions of college outcomes, almost doubling R-Square values (cols 5 and 6).

Importantly, the two main findings also obtain for all four postsecondary outcomes; for all selectivity tiers; and for both public and private institutions. That our estimates are robust to variation in institutional selectivity challenges claims that high-performing students from low-performing high schools are ill prepared for college work, even at the most selective institutions. This key premise underlying criticisms of both affirmative action policies nationally, and percentage plans such as that used in Texas, Florida and California, warrants consideration by policy analysts and admissions officers seeking to broaden access to post-secondary education. In particular, the designation of minimum thresholds for standardized tests, such as that recently proposed by Texas, can significantly narrow the talent pool if applied without regard to high school grades and other measures of academic success, as the next sections show.

Test Score Advantages and Academic Performance

We compare the academic performance of top decile graduates from Longhorn/Century high schools with that of students from affluent and average high schools who do not rank in the top decile of their class but average higher SAT scores. These comparisons are designed to maximize contrasts because graduates from Longhorn and Century high schools are alleged to be least well prepared for college both because they serve large poor and minority student bodies and because of their low college-going traditions.¹⁰ Some recipients of Longhorn and Century schools receive additional academic and social support at the flagships, but only the very top-ranked among students eligible for the admission guarantee are awarded fellowships (Domina, 2007). Because graduates from Longhorn and Century high schools who enroll at other institutions do not enjoy targeted program support, Texas Tech, UTSA and SMU serve as important comparisons for these analyses.

Table 5 reports simple mean differences and associated t-test statistics. Relative to students who graduated in the top decile of a Longhorn or Century high school, Texas college students who graduated in the second decile of an average high school, and those who graduated in the third decile or below from feeder and affluent high schools, underperform academically (col. 2–5) despite their higher test scores (col. 1). Importantly, this generalization obtains across the spectrum of institutional selectivity.

Table 5.

Academic Performance of Enrollees: Deviations from Top Decile Longhorn/Century School Students

	Testscore Advantage	Freshman Year CGPA	4 Year CGPA	Freshman Year Attrition	Graduated in 4 Years
2nd Decile Feeder
UT-Austin	183 ***	0.36 ***	0.29 ***	−0.06 ***	0.20 ***
Texas A&M	78 ***	0.22 ***	0.17 ***	−0.04 ***	0.10 ***
Texas Tech	110 ***	0.33 ***	0.12 *	−0.12 ***	0.15 **
UT-San Antonio	158 ***	0.73 ***	0.60 ***	0.00	0.08 ***
SMU	167 ***	0.31 ***	0.42 ***	0.05	0.24 ***
2nd decile Affluent
UT-Austin	117 ***	0.00	0.07 ***	−0.01	0.07 ***
Texas A&M	33 ***	−0.03 †	0.00	0.00	0.05 ***
Texas Tech	42 ***	0.07	−0.10 *†	−0.11 ***	0.11 **
UT-San Antonio	130 ***	0.38 ***	0.20 ***	−0.02	0.04 ***
SMU	100 ***	0.20 ***	0.25 ***	−0.04	0.24 ***
2nd Decile Average
UT-Austin	85 ***	−0.21 ***	−0.06 **	0.03 ***	−0.01
Texas A&M	−10 *	−0.23 ***	−0.13 ***	0.03 **	−0.01
Texas Tech	16	−0.07	−0.17 **	−0.10 ***	0.04
UT-San Antonio	73 ***	0.05	0.01	0.01	0.01
SMU	32 *	−0.11 †	0.03	−0.01	0.07
3rd Decile or Lower Feeder
UT-Austin	125 ***	−0.09 ***	−0.03 †	0.01	0.05 ***
Texas A&M	47 ***	−0.11 ***	−0.09 ***	0.01	−0.01
Texas Tech	28 **	−0.22 ***	−0.32 ***	−0.08 **	0.04
UT-San Antonio	94 ***	0.04	−0.15 **	0.14 ***	0.00
SMU	88 ***	−0.08	0.06	0.02	0.12 *
3rd Decile or Lower Affluent
UT-Austin	82 ***	−0.31 ***	−0.16 ***	0.05 ***	−0.04 **
Texas A&M	11 **	−0.27 ***	−0.20 ***	0.04 ***	−0.06 ***
Texas Tech	−16	−0.32 ***	−0.40 ***	−0.06 *	−0.01
UT-San Antonio	48 ***	−0.27 ***	−0.20 ***	0.08 ***	0.00
SMU	34 **	−0.20 ***	−0.08	0.01	0.08
3rd Decile or Lower Average
UT-Austin	55 ***	−0.47 ***	−0.26 ***	0.10 ***	−0.14 ***
Texas A&M	−33 ***	−0.42 ***	−0.30 ***	0.08 ***	−0.09 ***
Texas Tech	−44 ***	−0.43 ***	−0.42 ***	−0.03	−0.05
UT-San Antonio	6	−0.44 ***	−0.31 ***	0.12 ***	−0.01 †
SMU	−2	−0.48 ***	−0.25 ***	0.05	−0.05

Source: Texas Higher Education Project (THEOP) administrative data.

Note:

^***p<0.001,

^**p<0.01,

^*p<0.05,

^†p<0.10

For example, at “highly competitive” UT-Austin, second decile graduates from average high schools enroll with an 85-point test score advantage compared with top decile Longhorn school students, yet earn lower freshman year and 4^th year cumulative GPAs (0.21 and 0.06 points less, respectively); moreover, they are about three percentage points more likely to drop out during or after their freshman year. Conditional on remaining enrolled, however, they are about equally likely to graduate in four years. At UT-San Antonio, second decile students who attended average public high schools enroll with a 76-point test score advantage, yet their academic performance is comparable to that of top decile Longhorn/Century school students.¹¹ SMU enrollees who graduated in the second decile of an average Texas public high school average a 32-point test score advantage over top decile Longhorn/Century school enrollees. They too average lower freshman year cumulative GPAs, but perform equally well on the other three achievement outcomes.

Two additional considerations warrant discussion. First, although the reported mean differences average out year over year fluctuations, it is possible that an abnormality in one or two years, such as the year before the admission guarantee was in force and affirmative action was banned (1997), would skew an average result. To further verify the robustness of our findings, we compare mean differences using annual data. Although we observe fluctuation in the annual performance differences, the general pattern holds.¹²

Second, we investigated whether the college performance advantages corresponding to top-ranked Longhorn/Century school enrollees reflects their choice of easier academic fields of study. At all five universities, top 10% students from Longhorn/Century high schools are more likely to major in natural science, engineering and computer science compared with students from feeder, affluent and average high schools ranked at or below the second decile of their high school class.¹³ For example, at UT-Austin, over one-quarter of top decile Longhorn school students chose these majors, which is comparable to or higher than the share of second decile students from feeder, affluent and average high schools. Among students who graduated at or below the third decile from these three types of public high schools, one-in-five majored in these fields.

Because opponents of affirmative action and percentage plans use test scores as the primary rationale for opposing these admission policies, our discussion emphasizes comparisons between top decile graduates from Longhorn/Century schools and students from affluent and feeder high school ranked at or below the third decile in order to illustrate that test score advantages do not predict college success. When compared with second decile students from feeder and affluent high schools, Longhorn/Century school students who qualified for automatic admission achieve lower test scores and exhibit weaker collegiate performance based on all four outcomes. This finding demonstrates that top decile class rank cannot fully compensate for unequal academic preparedness associated with attending a low performing-high school.

Minimum SAT Threshold: A Cautionary Tale

Despite extensive evidence that grades are more reliable predictors of college success than test scores, the admissions officers have raised the relative weight assigned to entrance examinations for rationing slots in US higher education (Alon and Tienda, 2007). In Texas, this preoccupation is evident in the recent amendment of the top 10% law, which instated a SAT score minimum of 1,000 points applicants ineligible for automatic admission. Although the test score filter is not used to screen out applicants qualified for automatic admission, we show that many highly ranked students from poor high schools would not be admitted to UT-Austin if the minimum threshold were required of them.

A detailed examination of the cumulative class rank and test score distributions for enrollees automatically admitted to UT in 2003 (left panel of Figure 1 shows that the class rank distributions of automatically admitted students do not differ appreciably according to their high school’s economic status; however, automatically admitted Longhorn students are more concentrated at the lower percentiles (i.e., better ranks) of the top decile compared with rank counterparts from feeder schools. The mean percentile class rank for top decile students is nearly five for feeder and affluent school students, compared with 4.2 and 3.9 for students from poor and Longhorn schools, respectively.

Figure 1. UT-Austin Top Decile Enrollees in 2003: Distributions by High School Economic Status.

Source: Texas Higher Education Project (THEOP) administrative data.

The right panel of Figure 1, which displays percentages of students with test scores below 1,000 points, reveals large differences among the high school economic strata. ¹⁴ Among students ranked from the 1^st to the 10^th percentile of their class, less than one percent of feeder school students score below 1,000 points on the SAT and approximately three percent of similarly ranked graduates from affluent schools score 1,000 points or less. By contrast about 20 percent of Longhorn school students who graduated in the 1^st percentile of their class score below 1000, and this share rises to about 40 percent for the 5^th percentile students. Nearly half (45 percent) of Longhorn school graduates ranked in the 10^th percentile achieved test scores below 1,000 points.

Given the growing saturation of the top decile students at UT since the law was implemented, capping automatic admissions became necessary in order to prevent the institution from exceeding its carrying capacity. Figure 1 indicates that capping based on high school class rank would have roughly uniform consequences across high schools; however, capping based on test scores would greatly reduce the shares of highly ranked students from Longhorn schools while leaving students from feeder and affluent high schools unaffected. Because the number of top decile enrollees at UT differs greatly by high school economic status, releasing automatic admission slots by capping based on high school class rank is a more efficient and equitable policy lever than capping based on test scores, which is what the test score minimum does.

Table 6 reports the number of top decile enrollees at UT in 2003, the number of students with test scores below 1000, and the number of students ranked in the upper 9^th and 10^th percentiles of the class rank distribution. Capping automatic admissions using the 8^th percentile rather than the 10^th percentile as the cut-point would free 622 slots, but imposing a 1,000-point test score minimum as a filter would not only free fewer (N=428) slots, but also would toll disproportionately on students from Longhorn and poor schools. Stated differently, almost two-thirds ([139 + 131]/428 = 0.63) of “released” seats would come from poor and Longhorn schools if the test score filter were imposed to screen applicants versus 14 percent ([55+31]/622 = 0.14) of their current shares based on class rank.

Table 6.

Number of Top 10% Enrollees at UT-Austin, 2003

	Top decile students	Test Score Less than 1000		Top 9th and 10th percentile class rank
	N	N	%	N	%
Feeder	658	3	0.5	109	16.6
Affluent	1385	45	3.3	253	18.3
Average	1186	110	9.3	174	14.7
Poor	472	139	29.5	55	11.7
Longhorn	282	131	46.5	31	11.0
Total	3983	428	10.8	622	15.6

Source: Texas Higher Education Project (THEOP) administrative data.

Lastly, we examine the consequences on college performance of imposing the 1,000 SAT point minimum as an exclusion restriction for admission. Under this scenario, at least 80 percent of enrollees who did not graduate in the top decile of their class at feeder, affluent and average high schools would still be admissible, but 30 to 45 percent of top decile Longhorn school students would be rendered inadmissible. Table 7 presents average test scores and college performance differences for students who do and who do not meet the minimum threshold. For this exercise, top 10% graduates from Longhorn high schools with test scores below 1,000 points—the hypothetical inadmissible group—serve as the reference group, which is compared with students who meet the test score threshold but do not rank in the top 10 percent of their class. We find numerous instances of collegiate underperformance among “test-eligible” enrollees relative to top decile students from Longhorn high schools with lower test scores.

Table 7.

Academic Performance of UT-Austin Enrollees with Testscore ≥1000: Deviations from Top Decile Longhorn School Students with Testscore <1000

	Test score	Testscore Advantage	Freshman Year CGPA	4 Year CGPA	Freshman Year Attrition	Graduated in 4 Years
2nd Decile Feeder	1245	344 ***	0.54 ***	0.51 ***	−0.09 ***	0.28 ***
2nd Decile Affluent	1190	289 ***	0.19 ***	0.29 ***	−0.03 **	0.16 ***
2nd Decile Average	1169	268 ***	−0.01	0.17 ***	0.01	0.07 ***
3rd Decile or Lower Feeder	1202	301 ***	0.09 ***	0.19 ***	−0.02	0.13 ***
3rd Decile or Lower Affluent	1173	272 ***	−0.12 ***	0.08 **	0.03 *	0.05 *
3rd Decile or Lower Average	1166	265 ***	−0.27 ***	−0.01	0.08 ***	−0.03

Source: Texas Higher Education Project (THEOP) administrative data.

Note:

^***p<0.001,

^**p<0.01,

^*p<0.05,

^†p<0.10

For example, UT enrollees from average high schools ranked at or below the third decile who meet the SAT minimum threshold average a test score of 1,166. Despite their 265-point advantage relative to top decile Longhorn school students with test scores below the threshold, they fare significantly worse academically in their freshmen year, earning a 0.21 point lower cumulative GPA. Moreover, as a group, they are five percentage points more likely to withdraw before their sophomore year. Even after excluding dropouts, they achieve a comparable, but not superior, four-year cumulative GPA and four-year graduation rate. These results strongly caution against imposing the 1,000-point test score requirement for top-performing students from underperforming high schools, which is tantamount to imposing a penalty for family background.

College Performance: Beyond Averages

To further support our claims about the predictive power of standardized tests and high school grades, we examine the entire distribution of college performance, which goes beyond simple mean comparisons reported above. Figures 2 and 3 display Kernel density estimates, which are essentially smoothed histograms of freshman and 4^th year cumulative GPA’s earned by UT-Austin enrollees in 2000. This is the latest cohort for which 4^th year cumulative GPA is available in our data.

Figure 2. Kernel Density Estimation for Groups Defined by High School Class Rank, High School Economic Status, UT-Austin Enrollees in 2000.

Statistics of Kolmogorov-Smirnov test for equality of distribution functions are in parentheses,

Reference group is top decile Longhorn school students

Figure 3. Kernel Density Estimation for Groups Defined by High School Class Rank, High School Economic Status and Test Score, UT-Austin Enrollees in 2000.

Statistics of Kolmogorov-Smirnov test for equality of distribution functions are in parentheses,

Reference group is top decile Longhorn school students with test score less than 1000 points

Top two graphs in Figure 2 compare the freshman cumulative GPA distribution of top decile Longhorn school students with those of lower-ranked graduates from feeder, affluent and average high schools. Kolmogorov-Smirnov tests for equality of the distribution functions indicate that the distributions do not differ statistically between top decile Longhorn school students and second decile students from affluent high schools. However, the distributions for second decile graduates from average schools, and all students ranked at the third decile or lower not only differ from that of top decile Longhorn school students, but also contain lower GPA values. For the bottom two graphs in Figure 2, which compare 4-year cumulative GPA distributions, the Kolmogorov-Smirnov test results are similar to those based on the freshman year GPA distributions with one exception – feeder school students ranked at the third decile or lower have a statistically similar grade distribution as top decile Longhorn school students. This reveals that the performance gap narrows over the college career.

Graphs in Figure 3 compare freshman and 4^th year cumulative GPA distribution of the subgroups using the 1000-point test score minimum as a screen. The two upper graphs portray freshman year cumulative GPA distributions for top decile Longhorn school students with test scores below 1000-point threshold and for lower-ranked students from feeder, affluent and average high schools who meet the minimum test score requirement. Kolmogorov-Smirnov tests for equality of distribution functions indicate no statistical differences between top decile Longhorn school students with sub-par test scores and the three groups that score at or above the 1,000 point threshold, namely second decile graduates from affluent and average high schools and feeder school students ranked at or below the third decile.

The grade distributions for affluent and average school students ranked at the third decile or lower do differ from those of top 10% Longhorn school students, however, and they also include lower GPA values. Over their college careers, these lower rank students from feeder, affluent and average high schools improve their college performance. Kolmogorov-Smirnov tests cannot reject the hypothesis that students from affluent schools who ranked at or below third decile and also meet the test score threshold have statistically identical distributions for 4^th year cumulative GPA as top decile Longhorn school students who do not meet the test score threshold.

The Kernel density estimates are entirely consistent with findings based on mean comparisons. More importantly, despite statistical differences for some comparisons of paired GPA distributions, the Kernel density estimates reveal remarkable overlap among the curves. For example, the freshman year cumulative GPA distribution for top decile Longhorn high schools differs significantly and contains lower GPA values than that of second decile feeder school students. Yet, about 38 percent of the former earn a freshman cumulative GPA better than 3.3—the mean of the latter group; and about 47 percent of the latter group earn a freshman year cumulative GPA below 3.0—the mean of the former group.

Even for the test-score subgroups that fall above and below the 1000-point threshold, Kernel density estimates show considerable overlap in the respective cumulative GPA distributions. For instance, the 4^th year cumulative GPA distribution for top decile Longhorn school students with test scores below the 1,000-point threshold differs significantly and contains lower GPA values than that corresponding to feeder school students ranked at the third decile or below who meet the test score threshold. Given the 300-point test score difference, it is remarkable that about 59 percent of the Longhorn students earn a four-year cumulative GPA better than 2.9, which is the mean achieved by feeder school students; moreover, about 20 percent of the feeder school graduates earn a four-year cumulative GPA below 2.7, which corresponds to the mean GPA of Longhorn school students. Substantively this indicates that test score advantages and competitive high school attendance do not ensure college success for students who earn average grades in high school. Yet most top 10% graduates from Longhorn schools who enroll at UT-Austin rise to the bar set before them, including those with test scores below 1000 points.

Conclusion and Discussion

This study evaluates the predictive power of two key indicators used by college admissions officers to predict college success. Our analysis is unique in its use of administrative data for institutions that differ in the selectivity of their admissions as well as the economic status of high schools attended by enrollees. The empirical analyses warrant three major conclusions. First, consistent with many other studies, we affirm that high school class rank is a better predictor of college performance than standardized test scores. This conclusion holds for all four measures of college performance and across selectivity tiers and public/private status of universities. Considered alone, high school class rank predicts college success as well as or better than test scores; unlike class rank, however, the predictive power of test scores is conflated with the economic composition of high schools. This follows because every high school, irrespective of size or economic status, has a full class rank distribution; however, the test score distribution, which is normal for a national population, is truncated at low performing schools. As such, our finding that rank is an equivalent or better predictor of college success is all the more remarkable.

Second, at all universities considered, test score advantages do not insulate low rank students from academic underperformance. Relative to enrollees who graduated in the top decile from Longhorn or Century high schools, college students ranked in the second decile of average high schools, or the third decile and below from feeder and affluent high schools matriculate with substantial test score advantages, yet perform academically about the same or worse in college.

Third, a large share of top decile Longhorn school students and few non-top decile students from feeder, affluent and average high schools score below 1,000 points on their college board exams, which is a new minimum threshold imposed for applicants to UT who do not qualify for automatic admission. Simulations reveal that capping automatic admits based on high school class rank would have roughly uniform impacts across schools that differ in their economic status, but imposing minimum admission thresholds based on test scores would greatly reduce the admission eligibility of the highest performing students from poor high schools with low college going traditions while not jeopardizing the admission eligibility of graduates from feeder and affluent high schools. Yet, on average, top decile Longhorn school students who score below the 1,000-point threshold perform better than third decile or lower rank students from average high schools who score at least 1,000 points.

It is true that top decile Longhorn/Century school students achieve lower test scores and lower performance in college when compared with second decile students from feeder and affluent high schools. This fact underscores that a top decile class rank cannot fully compensate for weaker academic preparation associated with attending low performing high schools. Nevertheless, our analyses show that arguments about their propensity to fail based on their lower test scores have been greatly exaggerated. In situations where high school class rank and test score provide strongly conflicting evaluations of students’ academic excellence and readiness for college work, high school class rank is a more reliable predictor of college success than standardized test scores.

Implications of our results transcend Texas and the United States and are also germane for nations) with highly stratified primary and secondary education systems. In the United States disparities in the quality of primary and secondary schooling are buttressed by residential segregation along class and racial lines. Europe’s success integrating new immigrants and providing equal opportunity to the second generation will surely define the contours of educational inequality at all levels. As a nation with a long immigration tradition, for example, France has developed programs to increase postsecondary participation to students from disadvantaged family background (van Zanten, 2010). However, many European institutions are still adapting to the surge in college enrollments and currently confront additional fiscal and administrative pressures that have limited progress toward the goal of mass higher education (Trow, 1999). Whether the diversification of the college-eligible population accentuates these pressures will depend on not only on expansion of the postsecondary system, but importantly on prevention of instructional inequities at the primary and secondary levels.