An empirical evaluation of Chinese college admissions reforms through a natural experiment

Significance

While college admission decisions impact the educational experiences and labor market outcomes for millions of students each year, the best method to determine admissions has been vigorously debated by academics and policymakers across the globe. Driving this debate are a number of major theoretical and practical innovations over the past 2 decades. Using a natural experiment from China, we evaluate the performance of the immediate acceptance mechanism, used for many years for college admissions in China and school choice in the United States, and the newer parallel mechanism. We find that when provinces move to the parallel mechanism, students apply to more colleges and list more prestigious colleges first. We further find that the student–college matchings become more stable.

Abstract

College admissions policies affect the educational experiences and labor market outcomes for millions of students each year. In China alone, 10 million high school seniors participate in the National College Entrance Examination to compete for 7 million seats at various universities each year, making this system the largest centralized matching market in the world. The last 20 y have witnessed radical reforms in the Chinese college admissions system, with many provinces moving from a sequential (immediate acceptance) mechanism to some version of the parallel college admissions mechanism, a hybrid between the immediate and deferred acceptance mechanisms. In this study, we use a natural experiment to evaluate the effectiveness of the sequential and parallel mechanisms in motivating student college ranking strategies and providing stable matching outcomes. Using a unique dataset from a province that implemented a partial reform between 2008 and 2009, we find that students list more colleges in their rank-ordered lists, and more prestigious colleges as their top choices, after the province adopts the parallel mechanism in its tier 1 college admissions process. These listing strategies in turn lead to greater stability in matching outcomes, consistent with our theoretical prediction that the parallel mechanism is less manipulable and more stable than the sequential mechanism.

Since the 1990s, economic research has played an increasingly important role in the practical design of market institutions, including auctions for spectrums, electricity, and other commodities (1, 2); tradable permit systems for pollution abatement and other environmental regulations (3); labor market clearinghouses (4–7); formal procedures for student assignments to public schools or colleges (8–10); centralized systems for the allocation of organs (11); and other related matching and trading processes (12). In many of these cases, the insights drawn from theoretical, experimental, and empirical research have complemented each other in influencing market design choices.

Our study provides additional insight for the design of markets, specifically college admissions processes, obtained from a natural experiment to evaluate centralized matching procedures for student assignments to colleges. The college assignment process has a significant impact on the student educational experiences as well as on broader labor market outcomes in countries that use a centralized college admissions system based on standardized test scores. These countries include Australia, Chile (13), China (14), Germany (15–18), Greece, Hungary, Ireland, Russia, Spain, Turkey (19), and the United Kingdom.

Our study focuses on China in particular, where standardized test scores have been used since 1952 to match students to colleges via a centralized system. The National College Entrance Examination, also known as “gaokao,” forms the foundation of the Chinese college admissions system. Each year, roughly 10 million high school seniors compete for 7 million seats at various universities in China, making this system the largest centralized matching market in the world (14). Given the extent and importance of the Chinese admissions process, it is important to understand how the choice of an admissions mechanism impacts assignment outcomes.

The centralized college admissions problem (19) has several unique properties compared to other matching problems such as school choice (8). One major differentiator is that students’ priorities in college admissions are usually determined by their test scores on a standardized college entrance exam, rather than their place of residence, as in school choice problems. Therefore, college priorities are by and large identical across all colleges. Moreover, the prestige of a college is a major concern for virtually all students, leading to a near-universal preference for top universities with national prestige. This universal criterion implies that student preferences are often highly correlated. As a result, college admissions are typically much more competitive than student allocations to schools within a district. These two factors raise the stakes in the college admissions process and potentially affect how students strategize under different mechanisms.

In the past 2 decades, the majority of Chinese provinces have moved from a sequential mechanism to various versions of a parallel mechanism (PA) in assigning students to universities. In applying these mechanisms, universities are divided into tiers according to their level of prestige. The sequential mechanism is a priority matching mechanism (20) executed sequentially across tiers of decreasing prestige. Within each tier, the immediate acceptance (IA) mechanism is applied; e.g., once the assignments in the first tier are finalized, the assignment process in the second tier starts, and so on. Despite its dominance in the admissions process until 2001, a pervasive criticism of the sequential mechanism* is that many high-scoring students often remain unassigned or end up being undermatched due to poor strategizing in providing their preferred college rankings (24).

To combat this issue, Chinese provinces more recently have moved to some version of a PA, where students are provided with choice bands in which they can list several parallel colleges in decreasing desirability. Under PA, student applications are processed by these choice bands, wherein each student is guaranteed to retain her score advantage for any college she lists within the same choice band. This mechanism is perceived to alleviate the pressure experienced under IA by allowing students to aim for multiple colleges at the same time without the fear of losing their score advantage. For example, in Sichuan Province, where our dataset comes from, students can list up to five colleges within the same choice band. Students can choose to allocate their choices across a mix of desirable-yet-risky and less-desirable-yet-safer options.

It is plausible to argue that the PA falls somewhere between the IA and the deferred acceptance (DA) mechanism (25). In a theoretical study of the Chinese college admissions reforms, Chen and Kesten (14) consider a parametric family of application–rejection mechanisms where each member is indexed by some positive number $e\in \left\{\mathrm{1,2},\dots ,\infty \right\}$ of periodic choice band sizes that allow the application and rejection process to continue before assignments are made permanent. In this family of mechanisms, as parameter $e$ increases, one goes from IA ($e=1$) to PA ($e\in \left(2,\infty \right)$) and then to DA ($e=\infty$). Chen and Kesten (14) show that members of this family become more manipulable (26) and less stable (27) as one moves away from DA. While multiple equilibria may arise under any member of the family, their important insight is that it is students’ first $e$ choices that matter. We use these theoretical insights as a partial basis for our hypotheses in our natural experiment. Since the theoretical comparisons of IA and PA assume complete information and coordinated strategic play, it is important to test these predictions in the field to better gauge their policy implications.

Specifically, to complement and test the theoretical insights from Chen and Kesten (14), we study how different matching mechanisms in a centralized college admissions system affect students’ preference ranking strategies and matching outcomes. Our study is based on a natural experiment that takes advantage of the move from IA to PA in Sichuan Province from 2008 to 2009. Since the year students participate in the college admissions process is mostly determined by their year of birth, well before the change in mechanism, students in our study do not self-select into the different mechanisms. This timing feature eliminates any concern about selection bias in our study. Moreover, since the mechanism change affects only a portion of the students in our experiment (tier 1 students), we are able to use a difference-in-differences approach. Using this approach, we find that the results from our experiment confirm some of the theoretical predictions from Chen and Kesten (14). Last, our dataset is truly unique, as we have the complete rank-ordered list (ROL) from each applicant in addition to each applicant’s matching outcomes, whereas other empirical studies of the Chinese college admissions process do not have the ROL for each applicant (28–32). The ROL data enable us to make more precise inferences regarding students’ strategic responses to the change in the matching mechanism.

In particular, we find that students list more colleges in their ROL under PA relative to IA. Of the top-listed colleges, we observe a 5% increase in the preference rankings for the most prestigious colleges. Overall, our results show that the added insurance of being able to designate some safe options increases the stability of our matching outcomes.

Related Literature

Our study makes several important contributions to the literature on matching markets. Within this literature, a common approach in testing matching mechanisms is to conduct a laboratory experiment. Doing so makes it possible to induce true preferences and thus accurately obtain various performance evaluations. Indeed, the school choice problem has been extensively studied using laboratory experiments that yield support for various mechanisms. For example, Chen and Sönmez (22) find that DA performs well in terms of truthful preference revelation, while Pais and Pintér (33) find that the top trading cycles mechanism is more efficient and less vulnerable to manipulation than either IA or DA in the school choice scenario. In experiments under the interim information condition, Featherstone and Niederle (34) find that incomplete information on the student side changes both mechanism efficiency and truthfulness, while Calsamiglia et al. (35) find that constraining students’ ability to reveal their preferences leads to greater manipulation and lower efficiency. We refer the reader to a recent survey of the experimental literature on school choice and college admissions for further details (36).

Our paper contributes to the college admissions and broader matching literature by testing a common set of hypotheses using a natural experiment. The use of a field test provides higher external validity relative to laboratory experiments since the latter is unable to capture the large scale and high stakes nature of the real-world college admissions process.

Empirical evaluations have been used to study the properties and performance of different matching mechanisms. For example, Mongell and Roth (37) study the preferential bidding system that matches students to sororities and find that preference manipulation can prevent an unstable mechanism from unraveling. Braun et al. (15) study the centralized college admissions in Germany and find that high-performing students who truth-tell due to a lack of understanding of the mechanism receive suboptimal placements. More recently, several empirically studies have taken a structural approach to examine the performance of matching mechanisms (38–41) and uncover true preferences from reported ROLs when the mechanism is not strategy-proof. In a related study using school choice data from Beijing, He (39) finds that teaching middle school parents to play the best response under IA may yield better outcomes than switching to DA. Another strand of empirical literature takes a more direct approach by using preference reports under strategy-proof mechanisms or surveys (42–45). In particular, Fack et al. (44) provide theoretical and empirical evidence showing that assuming stability of the matching provides rich identifying information, while being a weaker assumption on student behavior, compared to assuming that students truthfully rank schools when applying for admission. The latter is corroborated by an online experiment using medical students immediately after their participation in the medical residence match which features a strategy-proof market design (46).

Finally, in the Chinese school choice and college admissions context, the college admissions mechanisms not only differ in their algorithm but also in the timing of students’ preference submissions. Wu and Zhong (31) find that under IA, better students are admitted to a top university when they submit their preferences before learning their test scores in the National College Entrance Examination, consistent with the theoretical prediction (47). Using laboratory experiments, Lien et al. (48) and Jiang (49) argue that requiring preference submissions before students take the examination can help correct the observed examination measurement error under IA. However, Pan (50) finds that preexamination IA rewards overconfidence and creates more mismatches between students and schools. Comparing all three mechanisms in the Chinese school choice context in the laboratory, Chen and Kesten (51) find that PA is less manipulable and more stable than IA. Compared to Chen and Kesten (14, 51) who first characterize the Chinese college admissions mechanisms theoretically and then test them in the laboratory, we use a unique naturally occurring dataset to test their theoretical predictions surrounding the switch from IA to the new PA mechanism. In doing so, we are able to provide a clean body of support for their basic theoretical predictions.

Theory and Hypotheses

In this section, we introduce the college admissions problem, describe a family of mechanisms, and summarize the main theoretical results pertaining to this family. These theoretical results form the basis of our empirical evaluation.

We begin by defining the college admissions problem. Specifically, a college admissions problem (19) is a tuple $\left(S,C,P_s,P_C\right)$, consisting of 1) a set of students $S=\left\{s_1,\dots ,s_n\right\}$; 2) a set of colleges $C=\left\{c_1,\dots ,c_m\right\}$; 3) a capacity vector $q=\left(q_{c_1},\dots ,q_{c_m}\right)$ where $q_{c_i}$ is the capacity of college $c_i$; 4) a list of student preferences $P_S=\left(P_{s_1},\dots ,P_{s_n}\right)$ where $P_{s_i}$ is the strict preference relation of student $s_i$ over colleges including the no-college option (with an unlimited quota); and 5) a list of college preferences $P_C=\left(P_{c_1},\dots ,P_{c_m}\right)$ where $P_{c_i}$ is the strict preference relation of college $c_i$ over a set of students, determined by students’ scores on the centralized college entrance examination. Therefore, $P_{c_i}=P_{c_j}$, $\forall i,j\in \left\{1,\dots ,m\right\}$. A matching $\mu$ is an allocation of college slots (and the no-college option) to students such that the number of students assigned to any college does not exceed its quota.

A matching $\mu$ is nonwasteful if no student prefers a college that has an unfilled quota. A matching $\mu$ is envy-free if there is no student–college pair $\left(c,s\right)$ such that student $s$ prefers college $c$ to the college she is assigned to and college $c$ prefers student $s$ to at least one student who is assigned to it. A matching is stable if it is nonwasteful and envy-free. A matching is Pareto efficient if there is no other matching that makes all students as well off and at least one student better off.

The recent literature focuses on analyzing weaker properties than stability, such as (justified) envy-freeness, i.e., fairness. Wu and Roth (52) consider envy-free matchings in a many-to-one matching environment and show that the set of such matchings forms a lattice. In a similar vein, Kamada and Kojima (53), motivated by various distributional constraints, focus on finding fair matchings that are student-optimal and apply their results to the Japanese daycare market.

A college admissions mechanism, or simply a mechanism, is an algorithm that selects a matching for each problem. A mechanism is Pareto efficient (stable) if it always selects Pareto efficient (stable) matchings. A mechanism is strategy-proof if no student ever gains by misrepresenting his preferences.

Prior to 2001, the sequential, mechanism (or IA) was the prevalent college admissions mechanism in China. However, after 2001, a number of provinces began to adopt various versions of the PA. By 2018, variants of PA had been adopted in all provinces. We next discuss an algorithm that describes a general family of mechanisms that nest IA, PA, and DA.

In the parametric application-rejection algorithm family, a member is indexed by a periodic choice band size $e$ that represents the number of choices the algorithm goes through when allocations are tentative before they become final.^† In this mechanism, students first submit their complete ROL before the allocation process starts. The algorithm is described as follows:

Round t $\ge 0$:

• •Each unassigned student from the previous round applies to his $te+1$ st-choice college. Each college $c$ considers its applicants. Those students with the highest score are tentatively assigned to college $c$ up to its quota. The rest of the applicants are rejected.

In general:

• •Each rejected student, who is yet to apply to his $te+e$th-choice college, applies to his next choice. If a student has been rejected from all his first $te+e$ choices, then he remains unassigned in this round and does not make any applications until the next round. Each college $c$ considers its applicants. Those students with the highest score are tentatively assigned to college $c$ up to its quota. The rest of the applicants are rejected.
• •The round terminates whenever each student is either assigned to a college (including the no-college option) or unassigned in this round, i.e., he has been rejected from all his first $te+e$ choices. At this point, all tentative assignments become final, and the quota of each college is reduced by the number of students permanently assigned to the college.

The algorithm terminates when each student has been assigned to a college or has received the no-college option. At this point, all of the tentative assignments become final. This family of mechanisms nests IA and DA as extreme cases and PA as an intermediate case (14). Specifically, IA is obtained when $e=1$, PA when $2\le e<\infty$, and DA when $e=\infty$. In this family, IA is the only Pareto efficient mechanism, whereas DA is the only stable or strategy-proof mechanism. In our study of college admissions in Sichuan Province, $e=5$.

In their theoretical study, Chen and Kesten (14) find that a move from one extreme mechanism to the other yields a trade-off in terms of strategic immunity and stability. At the individual strategy level, they show that whenever any given member can be manipulated by a student, any member with a smaller $e$ number can also be manipulated but not vice versa [theorems 1 and 3 in Chen and Kesten (14)]. This implies that the PA mechanism used in Sichuan Province (where $e=5$) is less manipulable than its predecessor, the IA mechanism. This leads to our first hypothesis:

Hypothesis 1 (Manipulability).

Students will manipulate their preferences less under PA compared to IA.

In our field setting, although true preferences are not directly observable, we can infer preference manipulation through a number of patterns, such as listing a safe college as one’s top choice, where a safe option may be a less prestigious college, or through the length of the submitted ROL. The theory in Chen and Kesten (14) suggests that under IA, in equilibrium, the choices other than the top choice do not matter, whereas the first five choices matter under PA (for Sichuan). If students understand this observation, we expect to see a longer ROL under PA.

Continuing with the theoretical predictions of Chen and Kesten (14), they suggest that students under PA are able to list their equilibrium assignments under IA as a safety option while also listing their more desirable options higher up in their preference list, which yields an outcome at least as good as that under IA [proposition 5 in Chen and Kesten (14)]. While the top choice is the most critical decision under IA, the first $e$ choices are of utmost importance when determining final assignments under PA. This leads to our next hypothesis:

Hypothesis 2 (Insurance).

Students will list more prestigious/more preferred colleges as their first choices under PA, compared to the IA mechanism.

In terms of choice accommodation, Chen and Kesten (14) show that the IA mechanism is more generous in allocating students to their first choice than PA. This leads to the following hypothesis:

Hypothesis 3 (Choice Accommodation).

IA will assign a higher number of students to their reported first choices than will PA.

In terms of stability, Chen and Kesten (14) show theoretically that the PAs are more stable than the IAs they replace [theorems 2 and 4 in Chen and Kesten (14)]. This leads to our final hypothesis:

Hypothesis 4 (Stability).

PA will be more stable than IA.

Finally, we note that Chen and Kesten (14) find no clear dominance of DA over PA, or PA over IA, due to the multiplicity of equilibria, even though the dominant strategy equilibrium outcome of DA Pareto dominates any equilibrium outcome of IA (21). Based on the predictions of Chen and Kesten (14), we are agnostic with regard to the efficiency comparison of the two mechanisms in our study.

Data and Empirical Methods.

Our dataset consists of the college admissions data of a county in the Sichuan Province in southwestern China for the years 2008 and 2009. The county had a population of 1.47 million with 87% rural in 2008 and 2009, with a per capita GDP of USD 994 in 2008 and 1,117 in 2009, below the national average of USD 3,524 and 3,828, respectively.^‡ For our study, we obtain the following student data: test score on the National College Entrance Examination, ROL of colleges, college admission outcome, and demographics. Compared to prior empirical studies of Chinese college admissions, our dataset is unique in that we have each student’s ROL.

Chinese colleges are categorized into tiers of decreasing prestige and quality. For example, tier 1 colleges are generally considered better than tier 2 colleges, etc. To determine college placement assignments, admissions mechanisms are executed sequentially across tiers. When assignments in the first tier are finalized, the assignment process in the second tier starts, and so on. Our dataset contains all students who participated in the tier 1, tier 2, and tier 3 admissions process in 2008 and 2009.

For the period of our dataset, students first received their test scores and relative standings among all of the students in the province and then completed their ROLs of colleges. The Provincial College Admissions Office determined whether a student was eligible to participate in the admissions of each tier by setting up an endogenously determined cutoff score, such that the number of students above the tier 1 cutoff was approximately 120% of the total quota of all tier 1 colleges, the number of students above the tier 2 cutoff was approximately 120% of the total quota of all tier 1 and tier 2 colleges, etc.^§

Additionally, there were two separate matching markets each year for the two academic tracks: humanities and social sciences (shortened as humanities henceforth) and science and engineering (shortened as STEM henceforth). Students self-select into one of the two tracks in their second year of high school and subsequently prepare for and then take the corresponding set of examinations. Likewise, each college has a separate quota for each of the two tracks.

Between the college entrance examinations of 2008 and 2009, the government of Sichuan Province announced that it would change the college admissions mechanism from IA to PA for only its tier 1 selection process. Since students participate in the college admissions process during their last year of high school, and the policy change was announced after the previous year’s admission was complete, students were essentially selected into different treatment groups by birth. Thus, this context allows us to use the policy change as a natural experiment to study the effects of different matching mechanisms on students’ behavioral responses and college admissions outcomes.

Even though students are randomly selected into the different years by birth, we consider the possibility that there may be other differences across the 2 y, such as students’ overall preferences for humanities versus STEM programs, that may impact our results. To address this possibility, we exploit the fact that only the tier 1 mechanism changed from 2008 to 2009 in Sichuan Province, whereas the tier 2 admissions mechanism remained the same. Therefore, we estimate the following difference-in-differences model:

$$\begin{array}{ll}\hfill y_i& ={\beta }_0+{\beta }_1\cdot Y2009_i+{\beta }_2\cdot Tier{1}_i+{\beta }_3\cdot \left(Y2009_i\cdot Tier{1}_i\right)\hfill \\ \hfill & +\gamma \cdot {\mathbf{X}}_i+{ϵ}_i,\hfill \end{array}$$

where $y_i$ is the outcome variable, measuring strategies or matching outcomes for each student. $Y2009_i$ and $Tier{1}_i$ are dummy variables that equal 1 for year 2009 and tier 1, respectively, and 0 otherwise. The vector, ${\mathbf{X}}_i$, contains students’ individual characteristics, including gender, residential status (rural or urban), academic track (humanities or STEM), and rank by test scores.

Table 1 presents the summary statistics for our dataset. These statistics show that students in different academic tracks and from different demographic backgrounds are similarly distributed across both years and tiers. Note that some students who were eligible for but did not receive a tier 1 admission placement subsequently participated in the tier 2 process. Students submitted their complete ROLs for all tiers at the same time, which was before the matching process was carried out, and no change was allowed once this process began.

Table 1.

Summary statistics

	2008				2009
	Total	Female	Rural	STEM	Total	Female	Rural	STEM
Participated in tier 1 admission	620	32.3%	80.2%	81.8%	768	36.3%	79.7%	80.7%
Participated in tier 2 admission	2,443	40.4%	80.8%	70.9%	2,735	42.7%	83.1%	73.8%
Participated in tier 3 admission	688	40.4%	75.0%	50.7%	605	48.3%	73.2%	57.4%
Participated in tiers 1 and 2	122	43.4%	81.1%	77.9%	135	46.7%	81.5%	77.0%
Participated in tiers 1, 2, and 3	2	100.0%	50.0%	100.0%	3	100.0%	100.0%	33.3%
Submitted tier 1 ROL	717	30.54%	79.9%	82.0%	849	35.7%	79.5%	80.1%
Submitted tier 2 ROL	2,967	38.5%	80.7%	72.8%	3,343	41.3%	82.7%	74.6%
Submitted tier 3 ROL	876	49.2%	72.3%	52.6%	787	48.9%	72.7%	57.8%
Submitted tiers 1 and 2 ROL	628	33.0%	80.6%	80.4%	723	37.2%	80.8%	78.2%
Submitted tiers 1, 2, and 3 ROL	4	100.0%	25.0%	25.0%	7	71.4%	57.1%	28.6%

Results

In this section, we first report our results regarding student strategies and then discuss our results regarding matching outcomes for the tier 1 admissions process, using the tier 2 process as our control. In SI Appendix, we use tier 3 as the control condition as a robustness check (SI Appendix, Tables S6–S9 and Fig. S1–S3).

Table 2 reports the summary statistics for the main outcome variables. At the individual strategy level, we investigate both the change in the number of colleges students rank (length of ROL), as well as the change in the prestige status of their top-choice colleges. At the outcome level, we examine the proportion of students admitted to their top choices, as well as the stability of the matching outcomes, using several measures to ensure robustness.

Table 2.

Summary statistics for outcome variables

	Tier 1		Tier 2		Tier 3
	2008	2009	2008	2009	2008	2009
Length of ROL (1 to 5)	3.676	4.473	4.166	4.239	3.245	3.392
Top-choice college prestige index (0 to 1)	0.519	0.458	0.403	0.368	0.458	0.406
First-choice accommodation rate	0.740	0.501	0.673	0.672	0.638	0.536
Stability based on cutoff score	0.118	0.070	0.273	0.292	0.237	0.358
Stability based on college prestige	0.109	0.086	0.134	0.117	0.170	0.189
Stability based on score distance	0.494	0.374	0.521	0.508	0.328	0.374

The prestige index (0, most prestigious; 1, least prestigious) is calculated by ranking colleges based on the average scores of admitted students in year 2006 and 2007 from the best to the worst, within each STEM/humanities track, tier, and year bracket (a total of eight); the rankings are then normalized to 0 to 1 by dividing the rankings by the total number of colleges within each bracket. The first-choice accommodation rate measures the percentage of students who are admitted by their first-choice colleges within each tier. A matching is stable when there does not exist any student–college pair where both prefer each other to their current matches. The measurement for stability based on cutoff score is described in Matching Outcomes, whereas the measurement for stability based on college prestige or score distance is relegated to SI Appendix; for each of the three measures, the larger the number, the more unstable the matching outcome is.

Student Preference Ranking Strategies.

In the college admissions process, students within a given tier are asked to rank order anywhere from one to five colleges. Fig. 1 reports the average length of the ROLs in 2008 and 2009 by tier. The red solid (green dashed) line refers to the length of tier 1 (2) students’ ROLs. From Fig. 1, we see an increase in the ROL length for tier 1 students from 2008 to 2009 by approximately one more college, whereas the average ROL length for tier 2 students remains the same across 2 y.

Fig. 1.

Average length of ROLs across year and tier. This figure reports the effect of changing the matching mechanism (tier 1; red solid line with circles) on the average length of ROLs compared to the baseline with no mechanism change (tier 2; green dashed line with triangles). Error bars indicate the 95% confidence interval of the mean.

Table 3 presents the results from nine ordinary least squares (OLS) specifications. In the length of ROL columns in Table 3, the dependent variable is the length of students’ submitted ROLs. The independent variables (omitted) include Y2009 (Y2008), tier 1 (tier 2), Y2009 $\times$ tier 1, STEM (humanities), rural (urban), female (male), and percentile ranking. To determine a student’s percentile ranking, we calculate rankings in each of the eight markets based on student test scores on the respective National College Entrance Examinations, as matching is carried out separately by year (2008/2009), tier (1/2), and academic track (humanities/STEM). To correct for different market sizes, we then normalize student rankings to their percentile rankings in their respective markets (the top-ranked student in each market is 1 [100%]. and the bottom-ranked student is 0). This measure of student rankings as our independent variable is used in all subsequent regressions.

Table 3.

Effects of matching mechanisms on the length of ROLs and the prestige of reported top choices (OLS)

	Length of ROL			Local prestige index of top choices			National ranking of top choice colleges
	First	Second	Third	First	Second	Third	First	Second	Third
Dependent variable	model	model	model	model	model	model	model	model	model
Y2009	0.073*	0.073*	0.076	−0.035***	−0.037***	−0.038***	0.051***	0.052***	0.052***
	(0.041)	(0.041)	(0.049)	(0.011)	(0.012)	(0.012)	(0.003)	(0.003)	(0.004)
Tier 1	−0.489**	−0.490**	−0.452**	0.115***	0.112***	0.109***	−0.191***	−0.200***	−0.198***
	(0.201)	(0.211)	(0.210)	(0.014)	(0.005)	(0.006)	(0.005)	(0.004)	(0.005)
Y2009 $\times$ tier 1	0.724***	0.724***	0.708***	−0.025**	−0.021*	−0.019	−0.044***	−0.046***	−0.047***
	(0.146)	(0.152)	(0.151)	(0.012)	(0.013)	(0.013)	(0.010)	(0.011)	(0.012)
Percentile ranking		−0.784***	−0.776***		−0.722***	−0.721***		−0.294***	−0.293***
		(0.170)	(0.168)		(0.026)	(0.026)		(0.015)	(0.015)
STEM			−0.306***			0.024***			−0.022
			(0.033)			(0.005)			(0.011)
Rural			−0.053			0.043***			0.009
			(0.033)			(0.011)			(0.006)
Female			0.133***			−0.019***			0.009
			(0.024)			(0.007)			(0.009)
Constant	4.166***	4.557***	4.767***	0.403***	0.767***	0.722***	0.433***	0.587***	0.592***
	(0.038)	(0.083)	(0.097)	(0.029)	(0.020)	(0.029)	(0.013)	(0.011)	(0.009)
Observations	7,876	7,876	7,876	7,706	7,706	7,706	6,757	6,757	6,757
R-squared	0.021	0.053	0.070	0.021	0.449	0.455	0.117	0.220	0.223
Y2009 + Y2009 $\times$ tier 1	0.797***	0.797***	0.784***	−0.060***	−0.058***	−0.057***	0.006	0.006	0.005
	(0.181)	(0.188)	(0.194)	(0.009)	(0.006)	(0.006)	(0.009)	(0.008)	(0.008)
Tier 1 + Y2009 $\times$ tier 1	0.235***	0.234***	0.257***	0.091***	0.091***	0.090***	−0.236***	−0.246***	−0.244***
	(0.086)	(0.087)	(0.089)	(0.017)	(0.013)	(0.012)	(0.011)	(0.011)	(0.011)

Standard errors clustered at the high school level are in parentheses; *, **, and *** denote significance at the 10, 5, and 1% levels, respectively. In the first model, the dependent variables are regressed on the year and tier dummies and their interactions using OLS. The second model adds students’ percentile rankings (0, lowest; 100%, highest) as control variable. The third model further adds students’ track and demographic information as additional control variables. The (local) prestige index (0, most prestigious; 1, least prestigious) is calculated by ranking colleges based on the average scores of admitted students in year 2006 and 2007 from the best to the worst, within each STEM/humanities track, tier, and year bracket (a total of eight); the rankings are then normalized to 0 to 1 by dividing the rankings by the total number of colleges within each bracket. The national ranking (0, highest ranked; 1, lowest ranked) is calculated by putting colleges into bins based on their national rankings in year 2008 and 2009 (top 2 colleges, Peking and Tsinghua, are in bin 1; top 3 to 10 are in bin 2; and every 10 colleges are in each subsequent bin) to account for correlated but heterogeneous preferences; then the bin numbers are normalized to [0,1] within each tier by dividing the numbers with total number of bins in that tier.

The results in the length of ROL columns in Table 3 show a positive significant coefficient for our main treatment effect, Y2009 $\times$ tier 1, indicating that the change from IA to PA in the tier 1 admissions process in 2009 increases the average ROL length by 0.724 ($p<0.01$). That is, a tier 1 student lists approximately one more college in 2009. Since the tier 2 ROL length remains stable across the 2 y, the change in the length of the tier 1 ROL is likely due to the change in the matching mechanism. Continuing with Table 3, we see that the coefficient for the tier 1 dummy is negative and significant, indicating that the average length of the tier 1 ROL is shorter than the corresponding tier 2 ROL in 2008. This finding may reflect the importance of a student’s first choice under IA, whereas lower-ranked colleges, such as a student’s fourth or fifth choice, are not that useful under IA. Under PA, however, students have an incentive to include a less-prestigious college as their fifth choice as insurance. Finally, the results for the third model in the length of ROL columns in Table 3 show that higher-ranked students as well as those in the STEM fields tend to submit shorter lists, while women tend to submit longer lists.

Next, we investigate whether students list more prestigious colleges as their top choices under PA, as predicted by theory. We use two measures to compute our prestige index. First, we compute a local prestige index, using province-specific calculations. We rank colleges from most (1) to least (n) prestigious, as measured by the average scores of students within a tier or track market. We calculate these rankings separately for 2006 and 2007 and then average the two to obtain a final prestige score for each college. These ranks are then normalized to range from 0 (most prestigious) to 1 (least prestigious) within each of the eight markets. Since not all colleges that admitted students in 2008 and 2009 did so in 2006 and 2007, observations with these colleges as top choices (2.2%) are dropped from our analysis. Compared to alternative measures, our local prestige index utilizes the same data and statistics published and distributed to students and their parents by the Sichuan Educational Examination Authority in the Gaokao Guide (55). While our prestige index is highly correlated with the published national rankings of colleges,^¶ using the average score of admitted students provides a more accurate aggregation of students’ revealed preferences for colleges compared to national rankings as students without complete preferences over colleges often use cutoff and average test scores to assess a given college’s prestige.^# Second, we use the national ranking of colleges as an alternative prestige measure, which has the advantage of being stable across years, even though it may not necessarily reflect the local preferences of students in Sichuan.

Fig. 2 presents the average local prestige index for students’ first-choice colleges by year and tier, with 0 (1) indicating the most (least) prestigious college. From Fig. 2, we see that on average, students choose more prestigious colleges in both tiers in 2009, compared to the 2008 choices, with a more pronounced increase for tier 1 students.

Fig. 2.

Average local prestige index of first-choice college by year and tier. This figure reports the effect of changing the matching mechanism (tier 1; red solid line with circles) on the local prestige of students’ first-choice colleges compared to the baseline with no mechanism change (tier 2; green dashed line with triangles). The prestige index (0, most prestigious; 1, least prestigious) is calculated by ranking colleges based on the average scores of admitted students in year 2006 and 2007 from the best to the worst, within each STEM/humanities track, tier, and year bracket (a total of eight). The rankings are then normalized to 0 to 1 by dividing the rankings by the total number of colleges within each bracket. Error bars indicate the 95% confidence interval of the mean.

We next examine the effect of the change from IA to PA on the prestige level of students’ first choices. In this analysis, the dependent variable is the local prestige level (from 0 to 1) of the student-reported top-choice colleges or the national ranking of top-choice colleges. The independent variables (omitted) again include Y2009 (Y2008), tier1 (tier 2), Y2009 $\times$ tier 1, STEM (humanities), rural (urban), female (male), and percentile ranking. The results in the national ranking of top-choice colleges columns of Table 3 show a negative and significant coefficient for our main treatment effect, Y2009 $\times$ tier 1, indicating that students list more prestigious colleges as their tier 1 first choices in 2009, with a magnitude of 2.5% ($p<0.05$) using the local prestige index and 4.4% ($p<0.01$) using the national ranking of colleges. This result is consistent with the theoretical prediction that students are more likely to pick prestigious colleges as their top choice under PA since they are also able to include a safer choice in their ROL (Hypothesis 2). Using the local prestige index, we find that the effect becomes insignificant when control variables are added (third model in the local prestige index of top choices columns of Table 3). We further find that students’ tier 2 first choices in 2009 are ranked 3.5% higher than their corresponding rankings in 2008 ($p<0.001$). It is not clear what drives this effect. Finally, the results for the third model in the local prestige index of top choices columns in Table 3 show that higher-ranked students as well as women list more prestigious colleges as their tier 1 first choices in 2009 (−0.72 and −0.019, respectively; $p<0.001$). When we use tier 3 as the control, the effect is also insignificant (SI Appendix, Table S7), indicating that the evidence is mixed. By contrast, using the national ranking as a measure of prestige, the treatment effect is robust to our model specifications.

We now summarize our treatment effect of the type of mechanism on student preference ranking strategies:

Result 1.

Changing the tier 1 admissions mechanism from IA to PA leads to an increase in the length of a student’s ROL by approximately one more college, as well as a 4.4% increase in the national ranking of students’ top-choice colleges.

These empirical results are consistent with the theoretical prediction that students view the PA as providing insurance or a fallback if they do not receive their ideal top choice, compared to the IA mechanism. Indeed, students appear to capitalize on the intuition that PA allows them to list more colleges and more prestigious colleges in the first tier without jeopardizing their admission chances to lower-ranked colleges within that tier.

Matching Outcomes.

Next we investigate the effects of the type of mechanism and its subsequent behavioral changes on matching outcomes. First, we examine the effects of the type of mechanism change on the likelihood that a student is admitted by her reported first-choice college. Recall that Hypothesis 3 predicts that IA will assign a larger number of students to their reported first choices than PA, as students have an incentive to aim higher under PA.

Fig. 3 depicts the first-choice accommodation rate by year and tier. As predicted by theory, we indeed see a drastic drop in the proportion of students admitted to their reported top-choice colleges in tier 1 in 2009 (red solid line), in contrast to no change for tier 2 admission rates (green dashed line). We next formally investigate this phenomenon through a regression analysis.

Fig. 3.

First-choice accommodation rate by year and tier. This figure reports the effect of changing the matching mechanism (tier 1; red solid line with circles) on the first-choice accommodation rate compared to the baseline with no mechanism change (tier 2; green dashed line with triangles). First-choice accommodation rate measures the percentage of students who are admitted by their first-choice colleges within each tier. Error bars indicate the 95% confidence interval of the mean.

The admitted to first choice columns in Table 4 report the results of our regression analysis using three probit specifications: the effects of the mechanism change on the likelihood of first-choice accommodation (first model), with student percentile ranking (second model), and with demographic controls (third model). The independent variables (omitted) again include Y2009 (Y2008), tier 1 (tier 2), Y2009 $\times$ tier 1, STEM (humanities), rural (urban), female (male), and percentile ranking.

Table 4.

Effects of matching mechanisms on first-choice accommodation and stability (probit)

	Admitted to first choice			Unstable matching
r0.5em)2-4l0.5em)5-7 Dependent variable	First model	Second model	Third model	First model	Second model	Third model
Y2009	−0.001	0.001	−0.001	0.019	0.017	0.018
	(0.023)	(0.021)	(0.022)	(0.024)	(0.023)	(0.023)
Tier 1	0.067	0.077*	0.073*	−0.154***	−0.157***	−0.156***
	(0.052)	(0.039)	(0.039)	(0.022)	(0.020)	(0.020)
Y2009 $\times$ tier 1	−0.238***	−0.248***	−0.246***	−0.067***	−0.063***	−0.064***
	(0.050)	(0.047)	(0.048)	(0.028)	(0.025)	(0.026)
Percentile ranking		0.558***	0.558***		−0.326***	−0.327***
		(0.017)	(0.016)		(0.025)	(0.023)
STEM			0.038***			−0.016**
			(0.014)			(0.008)
Rural			0.039***			−0.023**
			(0.012)			(0.009)
Female			0.005			−0.013
			(0.009)			(0.008)
Observations	6,566	6,566	6,566	6,300	6,300	6,300
Y2009 + Y2009 $\times$ tier 1	−0.239***	−0.247***	−0.247***	−0.048***	−0.046***	−0.046***
	(0.042)	(0.041)	(0.040)	(0.014)	(0.010)	(0.010)
Tier 1 + Y2009 $\times$ tier 1	−0.171***	−0.171***	−0.173***	−0.221***	−0.219***	−0.220***
	(0.026)	(0.027)	(0.026)	(0.017)	(0.020)	(0.020)

Standard errors clustered at the high school level are in parentheses; *, **, and *** denote significance at the 10, 5, and 1% levels, respectively. Marginal effects are reported, calculated at the mean level of the covariates. In the first model, dependent variables (whether a student is admitted by the first-choice college and whether a matching is unstable; 0 = false and 1 = true) are regressed on the year and tier dummies and their interactions using a probit model. The second model adds students’ percentile rankings (0, lowest; 100%, highest) as a control variable. The third model further adds students’ track and demographic information as additional control variables. A matching outcome is considered unstable if a student in tier 1 (2) has a listed college above her admitted college (within the same tier) whose cutoff score is lower than her test score or if she ends up in a tier 2 (3) college or lower even though her test score is high enough to obtain admission into one of her listed tier 1 (2) colleges.

Consistent with our theoretical prediction (Hypothesis 3), we find that the coefficient for our main treatment effect, Y2009 $\times$ tier 1, is negative and significant, indicating that students are 24 percentage points ( p $<$ 0.01) less likely to be admitted by their reported top choices in the tier 1 admissions process in 2009, whereas the likelihood of being admitted by first-choice colleges in tier 2 in 2009 does not change compared to the previous year (−0.001, $p>0.10$). Additionally, looking at the covariates, we find that students from rural areas are 3.9 percentage points more likely to be admitted into their reported first choices under PA. Finally, we see that students with a one-percentile increase in their entrance exam scores increase their likelihood of being admitted by their reported first choice by 0.558% under PA ($p<0.001$). Since PA incentivizes students to aim high, we also find a decrease in the acceptance rate of top-choice colleges after the change to PA.

In addition to examining the effect of the mechanism change on first-choice accommodation, we are interested in the performance of each mechanism in terms of matching stability. Recall that Hypothesis 4 predicts that PA will be more stable than IA. To measure stability, we first need to know students’ preferences over colleges. In our study, we examine students’ revealed preferences as indicated in their ROLs. This approach allows us to forego the assumption that students have identical preferences. With this measure, we assume that students preserve their preference order in their ROL; that is, they always list their more preferred colleges above their less preferred ones within the same choice band under PA, which is implied by remark 3 in Chen and Kesten (14).

To identify unstable matchings, we consider an outcome to be unstable in two possible situations. First, an outcome is considered unstable if a student in tier 1 has a listed college above her admitted college (within the same tier) whose cutoff score is lower than her test score, indicating justified envy. Second, an outcome is considered unstable if a student ends up in a tier 2 college or lower even though her test score is high enough to obtain admission into one of her listed tier 1 colleges. For tier 2 observations, the first condition is the same, whereas the second condition changes to receiving admission to a college below tier 2. While this approach ensures that all identified unstable matchings are truly unstable, it captures only a subset of all possible violations. For example, if the incentives of the IA mechanism lead a student to drop a highly desirable college from his list, violations of stability involving moving that student to an unlisted college are not detected. To address this issue, we use two alternative stability measures in SI Appendix. The first one uses college prestige as an approximation of students’ preferences over colleges, which gives us an (almost) complete student preference profiles over colleges. The second one uses a wasted score, or a consequence of an unstable matching, as an indirect measure. We discuss the pros and cons of each measure in SI Appendix, Tables S10–S15 and Figs. S4–S5.

Using our main stability measure, we report the proportion of unstable matchings by year and tier in Fig. 4. From 2008 to 2009, we see that the proportion of unstable matchings decreases for tier 1 students (solid red line), whereas that for tier 2 students remains almost constant (green dashed line). To examine this effect further, we next conduct a regression analysis on the same outcome variable.

Fig. 4.

Proportion of unstable matching by year and tier. This figure reports the effect of changing the matching mechanism (tier 1; red solid line with circles) on matching stability compared to the baseline with no mechanism change (tier 2; green dashed line with triangles). A matching outcome is considered unstable if a student in tier 1 (2) has a listed college above her admitted college (within the same tier) whose cutoff score is lower than her test score or if she ends up in a tier 2 (3) college or lower even though her test score is high enough to obtain admission into one of her listed tier 1 (2) colleges. Error bars indicate the 95% confidence interval of the mean.

The unstable matching columns in Table 4 report the results of our regression analysis of the effects of the mechanism change on matching stability. The dependent variable here is whether the student’s match is unstable. The independent variables (omitted) again include Y2009 (Y2008), tier 1 (tier 2), Y2009 $\times$ tier 1, STEM (humanities), rural (urban), female (male), and percentile ranking. From the results in Table 4, we see that the coefficient for our main treatment effect, Y2009 $\times$ tier 1, is negative and significant, indicating that the move to PA decreases the number of unstable outcomes by 6.7 percentage points ($p<0.01$).

We summarize our matching outcome analysis findings below:

Result 2.

Changing the tier 1 admissions mechanism from the IA to PA leads to a 24-percentage point decrease in the admissions students receive from their reported top-choice colleges and a 6-percentage point decrease in the likelihood of unstable matchings.

Our observed first-choice accommodation result is consistent with theoretical predictions (Hypothesis 3): students are indeed more focused on getting into their reported first choices under IA. The stability result is also consistent with theoretical predictions (Hypothesis 4) in that PA results in fewer unstable outcomes. This latter result is robust to different measures of stability, including a cardinal measure of the distance between a student’s exam score and the cutoff score (see SI Appendix for details).

To provide greater confidence in our findings, we conduct a robustness test excluding the bottom 20% of tier 1 students and the top 20% of tier 2 students from our analysis. We do so to address the potential concern that the switch to PA in the tier 1 process may impact the composition of students who participate in the tier 2 process, as different mechanisms may leave different students unadmitted after the tier 1 process concludes. Recall that of the students with the highest scores, the computer algorithm considers 120% of the tier 1 quotas for tier 1 admissions, with an end goal of admitting the number of students equal to the tier 1 quotas. This leaves 20% of the students rejected from the tier 1 process. These students then enter the tier 2 admissions process, and so on. This is important as our difference-in-differences estimates rely on the fact that the mechanism for tier 2 does not change between 2008 and 2009. Excluding these students from our analyses yields similar results to those from our main analyses. Finally, we rerun our analyses using tier 3 students as the control condition and find similar results except in the case of the local prestige index. SI Appendix summarizes the results from these robustness checks (SI Appendix, Tables S1–S5).

Conclusion

The assignment of students to colleges is one of the most important education policy issues throughout the world, with significant social welfare and economic development implications attached to the process. In China alone, 10 million high school students participate in the college admission process each year. Since 2001, the process of allocating available slots to students has changed from the immediate acceptance mechanism to various versions of the PA. While the PA has been shown to have numerous benefits on a theoretical level (14), its benefits have been examined empirically mostly in a laboratory setting (51). By contrast, our study examines the effect of the PA on student strategies and matching outcomes in a natural experiment using a unique dataset with individual-level ranking strategies before and after the adoption of the new mechanism.

Specifically, we analyze a natural experiment using difference-in-differences estimators. Although some theoretical properties of matching mechanisms cannot be directly tested empirically due to the lack of students’ true preferences, we can draw some analogies between the laboratory and the field using revealed preferences as seen in students’ ROLs of their preferred colleges. We find that when the mechanism changes from IA to PA, students list better colleges as their first choices. We also find that students list more colleges in their ROLs under PA. These behavioral responses lead to more stable matching outcomes.

As college admissions reforms continue in China and other parts of the world, theoretical, experimental, and empirical analyses of ongoing reforms not only deepen our understanding of the science of market design but also offer insights into how education and labor market policies should consider the adoption of better mechanisms in their implementation.

Data Availability.

The data and analysis code reported in this paper are available at the open Inter-university Consortium for Political and Social Research data repository (https://doi.org/10.3886/E121101V2) (57).