Failing at Remediation? College Remedial Coursetaking, Failure and Long-term Student Outcomes

Abstract

Colleges offer remedial coursework to help students enrolling in post-secondary education who are not adequately prepared to succeed in college-level courses. Despite the prevalence of remediation, previous research presents contradictory findings regarding its short- and long-term effects. This paper uses a doubly robust inverse probability weighting strategy to examine whether the degree completion and wage outcomes associated with remedial education vary by passing or failing remedial coursework. Using the NLSY Postsecondary Transcript-1997 data, we find that almost 30% of remedial course takers fail a remedial course. Students who took and passed their remedial coursework at both two-year and four-year colleges were more likely to graduate from college than similar students who did not take remediation. For both two-year and four-year college entrants, students who failed remedial coursework were less likely to obtain a bachelor’s degree and, among degree receivers, took longer to graduate. Students who entered two-year or four-year colleges and who failed remedial coursework earned lower wages over time compared to similar students who never took remediation. Among four-year college entrants, these wage differences seem to be explained completely by degree completion. However, wage differences for two-year college entrants still remain after accounting for degree receipt. Our findings thus suggest that while many students may benefit from remedial education, a substantial number of students struggle with remedial coursework and fail to realize the intended benefits.

Introduction

Approximately two-thirds of all students entering two-year colleges and 40% of all students entering four-year colleges enroll in some form of remedial coursework (Chen 2016). Remediation rates tend to be higher for delayed entrants and older returning students, as well as for Black and Hispanic students; remedial coursetaking is also generally higher at two-year, open-access institutions – institutions where many students begin higher education (Merisotis & Phipps 2000; Kurlaender & Howell 2012). While remediation rates are particularly high at community colleges and non-selective colleges and universities, even at selective four-year colleges and universities, 30% of students take at least one remedial course (Chen 2016). Course failure is a common experience among students in remedial courses. In fact, less than half of community college students who enroll in remedial courses ultimately pass them all; among four-year college students in remedial courses, that rate is just 59% (Chen 2016).

Considerable attention has been focused on the enormous cost of remediation to public colleges and universities as well as to students themselves. The annual cost of remediation is estimated to be nearly $7 billion nationally to colleges (Scott-Clayton, Crosta, & Belfield 2014), with many arguing that taxpayers are “double billed” for colleges teaching academic skills that students should have learned in high school (Saxon & Boylan 2001). A recent report from the Center for American Progress indicates that students and their families paid an annual $1.3 billion in out-of-pocket costs for remediation across the nation (Jimenez, Sargard, Morales, & Thompson 2016). Moreover, although research on the effectiveness of remedial education is mixed, over the past 30 years remedial course takers earned degrees at lower rates than their non-remediated peers (Adelman 1999; Adelman 2004; Chen 2016). However, simply eliminating remedial education is problematic, as evinced by Florida’s 2014 legislation that made remedial coursework optional for students (Hu et al. 2016). In the fall semester following this policy change, fewer students enrolled in remedial coursework and the passing rates for gatekeeper courses declined, as students who would have been placed in remediation were likely underprepared for those courses. This suggests that there may always be a need for remediation in college, particularly for students who are the most unprepared for college-level coursework (Bailey, Hughes, & Jaggars 2012). With growing demands for a skilled workforce in the United States (Bailey, Jeong, & Cho 2010), it is crucial for researchers and policymakers to better understand the efficacy of remediation.

Previous research has suggested that students receiving remediation rarely have better academic and labor force outcomes than similar students who were not placed in remediation. However, when analysts focus on students who successfully complete remediation, they find much more positive outcomes (Attewell, et al. 2006; Bahr 2008; Bettinger & Long 2009; Chen 2016; Hodara and Xu 2016). In this paper, we explicitly examine how the outcomes associated with remediation vary based on whether students pass or fail their remedial coursework. Drawing upon new postsecondary transcript data linked to the National Longitudinal Survey of Youth (1997), we are able to overcome a key limitation in previous research by distinguishing between students who fail a course versus those who do not complete credits and either withdraw or take an incomplete. Failure is an important signal in a student’s academic trajectory (Crisp, Nora, & Taggart 2009). We hypothesize that failing remedial coursework may be a particularly large setback for incoming college students, both since it has consequences for student aid eligibility (Staying Eligible 2017) and since many students may interpret remedial course failure as an indication that they do not belong in higher education. As such, we argue that examining remedial course failure is crucial to understanding remediation, college retention, and long-term student success.

Part of the difficulty in assessing the impact of remediation on student outcomes is that students who require remediation differ from those who do not, making it challenging to isolate the effect of remediation on college outcomes from the effects of remedial students’ relatively weak academic preparation and other student characteristics associated with remediation. Some researchers have attempted to address these selection issues by employing randomized controlled trials to evaluate interventions intended to supplement or ameliorate issues with remediation policies. Examples of these evaluations include Barnett et al.’s (2012) study, which examined the outcomes of developmental summer bridge programs offered in Texas, as well as Logue, Watanabe, and Douglas’ (2016) research, which evaluated the outcomes of students taking remedial mathematics simultaneously with introductory statistics. Another study reported findings from random assignment to “learning community” models designed to improve chances of college success (Visher, Weiss, Weissman, Rudd, & Wathington 2012). Overall, these studies find positive effects for the completion of college-level coursework across two-year community colleges, but they do not find a significant impact on persistence. These studies highlight the promise of remediation, as well as the challenges it faces in creating lasting change. In this paper we use a novel, nationally representative dataset to complement existing research on the effects of specific remedial interventions and underscore the important role of remedial course failure in whether students realize the intended benefits of remediation.

Literature Review

The Effects of Remediation on Educational Outcomes

A number of rigorous studies have focused on students who score near the cut-offs for remedial placement tests at two-year community colleges. Using a regression discontinuity (RD) design to compare students just above and below remedial placement cutoffs, Calcagno and Long (2008) find that remedial coursework promoted early persistence (into the second year), but did not affect eventual four-year degree completion in Florida, while Scott-Clayton and Rodriguez (2015) find no impact of remedial placement on enrollment, persistence, or eventual degree attainment in a large, urban community college system. Other researchers focus on remediated students in two-year colleges more broadly (and not just at the assignment threshold), finding that remediation lowers their odds of earning an associate’s degree (Clotfelter, Ladd, Muschkin, & Vigdor, 2015) and their likelihood of successfully transferring to a four-year college (Crisp and Delgado, 2014). These negative effects are congruent with Deil-Amen and Rosenbaum’s (2002) argument that remedial placement “cools out,” or lowers, students’ educational aspirations, potentially in part by diverting remedial students away from earning college-level credits towards a degree (Scott-Clayton and Rodriguez, 2015; Clotfelter et al., 2015).

Importantly, however, there is some evidence of heterogeneity in the effects of remediation at two-year colleges. Melguizo et al. (2016), for example, find evidence of school-level heterogeneity using an RD design to examine the effects of remedial placement in six community colleges in a large community college system, showing a range of negative, positive, and null findings across the different schools. Other studies find evidence of student-level heterogeneity: Using an RD design to examine remediation in a single large community college, Ngo (2018) finds that while students who are placed in remediation due to a lack of fraction knowledge subsequently fare worse than non-remediated students, remedial placement has no effects for those placed into remediation for other reasons.

While the effects of remediation at two-year community colleges have been studied extensively, there are fewer studies that examine remediation across two-year and four-year colleges. The findings from studies that do examine remediation in this context are largely mixed. Using an RD design, Bettinger and Long (2009) found that students placed in remedial math courses at nonselective four-year colleges were more likely to drop out or transfer to a two-year college. However, remediation did not lower the likelihood of obtaining a bachelor’s degree. Moreover, students who had completed their remedial math courses were more likely to obtain a bachelor’s degree, albeit taking more time to complete their degree, than students who attempted but did not complete their remedial math coursework. Boatman and Long (2018) also use an RD design to show that Tennessee students with lower levels of academic preparation actually benefit from taking remedial coursework, while those who only need a single remedial course do worse when placed in remediation. While Boatman and Long’s (2018) findings are primarily driven by two-year college students, they find a similar pattern among students at four-years schools. From these findings, we might expect that students who started at two-year colleges and take remedial courses are less likely to complete a bachelor’s degree than those who began at a four-year college, and remediation may add time to degree for two-year college entrants (see also Reynolds 2012; Scott-Clayton & Rodriguez 2015; Xu, Jaggars, Fletcher, & Fink 2018). On the other hand, there may be no differences across postsecondary institutions. Martorell and McFarlin (2011) directly examined the impact of remediation between two-year and four-year colleges, finding that remediation did not decrease the probability of receiving a bachelor’s degree, nor did it increase time to degree at either two-year or four-year colleges. Previous Research on Wage Outcomes

To date, little research has examined how remedial coursework affects wages, even though educational attainment more broadly is closely linked to social and economic advantages. Hout (2012) for example, found that annual earnings increased roughly 20% for each year of educational attainment. Moreover, the least educated workers were almost four times more likely than college graduates to be unemployed during the recession and also stayed unemployed longer than college graduates. To the degree that schooling provides students with important skills, we might expect remedial education to provide students with human capital (such as improved literacy) that is rewarded in the labor market (Johnson 2007). We might also expect that remedial coursework helps students succeed in the labor market by helping them succeed in college (though there is a dearth of evidence suggesting strong positive effects of remediation on college completion).

In one of the few studies to directly examine how remediation impacts labor market earnings, Martorell and McFarlin (2011) used a regression discontinuity approach for the Texas Academic Skills Program (TASP) test. Martorell and McFarlin do not find that students’ post-college earnings benefit from remediation in general, but rather find negative, albeit small, effects on six-year earnings in four-year schools with low remediation rates. On the other hand, Hodara and Xu (2016) find that students attending community college in North Carolina and Virginia have increased earnings after earning remedial English credits, which appears to be entirely driven by an increased probability of employment. However, they also find that students assigned to the lowest level of remedial math earned less over time (seven years after college entry). This study suggests that the time to take multiple remedial math courses may mean forgone earnings and that perhaps skills acquired in remedial math courses are not valued as much in the labor market as skills acquired in remedial English. More generally, recent research suggests that students who entered a two-year college and transferred to a four-year college may have lower earnings over time (Xu et al. 2018), even though two-year transfer students were just as likely as four-year college entrants to obtain a bachelor’s degree.

The Current Study

Our study contributes to this literature in three important ways. First and foremost, we attend to the important role of course failure, separately comparing students who pass and fail remediation to their non-remediated peers. Second, while regression discontinuity designs provide rigorous quasi-experimental evidence regarding the effects for students who were near the cut-off, they cannot speak to the effects for students who were not near the cut-off score, or for students in institutions with other assignment mechanisms for remedial coursework (e.g., where advising is an integral aspect of remedial course assignment). Our study includes students who took any remedial course and thus provides descriptive evidence regarding the schooling and labor market outcomes associated with remediation across a wider range of students. Third, our study examines remediation separately for two-year and four-year colleges across the country and thus offers insights on how the outcomes associated with remediation might vary by institution type (two-year or four-year colleges).

Our key research questions examine whether the degree completion and wage outcomes associated with remedial education vary by whether students pass or fail remedial courses. To motivate the analyses for our central research questions, we first ask (1) Who takes and who fails remedial coursework? Then, we address our key research questions: (2) What are the schooling outcomes associated with remedial coursetaking and how do they vary by whether students pass or fail their remedial coursework? and (3) What are the wage outcomes associated with remedial coursework and how do they vary by whether students pass or fail their remedial coursework? To understand how remedial courses may affect students across postsecondary educational institutions, we examine students who first entered a two-year college separately from those who first entered a four-year college. Given the important differences between two- and four-year colleges, as well as the potential relationships between the academic and wage outcomes we are interested in, we first examine the academic and wage outcomes for two-year college entrants, and then do the same for four-year college entrants.

Data

We use data from the National Longitudinal Survey of Youth 1997 (NLSY97), a nationally representative sample of approximately 9,000 youth born between the years 1980 and 1984. In 2011, when NSLY97 respondents were between the ages of 31 and 35, the study undertook a retrospective effort to collect complete postsecondary transcripts from all respondents who reported attending a postsecondary undergraduate degree program during any of the NLSY97 interviews (rounds 1 through 15). At least one transcript was received for 3,818 of the 4,399 youth for whom one or more transcripts were requested. Our sample consists of the participants who attended a post-secondary institution from 1997–2011, submitted valid post-secondary education transcripts, and who had work history information (n = 3,646). The postsecondary transcript data provides important chronological information about students’ enrollment patterns across two-year and four-year colleges, courses taken, and academic performance in these courses, including their bachelor’s degree attainment and time to degree completion.

In addition to data obtained from the NLSY Postsecondary Transcript Study 2011, the NLSY97 survey provides detailed information on employment history, such as work experiences, income, assets, and other economic characteristics. The NLSY97 work history data provide an annual work record of each respondent from January 1994 through 2011, and contain information on the respondent’s labor force status each year, the usual hours worked per week at all jobs, and earnings for all jobs. These data enable us to link detailed information about two- and four-year college students’ coursetaking patterns with their post-college wages.

Variables

We use NLSY97 transcript data to chart students’ remedial coursetaking experience. These data flag courses as remedial based on the 2010 College Course Map (CCM) taxonomy system.¹ We consider any student who took one or more remedial courses in college to be a student who was exposed to remedial instruction. We use the NLSY97 transcript course grade to identify students who failed remedial courses, coding students who received a “0” or “F” for the particular course and reported zero earned credits as failing the course.

After describing the correlates of remedial experiences, we explore the relationship between exposure to remediation, as well as failure in remedial courses, and two sets of student outcomes. First, we examine whether a student graduated with a bachelor’s degree, and for those who did, we also examine whether they finished within six years or after six years. These data are available for all NSLY97 respondents for whom transcript data are available. Second, we explore student labor force outcomes. These analyses use post-college wages reported in the years 2007 through 2011, the latest five years in the dataset.² Wages were averaged over the number of years of wage data that are available. If the student did not report their wages in any year between 2007 through 2011, the wages were considered missing and were excluded from the estimation of a student’s average wages over time. The average hourly wages were then converted to logged wages for the analyses.

All multivariate analyses control for a wide array of student demographic, socioeconomic, and academic background characteristics. Gender is a dummy variable, coded 1 for male and 0 for female. The racial categories we use are White, Black, Hispanic, Asian, and Other (which comprised approximately three percent of the entire sample). Birth cohort is an indicator for the year that the student was born, which in this sample ranges from 1980 to 1984. Age at college entry is measured in years by subtracting birth date from the student’s first term year. To account for family background, we also include biological mother’s age at first birth (in years), years of education of the respondent’s most educated parent, household income and logged per capita household income from when students were in high school, which is the earliest reported year for household income in the survey. We used the student’s test score percentile rank in the Armed Services and Vocational Aptitude Battery (ASVAB) test and high school grade point average (GPA) to measure students’ pre-college academic background. ASVAB measures the respondent’s knowledge and skills in topics such as mathematics knowledge, paragraph comprehension, and general science; most NLSY97 Round 1 respondents took the ASVAB test between 1997 and 1998. As an overwhelming majority of students (92%) worked at some point while attending college, we include a logged continuous measure of average hours worked per week. Finally, given the difference between two- and four-year colleges, we conduct separate analyses for students who first enrolled in two-year and four-year colleges as their primary institution after completing high school.

Missing Data

The percentage of missing values ranged from zero for some demographic variables, such as race, gender and birth year, to as high as 25%for household income. Only 45% (1,638) of the 3,646 students in the sample would have been available for analysis using listwise deletion. Data are primarily missing due to one of three reasons: the respondent did not participate at all in the survey year; the respondent did not provide a valid answer to the question; or it was a valid skip (e.g. a question only applied to respondents in a certain age range). We address the issue of missing data using in the data collection, the primary sampling unit, strata and weights were included in the multiple imputation (Rubin 1987). Because of the complex design employed imputation process (Allison 2003). Multiple imputation produces consistent estimates if the data are missing at random conditional on the variables in the model, and the imputation model is correctly specified.

Analytic strategy

Our initial analyses use these data to understand who takes remedial coursework, and how students who pass and fail their remedial coursework differ from their non-remedial course taking peers. We begin by comparing the demographic characteristics, socioeconomic background, and academic histories of students who take remediation with students who do not. We then separate students who passed and failed remedial coursework, first comparing students who passed their remedial coursework with their peers who did not take remedial coursework, and then comparing students who failed remedial coursework with their non-remediated peers. We estimate a series of models of the following general form:

$$Y_i={\alpha }_i+{\beta }_k{X}_{ki}+{\epsilon }_i$$ (1)

where Y_i represents respectively whether student i took remedial coursework, whether a student passed their remedial coursework (omitting students who failed remedial coursework), and whether a student failed a remedial course (omitting students who passed their remedial courses). We predict these outcomes as a function of a range of student characteristics, including gender, race, birth cohort, age at college entry, family background, and pre-collegiate academic background, represented as X_ki.

We then draw upon the results of these analyses to examine the relationship between remedial coursetaking and degree and wage outcomes using a doubly robust inverse probability weighting (IPW) strategy. In our observational data, we cannot randomly assign our treatment (i.e., remedial coursetaking and remedial course failure) to ensure that the treatment is independent of the outcome (e.g., degree completion). Thus, students who take remediation (and those who fail remedial coursework) are likely to differ from those who did not take any remediation (our “control” condition) during their college career in both observable and unobservable ways. Given these differences, we cannot estimate the effect of remedial coursetaking and failure on degree completion by simply comparing the point estimates for degree completion rates among those who did and did not take (or fail) remedial coursework. To create a more plausible counterfactual, we use a doubly robust IPW approach to account for differences in the observable characteristics of students who pass and fail remediation.

To execute this strategy, we use the covariates included in Model (1) to estimate the predicted probability that each student takes remedial coursework, and their predicted probability of passing or failing their remedial coursework. We then weight each student by the inverse of the predicted probability relevant for the comparison being made (e.g., those who took and failed remedial coursework versus those who did not take remedial coursework). To balance the groups on observable characteristics, the IPW scheme up-weights students who received a treatment condition they were unlikely to receive based on observable characteristics (e.g., students who were likely to take and fail remedial coursework but did not take remedial coursework, or who were likely to be non-remediated but took and failed remedial coursework). Conversely, the approach down-weights students who were highly likely to receive the treatment they received.

A limitation of IPW is that it assumes that the model used to predict the treatment (and therefore the weights) is correctly specified. If this model is not correctly specified, then the weighting will not account for the differences in these observable characteristics. We can relax the model specification assumption by using doubly robust IPW estimators, and including controls in our final weighted models predicting our outcomes. In these models, if either the weighting model or the final model is correctly specified, we will account for potential imbalance in our observable characteristics. It is important to clarify, however, that doubly robust models do not account for differences in the unobserved characteristics of respondents. For more details on how we calculate the doubly robust IPW estimators, see Appendix A.

Results

The first and fifth columns of Table 1 provide information about our controls and outcome measures separately for students who first entered a two-year college (n = 1,677) and those who entered a four-year college (n = 1,884), respectively.³ Students who entered two-year colleges are more likely to be male (47% compared with 45% of students who entered a four-year college); they are also less likely to be White (48% compared with 61% of students entering four-year colleges), more likely to be Black (25 % compared with 21%), more likely to be Hispanic (22% compared with 14%), and less likely to be Asian (two percent compared with three percent). On average, students entered a two-year college a year older (19.8 years) than did four-year college students (18.7 years).

Table 1.

Descriptive statistics of variables used in analyses (n=3.646)

	Two-Year College				Four-Year College
	Total	No R	Passed R	Failed R	Total	No R	Passed R	Failed R
	Mean	Mean	Mean	Mean	Mean	Mean	Mean	Mean
Gender
Male	0.47	0.51	0.43	0.48	0.45	0.45	0.44	0.51
Female	0.53	0.49	0.57	0.52	0.55	0.55	0.56	0.49
Race/Ethnicity
White	0.48	0.50	0.55	0.31	0.61	0.69	0.61	0.43
Black	0.25	0.23	0.19	0.39	0.21	0.13	0.22	0.41
Hispanic	0.22	0.23	0.21	0.24	0.14	0.13	0.13	0.16
Asian	0.02	0.01	0.02	0.02	0.03	0.03	0.03	0.00
Other	0.03	0.03	0.03	0.04	0.02	0.02	0.02	0.01
Age at entry (years)	19.8	20.3	19.5	19.7	18.7	18.9	18.5	18.9
Birth Cohort
1980	0.20	0.20	0.19	0.21	0.17	0.16	0.18	0.20
1981	0.20	0.20	0.19	0.23	0.20	0.21	0.21	0.13
1982	0.20	0.21	0.22	0.15	0.20	0.20	0.19	0.25
1983	0.19	0.18	0.20	0.20	0.22	0.23	0.21	0.19
1984	0.21	0.22	0.20	0.22	0.21	0.20	0.21	0.23
Socioeconomic Status
Mother’s age at first birth	22.9	22.5	23.2	22.5	24.5	25.0	24.2	24.1
Parent’s years of education	13.2	12.9	13.5	13.0	14.7	14.9	14.6	14.0
Household income (1997)	$44,650	$43,179	$47,700	$40,752	$62,964	$68,942	$61,460	$50,469
Income per capita	$10,670	$10,231	$11,463	$9,751	$15,582	$16,945	$15,354	$12,344
Academic Background
ASVAB percentile score	44.9	44.8	49.9	34.9	65.3	71.0	64.8	50.0
High school GPA	2.76	2.71	2.88	2.58	3.18	3.28	3.21	2.84
Employment in College
Employed students	0.92	0.89	0.95	0.93	0.93	0.92	0.93	0.95
Average hours per week	27.7	27.8	27.2	28.5	24.6	24.2	24.3	26.7
Degree Attainment
No degree	0.75	0.87	0.60	0.83	0.35	0.34	0.26	0.66
Associate’s degree	0.12	0.06	0.18	0.09	0.06	0.03	0.09	0.08
Bachelor’s degree	0.13	0.06	0.22	0.08	0.59	0.63	0.65	0.26
Time to Bachelor’s Degree
Earned within 6 years	0.63	0.64	0.68	0.41	0.84	0.88	0.83	0.59
Earned after 6 years	0.37	0.37	0.33	0.59	0.16	0.12	0.18	0.41
Hourly Wage
Total	$15.41	$16.47	$16.00	$12.22	$18.47	$18.75	$18.83	$16.26
No degree	$14.69	$15.69	$15.33	$11.87	$15.89	$15.01	$17.31	$15.47
BA degree	$18.76	$25.82	$17.98	$13.51	$19.75	$20.32	$19.40	$18.24
n	1,677	580	732	365	1,884	766	847	271

Source: National Longitudinal Survey of Youth (NLSY 1997), Postsecondary Transcript Study, 2011. R = remedial coursetaking. Sample restricted to students who had valid non-missing information on their postsecondary enrollment status and coursework. The sample sizes reported above provide information of the subgroups of students before multiple imputation, which will not add up to the total sample size used for analyses after multiple imputation. Age at entry and mother’s age at first birth reported in years. Income per capita is household income divided by the number of residents within household. Average hours per week are the reported hours the student worked per week. Hourly wage and degree category do not include students who obtained an associate’s degree. Average wages are reported after student’s last term in college, and do not include last term years from 2009 and onward.

To measure family socioeconomic status, we use the mother’s age at first birth, years of parental education (from the most educated parent), household income, and income per capita in the household. We see that the average age of mothers at first birth is 22.9 years among two-year college entrants (compared with 24.5 years among four-year college students). On average, a two-year student’s most educated parent had completed 13.2 years of education while a four student’s most educated parent-year completed 14.7 years of education. The average household income reported in 1997 is $44,650 for two-year students and $62,964 for four-year students. In our analyses, we use per capita household income, with an average of $10,670 for two-year students and $15,582 for four-year students.

We use the student’s ASVAB test score percentile and high school GPA to measure pre-college academic background. Students who entered a four-year college scored in the 65^th percentile on the ASVAB test, meaning that these students scored on average at the 65^th percentile of the national distribution of young adult test takers. By contrast, students who entered a two-year college scored at the 45^th percentile on the ASVAB test. Four-year college entrants had a higher GPA than those who entered a two-year college (3.18 GPA compared with 2.76). Both two- and four-year college students were overwhelmingly employed while in school. Employed students worked an average of 27.7 hours per week at a two-year college and 24.6 hours per week at a four-year college. Remediation rates were slightly higher in two-year colleges: Approximately 65% of students who entered a two-year college took a remedial course compared to 59% of students who entered a four-year college. About 22% of students who entered a two-year college failed a remedial course, while 14% of students who entered a four-year college failed a remedial course.

Regarding our key outcomes, 75% of two-year college entrants had not earned a degree, while 12% earned an associate’s degree as their highest degree, and 13% earned a bachelor’s degree. Among students who started at a four-year college, only 35% received no degree, while six percent earned an associate’s degree as their highest degree and over half of students received a bachelor’s degree (59%). Among those who earned a bachelor’s degree, 84% of students who entered a four-year college earned their degree within six years while 63% of students who entered a two-year college did so. Students who entered a four-year college had higher wages than students who started at two-year colleges ($18.47 versus $15.41), even among those with bachelor’s degrees ($19.75 compared with $18.76).

The second, third, and fourth sets of columns in Table 1 provide descriptive statistics for three groups of students who first entered a two-year college: those who did not take remedial coursework (column 2; n = 580), those who took and passed remedial coursework (column 3; n = 732), and those who took and failed remedial coursework (column 4; n = 365). We see that female and White students are overrepresented among students who passed remediation, and that Black students are overrepresented among students who failed remediation. Students who passed remedial coursework demonstrate higher levels of pre-college academic skills (scoring on average at the 50^th percentile on the ASVAB test) and achievement (2.88 GPA) than students who did not take remedial coursework, while students who failed remediation on average had lower levels of pre-college academic skills.

By contrast, among students who began at four-year colleges (columns 6–8), we find that students who did not take remedial coursework have the strongest academic backgrounds, followed by those who took and passed their remedial coursework. Socioeconomic advantage follows a similar pattern of advantage, where students who did not take remedial coursework are the most advantaged, and those who fail remediation are the least advantaged. Like students who began at two-year colleges, Black students are overrepresented among students who failed remedial coursework at four-year colleges, while White students are underrepresented among students who took remedial coursework, and particularly among students who failed remedial coursework.

Who Takes, Passes, and Fails Remedial Courses

Table 2 presents descriptive information about the characteristics of students who entered a two-year college and (a) took a remedial course while enrolled in college (compared to students who never took a remedial course),⁴ (b) passed their remedial coursework (compared to students who never took a remedial course) and (c) failed a remedial course (compared to students who never took a remedial course). Model 1 shows that there are relatively few significant predictors of taking remediation. Asian students are 17 percentage points more likely to take remediation than White students, and the likelihood of taking a remedial course increases by one percentage point for each additional year of education obtained by the respondent’s parent. Interestingly, there were no differences in pre-college academic ability and achievement between students who did and did not take remediation.

Table 2.

Linear Probability Models (LPM) predicting remedial coursetaking and performance among students who entered a two-year college

	Two Year College
	Remediation (vs no remediation)	Passed Remediation (vs no remediation)	Failed Remediation (vs no remediation)
	Model 1	Model 2	Model 3
Demographics
Female	0.05+ (1.80)	0.06* (2.08)	0.02 (0.59)
Black	0.06+ (1.82)	−0.01 (−0.34)	0.19*** (4.35)
Hispanic	0.03 (0.86)	0.00 (0.08)	0.10* (2.24)
Asian	0.17* (1.98)	0.15 (1.44)	0.32* (2.02)
Other	0.03 (0.45)	−0.03 (−0.35)	0.15 (1.65)
Age at entry	−0.11+ (−1.75)	−0.15* (−2.28)	−0.02 (−0.31)
Age squared	0.00 (1.40)	0.00 (2.03)	−0.00 (−0.11)
Birth Cohort 1981	0.01 (0.05)	−0.01 (−0.21)	0.04 (0.88)
Birth Cohort 1982	−0.04 (−0.88)	−0.02 (−0.48)	−0.05 (−1.12)
Birth Cohort 1983	0.00 (0.06)	0.00 (0.08)	0.02 (0.44)
Birth Cohort 1984	−0.02 (−0.47)	−0.03 (−0.61)	0.01 (0.23)
Socio-economic Status
Income per capita	−0.00 (−0.22)	−0.01 (−0.15)	−0.01 (−0.28s)
Parent’s years of education	0.01* (2.47)	0.01* (2.29)	0.01+ (1.68)
Mother’s age at first birth	−0.01 (−0.29)	−0.01 (−0.48)	0.00 (0.16)
Mother’s age squared	0.00 (0.49)	0.00 (0.64)	−0.00 (−0.05)
Academic Background
ASVAB percentile score	−0.01 (−0.83)	0.01 (0.43)	−0.07** (−3.08)
High school GPA	0.01 (0.33)	0.03 (1.62)	−0.05* (−2.19)
Employment Characteristics
Average hours worked	0.05 (0.98)	0.05 (0.93)	0.06 (0.99)
Constant	1.65 (2.27)	2.13 (2.60)	0.29 (0.34)
R2	0.04	0.05	0.10
n	1722	1354	981

Source: National Longitudinal Survey of Youth (NLSY 1997), Postsecondary Transcript Study, 2011.

Note. T-statistics underneath coefficients in parentheses. Controls are in reference to male, White, and birth cohort 1980.

⁺p < 0.1,

^*p < 0.05,

^**p < 0.01,

^***p < 0.001.

Model 2 compares those who passed remedial coursework to the sample of non-remedial takers at a two-year college (n = 1,354), and Model 3 compares those who failed remedial coursework to those who had never taken remedial classes (n = 981). We see that women are six percentage points more likely to take and pass remedial coursework (Model 2), while Black, Hispanic, and Asian students are more likely to fail a remedial course than White students (Model 3). Parental education similarly predicts the likelihood of passing and failing remedial coursework (though for failing this coefficient is only marginally significant). By contrast, Model 2 indicates that students who pass their remedial coursework have a similar academic background as those who take no remedial coursework, while Model 3 shows that students who fail their remedial coursework had lower GPAs in high school and ASVAB test scores than those who did not take remedial coursework.

Table 3 presents results from analyses that parallel those in Table 2, but for students who entered a four-year college. We see that Black students are ten percentage points more likely to take a remedial course compared with White students (Model 1), and that this difference is particularly pronounced when we compare non-remedial students to those who failed remediation. We also find that students who have higher GPAs and ASVAB scores are less likely to take remedial coursework. Although ASVAB scores predict both passing and failing remedial coursework similarly, the difference in GPA between those who pass their remedial coursework and those who did not take remedial coursework is not statistically significant (Model 2), indicating that the difference observed in Model 1 is driven by differences between students who did not take remedial coursework and those who took and failed remedial coursework (Model 3). Our measures of socio-economic background do not significantly predict remedial coursetaking at four-year schools.

Table 3.

Linear Probability Models (LPM) predicting remedial coursetaking and performance among students who entered a four-year college

	Four Year College
	Remediation (vs no remediation)	Passed Remediation (vs no remediation)	Failed Remediation (vs no remediation)
	Model 1	Model 2	Model 3
Demographics
Female	−0.03 (−1.35)	−0.02 (−0.67)	−0.07** (−2.64)
Black	0.10** (3.21)	0.09* (2.46)	0.19*** (4.25)
Hispanic	0.00 (0.01)	−0.01 (−0.23)	0.03 (0.61)
Asian	−0.09 (−1.34)	−0.06 (−0.73)	−0.14*** (−5.49)
Other	−0.04 (−0.48)	−0.02 (−0.25)	−0.02 (−0.24)
Age at entry	−0.14 (−1.60)	−0.13 (−119)	−0.09 (−1.02)
Age squared	0.00 (1.17)	0.00 (0.97)	0.00 (0.74)
Birth Cohort 1981	−0.04 (−0.94)	−0.02 (−0.50)	−0.08+ (−1.97)
Birth Cohort 1982	−0.01 (−0.36)	−0.02 (−0.53)	0.00 (0.02)
Birth Cohort 1983	−0.05 (−1.23)	−0.05 (−1.07)	−0.06 (−1.39)
Birth Cohort 1984	−0.02 (−0.60)	−0.03 (−0.73)	0.01 (0.14)
Socio-economic Status
Income per capita in household (logged)	0.04 (−0.30)	−0.00 (−0.06)	−0.01 (0.80)
Parent’s years of education	−0.00 (−0.28)	−0.00 (−0.30)	0.00 (0.08)
Mother’s age at first birth	0.02 (0.97)	0.02 (0.79)	0.03 (1.27)
Mother’s age squared	−0.01 (−119)	−0.00 (−1.08)	−0.00 (−1.18)
Academic Background
ASVAB percentile score	−0.09*** (−4.75)	−0.08*** (−3.84)	−0.08*** (−3.92)
High school GPA	−0.05** (−3.01)	−0.02 (−1.04)	−0.11*** (−5.74)
Employment Characteristics
Average hours worked per week	−0.01 (−0.26)	−0.03 (−0.71)	0.06 (1.47)
Constant	2.27 (2.32)	2.20 (2.07)	1.01 (1.00)
R2	0.07	0.05	0.21
n	1901	1629	1047

Source: National Longitudinal Survey of Youth (NLSY 1997), Postsecondary Transcript Study, 2011.

Note. T-statistics underneath coefficients in parentheses. Controls are in reference to male, White, and Birth Cohort 1980.

⁺p < 0.1,

^*p < 0.05,

^**p < 0.01,

^***p < 0.001.

Degree and Wage Outcomes among Two-Year College Students

Our results examining how remedial coursetaking and failure are related to degree attainment and wage outcomes among students who started at a two-year college are presented in Table 4. The specific outcomes we examine are bachelor’s degree receipt, receiving a bachelor’s degree after six years (among degree recipients), and post-college wages. For all three outcomes, we compare: those who had taken remedial coursework with those who had never taken a remedial course (Model 1; Remediation vs no remediation; this comparison most closely follows analyses in previous studies); those who had passed a remedial course with those who had never taken a remedial course (Model 2; Passed remediation vs no remediation); and finally, those who failed a remedial course with those who had never taken a remedial course (Model 3; Failed remediation vs no remediation).⁵

Table 4.

Doubly robust estimates of outcomes associated with remedial coursetaking and failure for students who entered a two-year college

	Model 1	Model 2	Model 3
	Remediation (vs no remediation)	Passed Remediation (vs no remediation)	Failed Remediation (vs no remediation)
Panel A: Predicting bachelor’s degree receipt
Coefficient	0.09***	0.12***	0.05**
T-statistic	(5.80)	(6.98)	(2.97)
N	1515	1181	909
Panel B: Predicting bachelor’s degree > 6 years (among degree receivers)
Coefficient	−0.03	−0.03	0.28+
T-statistic	(−0.45)	(−0.35)	(1.85)
N	219	190	62
Panel C: Wage
Coefficient	1.83	0.05	−0.14*
T-statistic	(0.00)	(0.94)	(−2.29)
N	1062	849	622
Panel D: Wage (controlling for degree receipt)
Coefficient	−0.04	0.00	−0.16**
T-statistic	(−0.79)	(0.03)	(−2.66)
N	1062	849	622

Source: National Longitudinal Survey of Youth 1997, Postsecondary Transcript Study, 2011. T-statistics underneath coefficients in parentheses. Includes demographic and prior achievement/academic skills controls for doubly robust estimates. Models in Panel D also control for both associate’s and bachelor’s degree completion.

⁺p < 0.1,

^*p < 0.05,

^**p < 0.01,

^***p < 0.001

In Panel A, we examine the likelihood of receiving a bachelor’s degree for students who entered a two-year college. We find that taking remediation is associated with a nearly nine percentage-point increase in bachelor’s degree completion for two-year college students after accounting for demographic, familial, and academic background characteristics. In Models 2 and 3, we see that both students who passed their remedial coursework and those who failed are more likely to graduate than observationally similar students who did not take remediation, although this difference is more pronounced when looking at students who passed remediation. Our results thus suggest that taking and passing remedial coursework increases the likelihood of completing a bachelor’s degree among students who started at two-year colleges. Moreover, while students who fail remediation do not benefit from remedial coursework to the same degree as students who pass, we find that among students who started at a two-year college, even those who fail remedial coursework receive bachelor’s degrees at higher rates than observationally similar students who do not take remedial coursework.⁶

In Panel B we examine whether students who initially entered a two-year college and completed a bachelor’s degree did so within six years, which is the national average time to complete a bachelor’s degree (Kena et. al. 2015). We find no evidence that their remedial students who passed coursework took more time to complete their degree (among bachelor’s degree completers). These results imply that remediation has little effect on increasing the time needed to complete a degree for those starting at a two-year college, if students pass their remedial coursework. One explanation is that entering a two-year college already increases a student’s time to degree if the student transfers up to a four-year school (Deil-Amen and Rosenbaum 2002), so that any additional time needed to complete remedial coursework and then earn a degree is not significant for students who are already on a pathway that takes longer to do so. However, Model 3 suggests that students who fail a remedial course and eventually attain a bachelor’s degree are substantially more likely to take over six years to do so, although this difference is only marginally significant.

Panels C and D report results from our wage analyses. Panel C shows that students who do and do not take remedial courses have similar post-college wages (Model 1). Importantly, however, we see a different pattern of results among students who pass and fail their remedial coursework. Students who pass their remedial coursework earn similar wages as their non-remedial coursetaking peers (Model 2), while students who fail remediation earn substantially less than students who do not take remedial courses (Model 3). Panel D examines the degree to which these differences in wages remain once we account for differences in associate’s and bachelor’s degree receipt.⁷ We find a largely similar pattern of results as in Panel C, suggesting that even after accounting for associate’s and bachelor’s degree receipt, students who take and fail remediation at two-year colleges earn less than observationally similar students who do not take remedial courses.

Degree and Wage Outcomes Among Four-Year College Students

Table 5 parallels Table 4, but presents results from analyses examining the degree and wage outcomes associated with remediation among students who initially entered four-year colleges. In Panel A, we examine the likelihood of receiving a bachelor’s degree among the three comparison groups for students who entered a four-year college. We find that students who take remedial coursework do not, on average, differ from observationally similar students in their probability of completing a bachelor’s degree (Model 1). However, this average null effect masks very different outcomes for students who pass and fail remedial coursework. Model 2 shows that students passing remediation are more likely (eight percentage points) to obtain a bachelor’s degree than those who did not take remediation. By contrast, Model 3 shows that four-year college students who fail remedial coursework are substantially less likely (26 percentage points) to receive a bachelor’s degree than those who did not take remediation. Panel B examines whether students who take remedial coursework at four-year colleges and receive a bachelor’s degree take longer to receive their degree, showing that only students who fail remedial coursework experience an increase in the likelihood that they will take over six years to finish their degree.

Table 5.

Doubly robust estimates of outcomes associated with remedial coursetaking and failure for students who entered a four-year college

	Model 1	Model 2	Model 3
	Remediation (vs no remediation)	Passed Remediation (vs no remediation)	Failed Remediation (vs no remediation)
Panel A: Predicting bachelor’s degree receipt
Coefficient	0.02	0.08***	−0.26***
T-statistic	(0.62)	(3.79)	(−9.62)
N	1777	1527	999
Panel B: Predicting bachelor’s degree > 6 years (among degree receivers)
Coefficient	0.05	0.03	0.11**
T-statistic	(1.46)	(1.17)	(2.99)
N	1074	1005	541
Panel C: Wage
Coefficient	0.03	0.01	−0.10*
T-statistic	(0.60)	(0.18)	(−2.14)
N	1413	1234	777
Panel D: Wage (controlling for degree receipt)
Coefficient	0.02	−0.01	−0.04
T-statistic	(0.46)	(−0.09)	(−0.83)
N	1413	1234	777

Source: National Longitudinal Survey of Youth 1997, Postsecondary Transcript Study, 2011. T-statistics underneath coefficients in parentheses. Includes demographic and prior achievement/academic skills controls for doubly robust estimates. Models in Panel D also control for both associate’s and bachelor’s degree completion.

⁺p < 0.1,

^*p < 0.05,

^**p < 0.01,

^***p < 0.001

In Panels C and D, we examine the relationship between remedial coursetaking and average hourly post-college wages among respondents who attended four-year colleges. Similar to two-year college entrants (who do worse if they fail), among four-year college entrants, we find that those who fail remedial coursework earn substantially less (approximately ten percent) than those who did not take remedial coursework, although this finding is only marginally significant. Panel D of Table 5 shows that the wage differences that we observe in Panel C appear to be largely driven by changes in the probability of receiving a degree, as once degree receipt is introduced as a control, we see that the wage differences are reduced and are no longer statistically significant. The results in Table 5 thus suggest that remedial coursework among students who start at four-year schools may help them complete their bachelor’s degree if they pass, but that if they fail they are much less likely to complete their degree and their average earnings are lower as a result. However, unlike students who start at two-year schools, where the earnings loss associated with failing a remedial course remains after taking degree receipt into account, for students who start at four year schools, the earnings difference we observe appears to be a function of degree receipt.

Discussion

Despite extensive research on the impact of remediation in college, little is known about whether the outcomes associated with remediation differ by whether students pass or fail their remedial coursework. Our first set of analyses show that different characteristics predict whether students take and fail remedial coursework at two-year and four-year colleges. We find that, although some factors predict remedial coursetaking and failure at both two-year and four-year colleges, others do not. Gender, for example, predicts taking and failing remedial coursework at four-year but not two-year schools, while Black students are more likely than White students to take and fail remedial coursework at both two- and four-year colleges. The characteristics of students who take remedial coursework are largely consistent with prior research using nationally representative data (Attewell et. al. 2006; Chen 2016); however, we find that academic background measured through high school GPA and ASVAB scores predicted failing remedial coursework at both two- and four-year schools.

Like prior nationally representative research on remediation, this study cannot account for potential differences in the unobserved characteristics of students who do and do not take (and fail) remedial coursework. As such, we view our results as descriptively instructive, rather than causal. We also note that our R-squared values in Tables 2 and 3 are relatively low, suggesting that there is still much to be learned about who takes and fails remedial courses. As our doubly robust IPW results from Tables 4 and 5 highlight the substantively important and statistically significant differences in outcomes based on who takes and fails remedial coursework, we believe that understanding the factors predicting remedial course failure will be an important undertaking for future research. In particular, we believe that it will be fruitful to examine information about whether students are placed into remediation through a college-entry test or are advised to take remediation, as well as other factors found to be associated with college completion, such as participation in precollege encouragement programs, quality of faculty-student contact, peer interactions, experiences or perceptions of diversity on the college campus, student satisfaction, perceptions of failure, motivation or self-efficacy, and participation in extracurricular activities while enrolled in college (Tinto 1993).

In spite of its limitations, this study provides important insights for understanding the outcomes associated with remediation. Prior research suggests that remediation has little to no benefits for students who begin their studies at two-year colleges (Calcagno and Long, 2008; Clotfelter et al., 2015; Crisp and Delgado, 2014; Deil-Amen & Rosenbaum, 2006; Scott-Clayton & Rodriguez, 2015). However, using nationally representative data and a wide range of controls, we find that taking remediation is positively associated with bachelor degree completion among two-year college entrants, regardless of whether they pass or fail. One possible explanation is that remedial course enrollment in our study may be capturing two-year college students’ persistence in remedial course sequences and intention to transfer to a four-year school, given that only one third of students assigned to remediation enroll in a remedial course at two-year colleges (Bailey, Jeong, & Cho 2010).

Our study calls attention to the important role of remedial course failure in understanding the outcomes associated with remediation. We find that students who pass remedial coursework generally do no worse than similar students who did not take remedial coursework, while students who fail remediation take longer to graduate and earn lower wages than similar non-remediated students. We also find notable differences in the outcomes associated with remedial coursetaking and failure among students who start at two- and four-year colleges, as among four-year college entrants failing remediation is also associated with a lower likelihood of degree receipt. Interestingly, students entering two-year colleges who fail remedial coursework appear to still be more likely to earn a bachelor’s degree than their counterparts who do not take remedial coursework. Thus, in contrast to Martorell and McFarlin’s (2011) findings, our results show disparate patterns across students who pass and fail remediation at two- and four-year colleges.

While we cannot speak to the mechanisms as to why failing remediation has different consequences at two-year and four-year schools, our findings might indicate that two-year colleges are better at remediation for degree completion than four-year colleges. We might also consider remediation (and remedial course failure) as a potential site for sorting students within higher education, as posited by Scott-Clayton and Rodriguez (2012) and Bettinger and Long (2009). From this perspective, failing a remedial course at a four-year college might signal to students that they do not belong in higher education. In this way, remedial course failure at a four-year college might divert students away from the college pathway altogether. By contrast, as remediation is more common among students entering two-year colleges, students could understand remedial coursework differently in this context. Given that Xu et al. (2018) find that students who begin at a two-year school have higher rates of bachelor degree attainment than students who enter a four-year school, we might speculate that remedial coursetaking among students entering two-year schools may indicate students’ willingness to make longer-term investments in their education, and their intention to transfer to a four-year school. While students who enter two-year colleges and fail remedial coursework do not receive degrees at the same rate as those who pass remedial courses, they nonetheless appear to receive some benefit from their remedial experience. However, these differences in how remediation works for students starting at two- and four-year schools may also be due to selection and differences with the non-remediated student body.

We also find differences in the relationship between post-college wages and remedial course failure at two- and four-year colleges. At both two- and four-year colleges, students who fail remedial coursework do worse compared to those who were not remediated. For students who entered a two-year college, the wage differences for failing remediation remain significant even after accounting for associate’s and bachelor’s degree completion, while the wage differences for those who fail remedial coursework among four-year college students appear to be completely explained by differences in associate’s or bachelor’s degree receipt. Although we are unable to speak to the precise mechanisms driving these results, it would appear that the negative repercussions of remedial course failure for the wages of four-year college students are largely a function of degree completion. Among two-year college students who failed a remedial course, however, remedial failure appears to matter for wages after accounting for degree receipt. This might indicate that remedial course failure has larger impacts on students’ confidence that remain even after we account for degree completion, though it is not immediately clear why this would be the case particularly at two-year schools, especially given our results for degree completion.

One explanation could be that students starting at two-year colleges who failed remediation have a delayed entry into the labor market and are not able to catch up in earnings over time. Similar to findings from Jaggars and Xu (2016) and Hodara and Xu (2016), these students are not able to receive the same positive increases in earnings growth over time compared to those who pass remediation, even after accounting for bachelor’s degree receipt. Failing remedial coursework may be a particularly large setback in delaying students’ entry into the workforce, not only in the loss of credits in the transfer process, but also in additional time needed for students to navigate the remedial coursetaking process after their transfer. Regardless of the precise mechanisms, these post-college wage differences are significant, as they are likely to grow over time as cumulative advantage processes widen these disparities in earnings.

The results of this study have implications for potentially identifying and meeting the different needs of students at two-year and four-year colleges. Policymakers should first be aware that two-year and four-year colleges structure remediation differently and that a one-size-fits-all approach would not be appropriate in these contexts. Four-year colleges would appear to benefit from emulating the support structures curriculum) for remediation (e.g. advising, tutoring, and at two-year colleges that help students eventually obtain a bachelor’s degree, given that students who pass their remedial coursework appear to benefit from a higher likelihood of obtaining a degree than their non-remediated counterparts. However, two-year schools also need to ensure that students at these institutions take and pass remediation, given the lasting wage penalties for failing remedial coursework among students entering two-year colleges. To the degree that the issues faced by students entering two-year colleges are related to delays, co-requisite programs are a potentially promising solution, as they combine the enrollment of both a transfer-level course in English and mathematics course with a support course for students underprepared for college-level coursework. While these programs are relatively new, early research indicates promising results, with higher course completion rates and improved academic performance (Logue, Watanabe, & Douglas, 2016). Other efforts to help struggling remedial students at two-year colleges should accelerate the degree attainment process so that their wages do not suffer relative to their non-remediated peers. Those who passed remediation should also be given consideration.

This study contributes to the existing literature on the effectiveness of remediation by highlighting that remedial course failure is relatively common at both two- and four-year colleges, and that students who fail remedial coursework do not appear to benefit to the same degree as their counterparts who pass. In doing so, we not only highlight the important differences in the outcomes associated with remediation at two- and four-year schools, but also call attention to the important role that failure plays in shaping students’ experiences. Future studies seeking to understand the effectiveness of remediation should attend to the important role that failure might play in determining the effectiveness of remediation, as well as how the effects of failing remedial coursework might be mitigated.

Conclusion

Remedial course enrollment is a common experience among students entering higher education, and the high rates of remedial course failure underscore the importance of understanding its impact on students’ future outcomes. Our study examined how passing or failing remedial coursework might differentially shape students’ educational pathways and wages over time, comparing students who entered a two-year or four-year college. Using detailed individual-level data from NLSY Postsecondary Transcript 1997–2011, we find that students who passed their remedial coursework complete bachelor’s degrees at higher rates than students who did not take remediation, and have otherwise generally similar outcomes. By contrast, students who failed their remedial coursework took longer to obtain a bachelor’s degree and earned lower wages than non-remediated students, even though at two-year schools they were slightly more likely to obtain a bachelor’s degree. While our study cannot speak to the precise mechanisms as to why remedial course failure operates differently at two-year and four-year schools, they highlight the import role of remedial course failure at both two-year and four-year schools. In doing so, we demonstrate that although students may benefit from remedial education, many students still struggle with remedial coursework and fail to realize its intended benefits.

Appendix A

Doubly Robust Inverse Probability Weighting

In the first step of doubly robust IPW, we estimate treatment propensities (P) for each student. Using covariates discussed in the paper, a propensity score is estimated for each student. An individual variable does not have to be a statistically significant predictor of treatment to be included in the propensity model since the objective is for students in the treated and control categories to be balanced on the covariates. The propensity scores are estimated using a multivariate logistic regression model predicting the probability of a student receiving the treatment (i.e., not taking a remedial course, taking and passing a remedial course, and taking and failing a remedial course). All covariates discussed in the paper were included in the multiple logistic regression equation to predict the probability of treatment:

$$Pr{\left(\mathit{\text{Remedical\hspace{0.17em}}}\mathit{\text{Group}}\right)}_i={\alpha }_i+{\beta }_k{X}_{ki}+{\epsilon }_i$$ (A1)

Equation (A1) predicts the probability of student i being in one of three groups: never took a remedial course, took and passed remedial coursework, and took and failed a remedial course. X_ki is a vector of control variables.

We estimate each student’s predicted probability of being in each of the remedial groups in Equation (A1), and use these probabilities to create inverse probability weights. For each treatment category t (never took remediation, took and passed remediation, or took and failed remediation), we define our inverse probability weight as:

$$W=1/{\widehat{P}}_t$$ (A2)

where ${\widehat{P}}_t$ is the predicted probability that a student received the treatment that he or she received.

For doubly robust IPW estimators, the same covariates used to estimate the probability weights for Equation (A1) are also included as controls in a linear probability model predicting our degree and wage outcomes. We estimate two sets of these models; the first set predicts bachelor’s degree completion in any field and the second set predicts the average wage outcomes. Thus, our first model predicts whether students complete a bachelor’s degree as a function of being in one of the three remedial groups: never took remediation, took and passed remediation, and took and failed remediation:

$$Pr{\left(\mathit{\text{Bachelors}}\right)}_i={\alpha }_i+{\beta }_1\mathit{\text{Remedica}}{l}_i+{\beta }_k{X}_{ki}+{\epsilon }_i$$ (A3)

where Remedial_i is a dummy variable equal to one if a student ever took remediation and zero otherwise and X_i is a vector of background controls for doubly robust estimates. To estimate the relationship between failing remedial coursework and our other outcomes, we estimate additional models that take the same general form as (A3), but instead of Remedial_i, we use a dummy variable equal to one if a student took and passed their remedial coursework and zero otherwise, or alternatively a dummy variable equal to one if a student took and failed their remedial coursework and zero otherwise. The error term, ε_i captures characteristics not accounted in the model that influence the outcome variable. We estimate these models separately for students who entered a two-year and four-year college and we use similar models to predict average post-college wages for the latest five years (2007 through 2011).

Table A1.

Linear Probability Models (LPM) predicting failure among remedial course takers

	Two Year College	Four Year College
	Failed Remediation (vs passed remediation)	Failed Remediation (vs passed remediation)
Demographics
Female	−0.04 (−1.53)	−0.04 (−1.47)
Black	0.19*** (4.34)	0.08+ (2.16)
Hispanic	0.08+ (1.80)	0.04 (0.91)
Asian	0.10 (0.87)	−0.19*** (−5.56)
Other	0.14+ (1.73)	−0.08 (−1.01)
Age at entry	0.12+ (1.75)	−0.03 (−0.31)
Age squared	−0.01+ (−1.94)	0.00 (0.30)
Birth Cohort 1981	0.07+ (1.67)	−0.07+ (−1.71)
Birth Cohort 1982	−0.03 (−0.74)	0.04 (0.90)
Birth Cohort 1983	−0.01 (−0.01)	−0.01 (−0.25)
Birth Cohort 1984	0.04 (0.04)	0.02 (0.57)
Socio-economic Status
Income per capita	−0.01 (−0.11)	−0.00 (−0.66)
Parent’s years of education	−0.01 (−0.61)	−0.00 (−0.12)
Mother’s age at first birth	0.02 (0.62)	−0.00 (−0.10)
Mother’s age squared	−0.00 (−0.60)	0.00 (0.37)
Academic Background
ASVAB percentile score	−0.07*** (−3.33)	−0.06** (−2.90)
High school GPA	−0.08*** (−4.17)	−0.11*** (−6.24)
Employment Characteristics
Average hours worked	0.03 (0.72)	0.07* (1.97)
Constant	−1.16 (−144)	0.43 (0.42)
R2	0.13	0.15
n	1109	1121

Source: National Longitudinal Survey of Youth (NLSY 1997), Postsecondary Transcript Study, 2011.

Note. T-statistics underneath coefficients in parentheses. Controls are in reference to male, White, and Birth Cohort 1980.

⁺p < 0.1,

^*p < 0.05,

^**p < 0.01,

^***p < 0.001.

Table A2.

Doubly robust estimates of outcomes associated with remedial course failure relative to passing remedial coursework

	Two-Year College	Four-Year College
	Failed Remediation (vs. passed remediation)	Failed Remediation (vs. passed remediation)
Panel A: Predicting bachelor’s degree receipt
Coefficient	−0.11***	−0.31***
T-statistic	(−4.93)	(−10.66)
N	940	1025
Panel B: Predicting bachelor’s degree > 6 years (among degree receivers)
Coefficient	0.27**	0.24***
T-statistic	(3.30)	(5.54)
N	186	602
Panel C: Wage
Coefficient	−0.13**	−0.10*
T-statistic	(−2.62)	(−2.15)
N	652	814
Panel D: Wage (controlling for degree receipt)
Coefficient	−0.11*	−0.05
T-statistic	(−2.12)	(−0.93)
N	652	814

Source: National Longitudinal Survey of Youth 1997, Postsecondary Transcript Study, 2011. T-statistics underneath coefficients in parentheses. Includes demographic and prior achievement/academic skills controls for doubly robust estimates. Models also control for the total number of remedial courses taken.

⁺p < 0.1,

^*p < 0.05,

^**p < 0.01,

^***p < 0.001