Improving College Enrollment of At-Risk Students at the School Level

Abstract

Many federal, state, and local education policy priorities are aimed at preparing high school students, especially those at risk, to be college- and career-ready when they graduate from high school. A number of programs across different institutional entities have been initiated to achieve these goals, encompassing individual partnerships with schools. Many of these programs include a variety of interventions, ranging from college and course counseling to college visits. Although there have been some evaluations of the larger federal programs, and some state and district programs, few have examined national observational data on the impact of these programmatic efforts on college enrollments. This study uses the HSLS:09 database to investigate the impact of specific treatments in at-risk schools on college enrollments. Results show that several of these programmatic initiatives have a positive effect on college enrollment; however the effects are small compared to some of those reported by other national studies.

Increasing college enrollment rates among at-risk students has become a widespread policy goal (Carnevale, Smith, & Strohl, 2010; Conley, Drummond, de Gonzalez, Rooseboom, & Stout, 2011). School and district leaders, teachers, and school communities, especially those that serve a high percentage of at-risk students, are under immense pressure to show rapid improvement and significant effects on college enrollment rates for their high school graduates (Martinez & Klopott, 2005). Political and policy pressures have compelled schools to increase their college-readiness and-enrollment rates; increasingly, these indicators are being used for accountability purposes, and in some instances, gauges of school effectiveness (Callan et al., 2006).

Recently, the National Center for Education Statistics (NCES) released a new national longitudinal study that includes data on practices used by schools to support college enrollment (Tierney, Bailey, Constantine, Finkelstein, & Hurd, 2009). Many of these practices, including course and college counseling, academic tutoring, financial-aid assistance, and college visits, have been implemented by schools and their partners, including those outside and in the academy, as formal or informal programs offered during or after the school day and in the summer (Domina, 2009; Schneider, Broda, Judy, & Burkander, 2013). This study examines how effective these practices are for schools that have lower-than-average college enrollment rates and serve at-risk students.

College Enrollment Among At-Risk Populations

In 2013, the college-going rate nationally was 65.9 % (U.S. Department of Labor, 2013). However, there is great variation in the college-going rate based on sociocultural factors such as race, gender, first-generation status, family income, and geography. For example, White students enroll in 2- or 4-year colleges immediately after high school at higher rates (71.7%) than Blacks (55.7%) and Hispanics (63.9%) (Aud, Fox, & KewalRamani, 2010). Males enroll in college at lower rates than females (43 % vs. 57 %) (Knapp, Kelly-Reid, & Ginder, 2012). This gender difference is most pronounced among Blacks (36 % vs. 64%), Hispanics (42 % vs. 58%), and American Indians/Alaska Natives (40 % vs. 60%) (Aud, Fox, & KewalRamani, 2010).

Similarly, first-generation students enroll in college at significantly lower rates than their counterparts. Choy (2001) found that although 82% of non-first generation students enroll in college, only 54% of those with parents who only had a high school diploma enrolled, and just 36% of students whose parents had less than a high school diploma enrolled in college immediately after high school graduation. Low-income students enroll in college at lower rates than their middle- and high-income counterparts. In 2012, 50.9 % of low-income students enrolled in 2- or 4-year postsecondary education programs while 64.7% of middle-income and 80.7% of high-income students did the same (Snyder & Dillow, 2013). Additionally, rural students enroll in college at lower rates than their counterparts in cities and towns, even though they outperform their nonrural peers on achievement exams (Aud & Wilkinson-Flicker, 2013).

Numerous characteristics are used to classify students as at risk, including background characteristics (such as age, socioeconomic status, race, gender, cultural barriers, language barriers, and technology proficiency), internal characteristics (weak self-concept, misaligned goals, low interest), and environmental factors (learning environment, academic support, guidance counseling, travel time) (McCabe, 2003; Schneider & Stevenson, 1999). At-risk students receive less family support when applying to college, lack crucial college knowledge, are less academically prepared, have less information about college life, and are less informed about financial aid (Schneider et al., 2013; Vargas, 2004). On average, at-risk students do not have the same levels of social capital and support as other students, resulting in a deficiency of experience and knowledge related to the college-going process, contributing to lower postsecondary enrollment rates (Roderick, Nagoaka, Coca, & Moeller, 2008). Special programs such as Upward Bound (Upward Bound, 2016), informational resources, financial aid programs, outreach programs, and school counselors are key tools schools use to try to mitigate those factors and help at-risk students achieve their postsecondary goals.

College Outreach Programs

Since the onset of the TRIO programs in 1964 (Groutt, 2003), college outreach programs have proliferated to number over 1,000, reaching over 5 % of all students in public high schools and 10 % of low-income 10th graders (Domina, 2009). College outreach programs tend to emerge in one of two models: 1) targeted programs, like many TRIO initiatives, that offer services to small cohorts of students that meet certain criteria, and 2) schoolwide models, such as the College Ambition Program (CAP), available to all students within a school (Domina, 2009; Schneider et al., 2013). Both models typically include informational outreach about college and careers, as well as academic support. Although school counselors have traditionally provided college-going activities and service, for schools in low-income or under resourced areas, dedicated college counselors or coaches can help to deliver these supports. Services provided by college outreach programs often aim to counteract negative community and school factors such as a weak high school curriculum and low social capital regarding the college-going process. Schools that have a stronger college climate demonstrate the ability to make a difference for students who may be at risk (Roderick et al., 2008).

Even though college outreach programs have expanded greatly, empirical research and evaluation of their effectiveness is quite limited (Perna, 2002). Evaluations that do exist show mixed results. Domina (2009) concluded that traditional cohort-based models are ineffective at influencing educational experiences and outcomes of participants, but that schoolwide outreach programs might modestly and positively affect outcomes of unengaged students. Likewise, Hoxby & Avery (2012) found that traditional college outreach programming for high-achieving low-income students, such as providing campus visits and access to college recruiters, was ineffective at increasing students' likelihood of attending a more selective college.

In contrast, Carrell and Sacerdote (2013) found that college coaching and cash incentives late in the senior year resulted in a 15 percentage point increase in the college-going rate among women. However, the study noted no effects for men. Similarly, in a randomized experiment, Bettinger and Baker (2011) also found positive effects of students participating in coaching, with treated students more likely to be enrolled in college 1 year after receiving treatment, demonstrating an estimated 12% increase in college retention compared to non-coached students. Hoxby and Turner (2013) used a randomized control trial to study the impact of providing high-achieving, low income students with semi-customized information about college applications and cost. They found that the low-cost intervention increased students' attendance at more selective institutions. Evidence from the study suggests that the combination of services rather than any one component provides the strongest effects.

Mixed outcomes may be the result of program design that does not mirror the most effective strategies. Perna (2002) concluded that just one fourth of college outreach programs that target underrepresented students emphasize the five most critical predictors of college going, and that just 6% target the 11 most important components. Four of these components, including course and college counseling, academic tutoring and homework assistance, financial aid support services, and college visits, are used to simulate college outreach program treatment in this paper and are discussed in further detail below.

College and Course Counseling

College and course counseling activities include providing students with pertinent information about the college experience, guidance on course taking, assisting with the college search, application, and selection processes. Contextual skills and awareness are a key component to students' college preparation and readiness and can include knowledge of collegiate academic, social, and cultural expectations and norms, in addition to knowledge of the requirements necessary to garner admission to a postsecondary institution (Conley, 2007). Research has consistently shown that risk factors such as low socioeconomic status negatively influence students' access to this kind of information about postsecondary enrollment and experiences, and that limited access constrains students' search, choice, and selection processes (Cabrera & La Nasa, 2000). For example, the college-application process is complex, but it can be a particularly challenging task for at-risk students (Hoxby & Avery, 2012; Roderick et al., 2008). Research indicates that relative to their peers, at-risk students are less likely to receive college-application help from their parents (Choy, 2001; Choy, Horn, Nuñez, & Chen, 2000). However, some students report that school-based resources are helpful for encouraging the completion of necessary college-going tasks such as submitting applications (Farmer-Hinton, 2008).

School counselors are one key resource available to students throughout the college-going process. The literature is clear on their potential to positively influence college enrollment, and students commonly report having conversations with school counselors about the college-going process, including financial aid and academic planning and alignment (Horn, Chen, & Chapman, 2003). McDonough (2004, 2005) argues that school counselors are the single most important school-based professional for improving college-going rates. Counselors often guide students through the course selection process as enrollment in higher level math and science classes in high school is a critical component of college matriculation (Riegle-Crumb, 2006). They are able to affect college information sharing activities, communicate with parents regarding their place in the college-going process, support academic preparation, assist student college choice, and play an instrumental role in sharing financial aid information (Hossler, Schmidt, & Vesper, 1999; McDonough, 2005).

Even though research shows counselors' ability to influence college-going, student access to school counselors can be limited. School counselors must meet multiple demands outside of college counseling, including offering personal and behavioral counseling, attendance obligations, scheduling, and administering testing processes (Perna et al., 2008). Additionally, structural constraints, such as high student-to-counselor ratios, as well as variations across and within high schools resulting from external federal, state, and local policy pressures also hinder school counselors' ability to offer robust college-going support services (McDonough, 2005; Perna et al.; 2008). Further, while college counseling is widely understood as an obligation of school counselors, school counselor degree and certification programs often lack specific training in the area (McDonough, 2005).

Academic Tutoring

At-risk students are less academically prepared for postsecondary education than their peers (Hossler, Schmidt, & Vesper, 1999; Perna, 2000). As a result, many college outreach programs (79%) report having a goal to improve academic skills (Educational Policy Institute, 2012). They offer a wide array of services ranging from skill-based supports such as study skills training and test-taking skills to instruction in math, science, reading, and writing (Educational Policy Institute, 2012). Although some evidence suggests that these programs are effective at improving students' academic achievement (Perna, 2002), it is unclear whether this kind of programming is effective at increasing college-going. Further, even less is known about individual components of academic-oriented college outreach programs and their effect on long-term student college-going outcomes.

Financial Aid Assistance

College outreach programs often use financial aid assistance as a strategy to promote college-going. This includes providing students with information about college costs, helping students and families complete the Free Application for Federal Student Aid (FAFSA), and providing support for scholarship applications awarded by institutions, organizations, foundations, and communities. Accurate and accessible financial aid information is a key area of college outreach because the college financial aid process is complex and can be especially difficult for at-risk students. Research suggests that students from low income families overestimate college costs, particularly at public colleges and universities, and their families are less aware of college costs than the families of their higher income counterparts (Avery & Hoxby, 2004; Bettinger, Long, Oreopoulos, & Sanbonmatsu, 2012). In addition to having less information about what college costs, low income students may find out about financial aid awards too late in the college-choice process (Heller, 2006).

Further, while low income families display less knowledge of college costs and payment options, they face a financial aid process that is complex and unfriendly to the neediest students. In a review of the federal aid system, the Advisory Committee on Student Financial Aid found that many students find the intricacies of the financial aid process overwhelming, and that the financial aid process can serve to discourage rather than encourage access to postsecondary institutions (ACSFA, 2005). This complex system results in many Pell Grant-eligible students (more than 10%) failing to complete necessary financial aid forms. There is some evidence that providing information on college costs and FAFSA completion assistance can improve FAFSA completion rates and college-going for low-income students. Bettinger et al. (2012) found that providing low income families with assistance filing taxes and completing the FAFSA, along with information about college costs, increased the rate by which students complete 2 years of college by 8 percentage points.

College Visits

A survey of college outreach programs shows that 75% offer campus visits and tours (Educational Policy Institute, 2012). This is consistent with prior research which indicates that college visits are one of the most commonly offered programmatic components of college-outreach programs. These services are most likely to be offered in programs which target first-generation students (86%), and least likely to be offered at programs that target low-achieving students (56%) (Perna, 2002). Hill (2008) used data from the High School Effectiveness Study and identified college visits as experiences that help students navigate the college-linking process. Research also indicates that visiting a college campus as a high school student can familiarize students with the college environment, help students become aware of campus resources, and improve the likelihood that a student will apply and matriculate at the college visited (Klein & Washburn, 2012).

With so many programs in place, under the auspices of various types of institutional partnerships or through formalized mandates by states or districts, it is unclear if these college outreach programs have an overall impact on college enrollment for at-risk students. To explore the impact of college-outreach interventions on college enrollment, we conducted a propensity-matched regression analysis that simulated the impact of specific college-going interventions on eventual college enrollment. Below, we describe the data and methods used to conduct this analysis.

Method

Data for this study were obtained from the first wave of the High School Longitudinal Study (HSLS:09), administered to approximately 21,000 students in 944 schools in the Fall of 2009. This is a large, nationally representative dataset created by NCES that serves as a third wave of national survey sampling, following on the heels of the previous National Educational Longitudinal Study (NELS:1988) and Educational Longitudinal Survey (ELS:2002). The schools were sampled from public and private schools enrolling students in Grades 9 through 12 in the 50 United States and the District of Columbia.

HSLS:09 data include responses to three separate surveys, administered to 10th grade students, high school guidance counselors, and school administrators, to provide detailed information about each school. The counselor survey, a new addition to NCES datasets, was administered to the lead counselor in each high school included in the sample. Given counselors' unique embedded perspective in high schools, they were asked to report on various aspects of school climate, culture, and curriculum. For the current study, we make use of counselors' reports on the presence of various programs, activities, and curricular structures that are closely related to college-going. This includes financial aid programming, school-led college visits, homework help and remediation programs, and mentoring programs.

Our methodological approach proceeds in three steps: 1) develop a set of measures based on the guidance counselor survey that reflect common intervention techniques, and used these measures to define treatment and control schools; 2) use propensity score matching (PSM) techniques (Frank et al., 2008; Guo & Fraser, 2009) to obtain a matched sample of HSLS:09 schools where the treatment schools and schools were similar in terms of their characteristics; 3) examine the impact that these activities have on college-going in matched schools that employ these strategies (treatment schools), as compared to schools that do not (control schools).

To create a treatment that reflects many of the aspects of a college-readiness program, we identified a series of questions on the HSLS counselor survey that related closely to core program features identified in the research literature. These included whether or not a school offered students: 1) financial aid planning for college; 2) homework and course tutoring; 3) school-led visits to college campuses; and 4) mentoring programs focused on college planning and enrollment. All of the 130 schools included in our final analysis reported providing financial aid information about college to students. However, beyond financial aid information, schools varied more significantly in the types of support they provided. If a school provided support for all of the interventions including a tutoring/homework assistance program (Homework), college counselor designated for college readiness, selection, and/or application (Mentor), and the school organizes visits to college campuses for students (Visits) then the school was considered a treatment school. If a school did not offer all three of these programs, it was given a control designation.

Creating a Matched Sample

Having established a measure of treatment, we then created an analytical sample. As a first step, we only included high schools with less than a 62% any-college enrollment rate of high school graduates. This represents the average rate of enrollment for the treatment schools, and served as an additional attempt to create a sample of schools that might benefit from having a college-readiness program. Initially, 150 schools in the HSLS 2009 data met this criterion. Of the sample set of 150, 10 schools did not report any data. These schools were dropped from this analysis leaving 140 schools to compare. Additionally, using the propensity score matching technique (described below) we also eliminated 10 additional schools that had a propensity score of 0, or perfectly predicted failure in the model. This left us with 130 schools overall, 100 control sites and 30 treatment sites. For the full propensity match sample, 50 schools were identified for analysis; 30 of them were treated as treatment schools and 20 are control schools. These 50 schools were the most closely matched according to their covariates, and should therefore offer the strongest comparison point for detecting treatment effects.

Measures

The dependent variable for this investigation is schoolwide college enrollment rates at 4- and 2-year colleges. The rates reported were based on data collected from the National Student Clearinghouse for each school and determined by taking an average of the rate reported for 4 years. Therefore, the average college-enrollment rate for each school is based on the 4 years prior to the cohort being studied. The next wave of HSLS:09 plans to report the actual numbers for students who were surveyed in 2009, but that information will not be available until 2015 or later.

Measures of school context, the predictors of enrollment, are critical to conducting a propensity score analysis (see below), which aims to match similar schools based on a set of characteristics, including school type (comprehensive, specialized program, and alternative),¹ location (city, suburban, town, rural), socioeconomic status, English language learners (ELLs), special education population, proportion of students who repeat a grade, and racial and ethnic composition, as measured by proportion of Black, Hispanic, and White students.

Analytic Procedures

The analysis begins by employing basic techniques for analyzing treatment effects. Using a t-test, the means of high schools who report using a set of college-going practices are compared with those who do not. Then we analyze if that difference is statistically different from zero. Next we conduct an ordinary least squares (OLS) regression to measure treatment effects while concurrently controlling for other measures of school context that may bias our estimation.

Using only t-tests and OLS regression to estimate a treatment effect would fail to recognize that a school's treatment assignment may be confounded with many other factors, such as a school's socioeconomic or racial or ethnic composition. For example, schools in more affluent communities (with higher SES levels and more students attending college) may be more likely to report having these practices in place relative to schools in less resourced communities.

To strengthen our inference, we turn to the statistical matching literature (Abadie & Imbens, 2009) to match schools or units using propensity scores. First we identify a set of covariates to match, including degree to which the school is urban, schoolwide socioeconomic status, and racial or ethnic composition. Then, using logistic regression, we employ these covariates to predict assignment to the treatment. The predicted values from this regression, which range from 0 to 1, can be interpreted as an individual schools' propensity to report the treatment. Each school in the HSLS:09 sample is thus assigned a propensity value, to match it with the most similar schools. Although PSM techniques do not eliminate the risk of confounding variables such as household address (an obstacle in all observational studies, see Rosenbaum & Rubin, 1983), they do further strengthen statistical controls by simultaneously accounting for a large set of covariates that when combined may capture the most potential confounders.

Having generated propensity scores for the matched sample of 50 schools, we then compared treatment and control schools that had the same or similar propensity scores to estimate the treatment effects. The data analysis was conducted in STATA 13 using the “teffects psmath” package which assists in generating propensity scores and in calculating treatment effects. This package uses nearest neighbor matching methods (Abadie & Imbens, 2009), an approach that pairs each school with its closest match taking into account their covariates.

To estimate the treatment effects, after the propensity score matching, we used ordinary least square (OLS) regression analysis on the matched sample. Specifically, the model is

$$Y_i={\beta }_0+{\beta }_1{\mathit{\text{TREAT}}}_i+{\mathbf{X}}_{\mathbf{i}}{\mathbf{\beta }}_{\mathbf{2}}+{\epsilon }_i$$ (1)

where Y_i is the outcome (college enrollment) for school i, TREAT_i is a binary variable coded as 1 if school i is in treatment group and 0 otherwise, X_i is a vector of school covariates, and ε_i is the error term. The school covariates includes a group of binary school-type indicators (comprehensive school is the reference group), a group of binary school location indicators (city school is the reference group), proportion of low-SES students, proportion of ELL students, proportion of special education students, proportion of Hispanic students, proportion of White students, proportion of Black students, proportion of American Indian or Alaska native students, and proportion of students who repeated a grade. We estimated Equation 1, above, with three different outcomes (4-year, 2-year, and any college enrollment²) using the school sample weights recommended by NCES (Zanutto, 2006).

Because the primary outcomes are percentages of students that went to college, the measure is bounded between 0 and 1. One issue with binary outcomes in this context is that the predicted values from OLS regressions can lie out of the unit range (either above 1 or below 0). To overcome this potential drawback, we estimated the treatment effects using fractional logistic regression models with the same covariates as those in Equation 1 (Papke & Wooldridge, 1996; Wooldridge, 2010). Specifically, we estimated the following equation, which uses a logit link function:

$$Y_i=G({\mathbf{Z}}_{\mathbf{i}}\mathbf{\beta })=\frac{1}{1+exp(-{\mathbf{Z}}_{\mathbf{i}}\mathbf{\beta })}$$ (2)

Where Z_iβ = α₀ + α₁TREAT_i + X_iα₂ + ξ_i. ξ_i is a random error and all the other covariates are the same as defined in Equation 1. It should be noted that α₁ cannot be interpreted as the marginal treatment effect as one might interpret the coefficients from linear regression models. Further, β₁ from this equation is only an approximate estimate of the average treatment effects on the matched sample. To obtain more precise estimates of the average treatment effect on the matched sample, we use the following equation:

$$\tau =N^{-1}\sum _{i=1}^N[G({{\mathbf{Z}}^{(\mathbf{1})}}_{\mathbf{i}}\mathbf{\beta })-G({{\mathbf{Z}}^{(\mathbf{0})}}_{\mathbf{i}}\mathbf{\beta }]$$ (3)

where Z⁽¹⁾_i has TREAT_i=1 and Z⁽⁰⁾_i has TREAT_i=0. (See Wooldridge, 2010 for a more detailed discussion.) We take advantage of the margin command in STATA to evaluate the average treatment effect that can also be used to compare the OLS estimate (β₁) in Equation 1.

Results

Descriptive Statistics

Table 1 summarizes descriptive statistics for the analytical sample, including outcome measures and school and student characteristics. On average, 4-year enrollment, 2-year college enrollment and any college enrollment were 23%, 22.5% and 45.5%, respectively. More than half of the schools were rural schools. Nearly 95% of schools were comprehensive high schools. On average, about 57% students were eligible for free or reduced-price lunch in each school. More than 60% of the students were White. Table 1 also presents the means and standard deviations of the outcome measures and covariates for matched treatment and control schools separately. In addition, we used t-tests to compare differences. We find that treatment schools had a significantly higher 4-year college enrollment and any college enrollment rate.

Table 1. Descriptive Statistics for Analytic Sample.

Variables	Full Sample		Treatment Group		Control Group		t
Outcome	Mean	SD	Mean	SD	Mean	SD
Four-year College-going Enrollment	0.230	0.113	0.285	0.082	0.215	0.116	2.78*
Two-year College-going Enrollment	0.225	0.100	0.250	0.093	0.218	0.101	1.15
Any College-going Enrollment	0.455	0.149	0.536	0.070	0.433	0.157	2.94*
School Characteristics
City School	0.100	0.301	0.115	0.324	0.096	0.296	0.27
Suburban School	0.118	0.324	0.158	0.371	0.107	0.311	0.58
Town School	0.208	0.408	0.189	0.399	0.213	0.412	-0.15
Rural School	0.574	0.496	0.538	0.508	0.583	0.496	-0.24
School Type
Comprehensive High School	0.948	0.222	0.937	0.247	0.951	0.216	-0.27
Special Program School	0.020	0.141	0.039	0.197	0.015	0.123	0.73
Vocational School	0.032	0.175	0.024	0.155	0.034	0.181	-0.25
Student Characteristics – Proportion of:
Low-SES Students	0.570	0.196	0.593	0.171	0.564	0.202	0.54
ELL Students	0.053	0.108	0.064	0.129	0.050	0.101	0.49
Special Ed Students	0.153	0.077	0.154	0.096	0.153	0.072	0.03
Hispanic Students	0.182	0.260	0.101	0.195	0.203	0.271	-1.65
White Students	0.622	0.342	0.636	0.361	0.619	0.338	0.14
Black Students	0.160	0.261	0.223	0.293	0.143	0.251	0.70
American Indian or Alaska Native Students	0.022	0.130	0.002	0.005	0.027	0.145	-1.16
Who Repeated Grades	0.048	0.072	0.083	0.104	0.039	0.058	1.57
Sample
Full	130 (30 treatment)
Matched	50 (30 treatment)

Note. NCES-recommended weights are used.

^*p < 0.05.

Table 2 presents the treatment effects from a linear regression model (before propensity score matching) as well as a fractional response model using the full analytic sample of 130 schools. Columns 1 to 3 present the results from the OLS regression. We find that the treatment effect was significant and positive at the .05 level for both 4-year enrollment and any college enrollment. The treatment was positive but insignificant for 2-year college enrollment. Columns 4 to 6 present the results from the fractional response model. The results were quite similar to those from linear regression models. To facilitate easier interpretation, in Table 2-1 below we calculated the average treatment effects based on the results from the fractional response models.

Table 2. Regression Estimates of Treatment Effect, Before Propensity Score Matching.

Variables	Linear Regression Fractional Response Models			Fractional Response Models
	(1)	(2)	(3)	(4)	(5)	(6)
	Four-Year Enrollment	Two-Year Enrollment	Any Enrollment	Four-Year Enrollment	Two-Year Enrollment	Any Enrollment
Treatment	0.042* (0.020)	0.038 (0.028)	0.080* (0.032)	0.230* (0.107)	0.226 (0.150)	0.333* (0.126)
Suburban School	-0.009 (0.037)	0.042 (0.050)	0.03 (0.041)	-0.057 (0.201)	0.223 (0.259)	0.130 (0.162)
Town School	0.057 (0.035)	0.009 (0.043)	0.066 (0.049)	0.288 (0.187)	0.051 (0.235)	0.261 (0.193)
Rural School	-0.015 (0.035)	-0.024 (0.041)	-0.039 (0.044)	-0.103 (0.194)	-0.156 (0.231)	-0.171 (0.173)
Special Program School	0.064* (0.032)	0.063 (0.052)	0.127* (0.044)	0.329* (0.165)	0.346 (0.281)	0.514* (0.177)
Vocational School	0.025 (0.047)	0.067 (0.053)	0.093 (0.058)	0.165 (0.312)	0.392 (0.292)	0.398+ (0.240)
Student Characteristics – Proportion of:
Low-SES Students	0.037 (0.097)	-0.134 (0.090)	-0.097 (0.113)	0.131 (0.538)	-0.764 (0.489)	-0.437 (0.462)
ELL Students	-0.047 (0.191)	0.261* (0.117)	0.214 (0.212)	-0.237 (1.279)	1.430* (0.653)	0.922 (0.864)
Special Ed Students	-0.292 (0.172)	0.038 (0.179)	-0.254 (0.305)	-1.607 (1.045)	0.313 (1.049)	-0.998 (1.228)
Hispanic Students	-0.355* (0.165)	0.307 (0.196)	-0.048 (0.176)	-2.076* (0.952)	1.633 (1.101)	-0.208 (0.691)
White Students	-0.148 (0.184)	0.335 (0.214)	0.187 (0.199)	-0.767 (1.125)	1.810 (1.197)	0.787 (0.798)
Black Students	-0.293 (0.195)	0.290 (0.216)	-0.004 (0.231)	-1.586 (1.202)	1.459 (1.258)	-0.021 (0.950)
American Indian or Alaska Native Students	-0.401* (0.196)	0.343 (0.207)	-0.058 (0.218)	-2.422* (1.194)	1.846 (1.169)	-0.237 (0.886)
Students Repeated Grades	0.173 (0.166)	0.035 (0.242)	0.208 (0.170)	1.049 (0.847)	0.276 (1.303)	0.912 (0.674)
N	130	130	130	130	130	130

Note. Standard errors in parentheses.

^‡Row indicates proportion of each group

^*p < 0.05.

⁺p < 0.10.

Table 2-1. Average Treatment Effects on the Treated Based on Fractional Response Models in Table 2.

Outcome	Average Treatment Effect	SE	z
Four-year Enrollment	0.040*	0.018	2.22
Two-year Enrollment	0.039	0.026	1.52
Any Enrollment	0.079*	0.029	2.69

^*p < 0.05.

⁺p < 0.10.

The average treatment effects were about 0.040 for 4-year enrollment and 0.079 for 2-year enrollment, and both were significant at the .05 level. These effects are presented as mean differences in enrollment, with 0.04 corresponding to about a 4% increase in 4-year college going, and 0.079 corresponding to about an 8% increase in any college going. The average treatment effects for 2-year enrollment were not found to be significantly different from 0. These initial results, useful as a preliminary benchmark, will be supported by additional models with more rigorous approaches.

Table 3 presents the differences in means for our propensity-matched sample of 43 schools. In short, we find no statistically significant differences in covariates between the treatment schools and control schools in the matched sample. This suggests that the propensity match process appropriately matched treatment and control schools with similar characteristics.

Table 3. Difference in Means for Propensity-Matched Sample.

School Characteristics	Treatment Group Mean	Control Group Mean	t
Suburban School	0.286	0.200	0.600
Town School	0.071	0.133	-0.650
Rural School	0.321	0.400	-0.500
Special Program School	0.107	0.067	0.430
Vocational School	0.036	0.000	0.730
Student Characteristics – Proportion of:
Low-SES Students	0.568	0.517	0.840
ELL Students	0.096	0.071	0.540
Special Ed Students	0.146	0.138	0.340
Hispanic Students	0.160	0.175	-0.210
White Students	0.570	0.627	-0.550
Black Students	0.189	0.157	0.460
American Indian or Alaska Native Students	0.007	0.012	-1.430
Students Repeated Grades	0.110	0.057	1.720

Table 4 summarizes the treatment effects using the propensity-matched sample. Columns 1 to 3 present the results from linear regression models. We find no significant treatment effects on the matched schools for 4-year college enrollment or any college enrollment. The treatment effect for 2-year college enrollment is about -0.09 and significant at the.05 level. Because the coefficients from linear regression only provide an approximate estimate of the average treatment effects on the matched sample (Wooldridge, 2010), we check the robustness of these findings using fractional logistic regression models. Their results are summarized in Columns 4 through 6. We find a positive and borderline significant (at the .10 level) treatment effect on 4-year college enrollment. The treatment effect on 2-year enrollment was significant and negative at the .05 level. The coefficient for treatment on any college enrollment was negative but insignificant.

Table 4. Estimated Treatment Effect Using Propensity-Matched Sample.

Covariates	Linear Regression Model			Fractional Response Model
	(1)	(2)	(3)	(4)	(5)	(6)
	Four-Year Enrollment	Two-Year Enrollment	Any Enrollment	Four-Year Enrollment	Two-Year Enrollment	Any Enrollment
CAP-like Treatment	0.050 (0.036)	-0.089* (0.032)	-0.039 (0.050)	0.270+ (0.155)	-0.460* (0.138)	-0.158 (0.164)
Suburban School	-0.001 (0.048)	0.044 (0.042)	0.043 (0.045)	-0.016 (0.215)	0.205 (0.174)	0.175 (0.148)
Town School	0.102* (0.039)	-0.005 (0.051)	0.097* (0.046)	0.520* (0.176)	-0.049 (0.224)	0.392* (0.150)
Rural School	0.017 (0.057)	0.081 (0.060)	0.098+ (0.056)	0.063 (0.262)	0.403 (0.251)	0.395* (0.184)
Special Program School	0.065 (0.062)	-0.032 (0.065)	0.032 (0.068)	0.359 (0.295)	-0.117 (0.266)	0.133 (0.222)
Vocational School	0.077 (0.102)	-0.212 (0.142)	-0.136 (0.128)	0.381 (0.468)	-1.502* (0.577)	-0.547 (0.421)
Student Characteristics – Proportion of:
Low-SES Students	-0.049 (0.135)	0.207 (0.125)	0.158 (0.159)	-0.252 (0.595)	1.127+ (0.600)	0.638 (0.521)
ELL Students	-0.085 (0.219)	-0.154 (0.333)	-0.239 (0.313)	-0.574 (0.981)	-0.905 (1.346)	-0.965 (1.033)
Special Ed Students	0.127 (0.161)	-0.321+ (0.166)	-0.195 (0.174)	0.891 (0.768)	-1.503* (0.739)	-0.786 (0.572)
Hispanic Students	-0.337 (0.309)	0.043 (0.382)	-0.293 (0.370)	-1.642 (1.364)	0.509 (1.570)	-1.182 (1.220)
White Students	-0.264 (0.389)	-0.034 (0.469)	-0.298 (0.429)	-1.296 (1.739)	0.087 (1.915)	-1.203 (1.413)
Black Students	-0.400 (0.326)	0.081 (0.427)	-0.319 (0.411)	-2.080 (1.457)	0.644 (1.731)	-1.287 (1.355)
American Indian or Alaska Native Students	1.197 (2.372)	-1.403 (1.712)	-0.206 (2.591)	6.565 (9.585)	-6.575 (7.632)	-0.830 (8.536)
Students Repeated Grades	-0.144 (0.192)	0.417* (0.192)	0.273 (0.182)	-0.999 (0.936)	1.921* (0.744)	1.104+ (0.599)
N	50	50	50	50	50	50

Note. Standard errors in parentheses

^*p<0.05.

⁺p<0.10.

Based on the results from the fractional logistic regression, in Table 4-1 below calculated the average treatment effects on the matched sample using equation (3)

Table 4-1. Average Treatment Effects on the Treated Based on Fractional Response Models in Table 4.

Outcome	Average Treatment Effect	SE	z
Four-year Enrollment	0.054+	0.031	1.76
Two-year Enrollment	-0.086*	0.025	-3.42
Any Enrollment	-0.039	0.041	-0.97

^*p < 0.05.

⁺p < 0.10.

The average treatment effect for 4-year enrollment was about 0.054, or 5.4%, and significant at the .10 level. This indicates that the treatment increased 4-year enrollment by about 5% on average. The effect for 2-year enrollment was about -0.086, or -8.6%, and was significant at the.05 level. This suggests that the treatment decreased 2-year college enrollment by about 9% on average. The treatment effect for any college enrollment was -0.039 but insignificant.

In summary, the fractional regression and OLS regressions show that when schools used at least three of these college-going practices, the expected increase in 4-year college enrollment is approximately 5 %, however, 2-year enrollment decreases by at least 8 %. This suggests that the college enrollment rate does not increase substantially but that there is a shifting within choice of college destination.

Discussion

Although this analysis included simulation of treatment components that are often a part of college-outreach partnerships, it shows that schools can engage in key components of college outreach programs, and receive positive results, without having to develop a formal partnership with an external program. This is good news for school districts serving at-risk students with limited budgets. Although formal partnerships may help schools improve the quality of these efforts through access to additional resources, evidence from the activities of current school counselors in this study suggest that even within the population of schools with lower college-enrollment rates, some counseling staff have found ways to integrate an array of college-preparation activities that appear to have at least some positive impact on their students' enrollment patterns.

Further, although positive, the results of this study also should serve to temper the expectations for these activities and for the results of college-outreach partnerships. Although engaging in the four outreach components shows positive results, they are modest in magnitude. High-need schools serving at-risk students present educators and researchers with a host of challenges, and college-outreach programming, whether through formal partnerships or activities undertaken at the direction of school counselors, may prove to have modest but significant effects for addressing the problem of low college-going rates among at-risk students.

Additionally, the results suggest a possible dilemma of purpose inherent in building capacity for increased college enrollment in high-needs schools. Moving from the traditional OLS model (Tables 2 and 2-1), which reported positive effects for both 2- and 4-year enrollment, to the stronger propensity match comparison (Tables 4 and 4-1), we see that an apparent positive effect on 4-year college enrollment is balanced with a slightly negative effect on 2-year enrollment. Several explanations for this tradeoff merit consideration, especially the possibility that increased pre-college outreach may be more effective at encouraging students who would otherwise enroll in 2-year colleges to consider 4-year options, thus simultaneously driving the reduction in 2-year and an increase in 4-year enrollment.

At the same time, this suggests that current efforts may not be as effective at encouraging students who would otherwise not attend any college to enroll. Given the urgency of working with this particular population, this may suggest that other types of assistance may be required to see positive effects from these students. In the end, however, the HSLS dataset in its current form lacks the additional student-level enrollment information to parse out these subgroup impacts. Future research, possible once the data collection is complete over the next few years, could shed more light on this dilemma.

A final limitation to note is the broad definition of each outreach activity found in the counselor survey. For example, all schools in both the treatment and control groups indicated that they provide financial aid assistance. In our experience in the field, we have observed many different types of financial aid support, executed with very different levels of effectiveness. This includes simply providing a FAFSA form and more intensive activities like one-on-one support, parental coaching, or hosting schoolwide information sessions. On the counselor survey, no differentiation is made between schools (or programs) that offer one of the above versus schools that offers all of the above services. This suggests that more data are needed to capture the depth and breadth of services offered within a particular area of outreach.

Another limitation of this work is that it is based on observational data, and is not a true randomized trial. Even the most rigorous trials are bound to have problems with fidelity of implementation and eventual scale-up to diverse populations and multiple sites. Nevertheless, we would recommend that researchers continue to design and implement such designs and take into account what motivates all types of at-risk students to pursue further training and education. The real issue, it seems, is providing a pre-college outreach that supports a wider menu of education and training opportunities for at-risk students while being more cautious about interpreting the results of those interventions.