Short- and Long-Term Impacts of Engagement Experiences with Faculty and Peers at Community Colleges*

Abstract

This study uses nationally representative longitudinal data and propensity score modeling to evaluate the effects of first-year engagement experiences at community college—including social and academic contact with faculty and participation in study groups and clubs—on achievement, persistence, degree attainment, and vertical transfer. Speaking with faculty about academic matters improved short- and long-term outcomes, while engaging in study groups and clubs improved early outcomes, with less sizeable long-term impacts. The findings highlight the need for continued inquiry into community college engagement using longitudinal data with detailed engagement and outcome measures to determine best practices.

Community colleges increase students’ access to postsecondary education but are plagued by high rates of non-completion (Bailey, Jaggars, & Jenkins, 2015; Bound, Lovenheim, & Turner, 2010). Among students who initially enroll at public two-year institutions, 46 percent drop out within three years, thwarting degree attainment or transfer to a four-year institution and, for many, the promise of social mobility (NCES, 2011). Within six years, only 14 percent of community college entrants earn an associate’s degree and 12 percent earn a bachelor’s degree (Schudde & Goldrick-Rab, 2016).

Recent efforts confirm that engaging with faculty and peers integrates community college students into campus life (Barnett, 2011; Deil-Amen, 2011; Karp, Hughes, & O’Gara, 2010). Yet, to inform effective updates to institutional policy and practice, we need stronger evidence regarding whether engagement experiences with faculty and peers improve short- and long-term student academic outcomes and which experiences are most effective.

To build empirical evidence in this regard, this study examined the following first-year engagement experiences: social contact with faculty; speaking with faculty about academic matters outside of class; studying with peers; and participating in school clubs. Using propensity score modeling and rich nationally representative data, I modeled selection into each experience to capture students’ propensities to engage. Leveraging the selection models, I estimated the effects of engagement experiences on first-year grade point average, retention into the second year, associate degree attainment, and transfer to a four-year college within six years.

The study contributes to the literature on the impact of engagement on community college student outcomes in several ways. First, the analyses included various demographic and academic background measures unavailable in other studies, relying on the rich survey data from the Beginning Postsecondary Students Longitudinal Study. The longitudinal nature of the dataset allowed me to double the length of follow up compared to previous studies on the impact of engagement on outcomes like degree attainment and transfer. Finally, leveraging quasi-experimental methods produces a less biased estimate of the impact of engagement experiences by controlling for student selection into engagement.

Engaging with faculty about academics outside of class improved both short- and long-term student outcomes. Engaging with other students in study groups improved student persistence into the second year, with less sizeable long-term impacts. The paper concludes with a discussion of the implications, including the need for longitudinal data that captures more detailed engagement measures, in addition to background and outcome measures, to determine best practices.

Engagement in the Community College Context

Community colleges enroll a diverse student body, comprised of students from an array of backgrounds with varying educational goals (Bailey et al., 2015; Schudde & Goldrick-Rab, 2015). While demographics and academic preparedness partially explain low rates of degree attainment and transfer, the community college experience is also accompanied by significant navigational challenges for students (Rosenbaum, Deil-Amen, & Person, 2007). Institutional barriers, including complex bureaucratic processes and insufficient support services, impede students’ academic progress, especially for students faced with competing demands of work, family, and school (Bailey et al., 2015 ; Brock & LeBlanc, 2005; Rosenbaum et al., 2007). Community colleges offer varied choices, including competing classes, course sections, instructors, and majors (Bailey et al., 2015).

Research argues that increased student support improves students’ navigation of bureaucratic hurdles, varied choice sets, and conflicting demands, ultimately increasing their success (Deil-Amen & Rosenbaum, 2003; Karp, 2011). Given budget constraints, institutions cannot always offer the structure and guidance of one-on-one advising that may be necessary to support students in making informed choices. Interactions with faculty and peers may serve as mechanisms through which students can receive support and validation, beyond traditional support services like academic advising (Barnett, 2011; Deil-Amen, 2011). If engagement with faculty and peers fills this advising gap and improves success, it may represent a missed opportunity to improve outcomes for those who do not engage with their peers and instructors.

In this study, the phrase “engagement experiences” serves to broadly capture the campus interactions that community college students have with faculty and peers. The construct of engagement has taken on varied meanings and models over the span of several decades. The Community College Survey of Student Engagement (CCSSE) refers to engagement as the “amount of time and energy that students invest in meaningful educational practices,” similar to that use by the National Survey of Student Engagement, CCSSE’s predecessor and four-year counterpart (Kuh, Cruce, Shoup, Kinzie, & Gonyea, 2008; McClenney, 2007). Harper and Quaye (2009) broadly define engagement as “participation in educationally effective practices” that occur both inside and outside the classroom (p. 3). There are several models of engagement in campus life, each with their own nuances, aims, and names: Astin’s (1993) involvement, Tinto’s (1993) integration, Kuh’s (2001) student engagement.¹ The models emerged in the four-year college setting and each theorize how engaging in college life impacts student development.

There is growing interest in applying theories of engagement to two-year colleges. To determine whether engaging with faculty and peers directly influences student outcomes, it is essential to understand which factors influence participation in engagement experiences at community colleges. The reason for this step is practical: if students with a high probability of success are also more likely to engage, then descriptive patterns of engagement are likely influenced by student selection into engagement experiences. To effectively control for systematic variation in the type of student who engages, it is necessary to understand the characteristics that influence student selection into engagement. In the next section, I organize research on selection into engagement in the two-year setting in order to develop a theory-driven statistical model of how students select into engagement experiences.

Who Engages with Faculty and Peers at Community College?

There is a robust literature on who engages at community college campuses, mostly due to the availability of detailed information on community college experiences. Much of the research on community college student experiences relies on data from the annual Community College Survey of Student Engagement (CCSSE), which offers insight on the distribution of engagement at participating colleges. CCSSE aims to help institutions evaluate the quality of student experiences in order to inform institutional efforts to improve student learning and retention (CCSSE.org, 2016). The survey items capture educationally meaningful experiences that should theoretically influence student learning, achievement, and persistence, including active participation in class, collaboration with peers, and interactions with faculty.

Findings from CCSSE suggest that engagement varies across race and gender. In CCSSE-participating institutions, racial minority students, with the exception of Asian Americans, are more likely than whites to be engaged with faculty and staff (CCSSE, 2005; Greene, Marti, & McClenney, 2008). Gender also influences participation in engagement experiences, but in different ways across different types of interactions. Women are more likely than men to communicate with an instructor or advisor in terms of logistics (grades, academic progress, career plans), while men are more likely to discuss ideas from coursework with instructors and to work with them on activities outside of coursework (CCSSE, 2013c).

Intensity of enrollment and age also influence engagement. Full-time enrollees are more engaged with faculty, staff, and other students than part-time students (CCSSE, 2013b). This is likely partially related to demographics, as full-time enrollees spent less time working for pay and caring for dependents (CCSSE, 2013b). Older students (those 25 years of age or older) spend more time caring for dependents, but they also report spending more time preparing for class, more frequently asking questions during class, and fewer absences than their traditionally-aged peers (18- to 24-year-olds) (CCSSE, 2013a).

According to recent reports, two-thirds of community college students are underprepared for college-level coursework (CCSSE, 2016). These students often must enroll in “developmental” (remedial) coursework prior to college-level coursework. Students enrolled in remedial courses appear to be more engaged with faculty and peers, though it is unclear whether this is a function of course design or academic preparedness (CCSSE, 2016). Remedial classes may encourage more engagement than traditional coursework, but students who are less prepared may also be more likely to ask questions.

Socioeconomic status is also likely to influence engagement, but there is little empirical research to support this relationship in the community college context. Data sets that focus primarily on community college engagement, like CCSSE, include limited information on socioeconomic status. Research on four-year college students shows that students from low-income families are more likely to work for pay during college than their peers (Belley & Lochner, 2007; Bozick, 2007). Pressure to work and family obligations likely impact time allocation, leaving less time to interact with faculty and peers. Need-based financial aid, a policy endeavor to offset these challenges, may encourage engagement and subsequent outcomes by alleviating the need to work for pay and related time constraints (Boatman & Long, 2016; Broton, Goldrick-Rab, & Benson, 2016; Nora, Barlow, & Crisp, 2006). While research in four-year college settings suggests that financial constraints predict participation in campus engagement, it is unclear if they are also predictive in the community college setting.

Effects of Engagement Experiences in Community College Settings

In this section, I examine extant research on the link between engagement experiences and various student outcomes, highlighting the need for stronger empirical evidence. Beginning with early work on student involvement (Astin, 1993) and integration (Tinto, 1993), higher education research consistently touts the benefits of engaging in campus life (Carini, Kuh, & Klein, 2006; Flynn, 2014; Kuh et al., 2008). While the literature largely focuses on the effects among four-year college students, engagement experiences on campus may be as useful for community college students as their four-year counterparts (Deil-Amen, 2011; Karp, 2011; Karp et al., 2010). In-depth contact with faculty may play a salient role in the student experience, as mentorship has a positive impact on community college students’ ability to integrate socially and academically and on their commitment to earn a degree (Crisp, 2010). Contact with other students can increase social support, self-esteem, and social competence, improving persistence and transfer readiness (Johnson, 2006; Napoli & Wortman, 1998).

An early examination of Tinto’s (1993) theory in the community college setting considered the role that social and academic integration play in the pathway to student persistence (Braxton, Hirschy, & McClendon, 2004; Braxton, Shaw Sullivan, & Johnson, 1997). The research found support for just one of Tinto’s original propositions in the two-year setting, that student entry characteristics predict students’ persistence in college. In contrast, neither academic nor social integration were strongly associated with student departure. The authors concluded that more research is necessary to understand whether Tinto’s theory accounts for student departure at community colleges (Braxton et al., 2004, p. 18).

Recent research heeds that call, applying and adapting Tinto’s theory of integration to the community college experience. Deil-Amen (2011) pushed the literature beyond the consideration of “academic” versus “social” experiences, arguing that Tinto’s bifurcated model of integration may not translate to two-year colleges, where the lines between social and academic blur. Using qualitative data from several two-year colleges, she found that student perceptions of what helped them integrate into campus were tied to institutional actors who facilitated the process—instructors, support staff, and other students were instrumental to integration. Her work suggests that “socio-academic” moments, in which social and academic integration are simultaneous, are fundamental to feelings of social comfort at community college. Integration transpires when within-classroom interactions deliver academic knowledge while enhancing students’ “feelings of college belonging, college identity, and college competence” (Deil-Amen, 2011, p. 73).

Likewise, Barnett (2011) found that when students are validated by their instructor—“made to feel recognized, respected and valued”—in the classroom, they increase their intention to persist (p. 194). Although Rendón (1994, 2002) presented validation as an alternative explanation for persistence to Tinto’s conception of integration, Barnett (2011) argues that validation may actually be a precondition for integration (p. 196). According to Barnett, discussing students’ goals, showing appreciation for their personal and cultural backgrounds, and exerting effort to help students learn course material are examples of validation as a precondition of integration. Faculty behaviors like these may serve to prime students for the socio-academic moments on community college campus that Deil-Amen (2011) describes.

Deil-Amen (2011) argued that socio-academic moments create the sense of membership that integrates students into the campus community and should ultimately improve persistence. While she emphasized the importance of in-class interactions, she notes five common mechanisms for socio-academic integrative experiences, including 1) in-class interactions and dynamics, 2) formal or spontaneous study groups, 3) social-capital relevant interactions and mentor relationships with faculty and staff, 4) access to communication with similar students, and 5) academically-relevant clubs and activities (Deil-Amen, 2011, p. 81).

Deil-Amen (2011) and Barnett’s (2011) work proved useful in extending student development theories to new postsecondary contexts but focused primarily on the process of engaging or integrating into campus life, what it looked like in the community college context, and how it influenced student intentions and interpretations of their community college experience and goals. Both studies focused on cross-sectional data—they were only able to examine students’ intentions, rather than following students over time to understand how these processes subsequently impact student achievement, persistence, and attainment.

Of the extant literature, the only longitudinal studies of community college experiences capture student outcomes within three years of entry. The CCSSE validation studies leveraged the survey’s measures of engagement experiences both in and out of the community college classroom, along with transcript data from three sets of institutions: Florida community colleges, Hispanic Association of Colleges and Universities participating institutions, and Achieving the Dream (ATD) participating institutions. McClenney and Marti (2006) used data from the three samples to demonstrate that the CCSSE benchmarks are correlated with a variety of student outcomes. Both the Florida and ATD results suggested that student-faculty interactions and collaborative learning with peers positively impacted associate degree attainment within three years of entrance (McClenney & Marti, 2006, Table A5 and B13). The impacts on cumulative GPA were mixed, with faculty interactions positively impacting GPA in all three data sets, while collaborative learning only had a significant impact in Florida. However, only the ATD validation study included control measures such as gender, race, and enrollment intensity. Variation in effects across the data sources may be due to selection bias.

Marti (2008) evaluated the validity of the CCSSE’s survey for capturing engagement using a pooled sample of all three data sources. Testing a theoretically driven model and model of best fit, he found a positive correlation between the benchmarks and GPA. However, Marti also acknowledged the need to include additional student measures to control for students’ behaviors and attitudes, as well as longer-term outcomes to capture the impact on students’ long-term educational pathways (p. 334).

Summary

While there is some support for the correlation between engagement at community college and student outcomes, the causal effects of various engagement experiences on short- and long-term outcomes among community college students have yet to be established. Qualitative efforts like Barnett (2011) and Deil-Amen’s (2011) focused primarily on whether existing theory is relevant in the community college setting and illustrated that engagement experiences look different than those envisioned in residential college settings. Quantitative analyses like those using CCSSE data included fine-grained engagement measures, but focused primarily on validating the measures and lacked background data and long-term outcomes necessary to rigorously evaluate their impact. Adequately understanding the relationship between engagement and outcomes requires controlling for variables beyond those measured in CCSSE, including a variety of background factors like socioeconomic status (Marti, 2008, p.16). Furthermore, the longest follow-up period in the extant literature—three years in the CCSSE validation studies—does not lend itself to tracking students through their college careers, as many community college students take long and meandering pathways through college (Attewell & Lavin, 2007; Bailey et al., 2015). To contribute to the literature, I used nationally representative, longitudinal data to examine whether engaging with faculty and peers increases achievement, persistence, attainment, and transfer for community college students, controlling for factors that influence selection into engagement using propensity score matching.

Methods

Data

To examine the effects of community college engagement experiences on first-year GPA, retention into the second year, earning an associate degree, and transferring to a four-year college, this study used the Beginning Postsecondary Students (BPS) Longitudinal Study 2004/2009, a large, nationally representative study of postsecondary students and institutions. The dataset, collected by the National Center for Education Statistics (NCES), surveyed first-time college students throughout their college career, gathering data on demographics, education and work experiences, family circumstances, and student experiences. Given its rich background data (financial, social, and academic measures), measures of college experiences, and longitudinal design (including reliable outcome measures drawn from transcripts), the BPS is well suited to examine selection into engagement experiences and their effects on outcomes.

The BPS is representative of first-time college students who enrolled in postsecondary institutions in Fall 2003. Members of the BPS: 04/09 cohort were initially surveyed at the end of their first academic year (2003–04) and received invitations to participate in follow-up surveys three and six years later, during the 2005–06 and 2008–09 academic years. The longitudinal nature of the dataset makes it especially useful for tracking the outcomes of community colleges students, as it allows six years for associate degree receipt or vertical transfer. Community college students take a variety of pathways through college, resulting in long pathways to degree completion, when it occurs (Attewell & Lavin, 2007). Attewell and Lavin (2007) suggest ten years as an ideal follow-up period for students attending broad-access institutions, but, unfortunately, postsecondary national longitudinal datasets do not use such a long follow-up period. While six years may not capture attainment for some students, it allows for the analysis of some long-term outcomes and increases the length of follow-up from the current literature.

The BPS dataset includes nearly 16,700 students. To understand the impact of engagement experiences on community college students, I restricted the sample to students attending two-year colleges in the public sector (n=7,040).² I eliminated 30 students without baseline survey data (they lacked adequate information to impute missing data), winnowing the analytic sample to 7,010 students. The results are generalizable to first-time college students attending public two-year institutions who at least partially completed the initial survey. To preserve a viable sample size for analysis and retain data from cases missing only some information, I performed multiple imputation (MI). MI relies on the assumption that non-response probabilities do not depend on unobserved information, creating several completed copies of the data set (in this case, 10 copies) in which missing observations are replaced by plausible values instead of assuming one “true” response model (Royston & White, 2011). This process adds variability to the analytic model to guard against artificially precise standard errors (Cox, McIntosh, Reason, & Terenzini, 2014). I combined the estimates from imputed datasets using Rubin’s (1987) rules and present the resulting set of average estimates.

Measures

Operationalizing “engagement experiences.”

To capture student participation in engagement experiences, I used measures of campus interactions with faculty and peers during the first year of college. I focused on four indicators, including whether students: (1) experienced informal or social contacts with faculty members outside of the classroom or office; (2) spoke with faculty members about academic matters outside of class; (3) attended study groups; and (4) participated in school clubs. The measures align well with the mechanisms proposed by Deil-Amen (2011) as producing socio-academic integration, though I am unable to capture in-class interactions. While recent research using the BPS to examine the impact of engagement on outcomes at four-year colleges focused exclusively on its social and academic integration indexes (Flynn, 2014), the measures comprising the indexes were not highly correlated within the two-year sample. While the academic and social integration indexes may intend to capture Tinto’s integration construct³, there is little evidence that the scale items reliably or validly capture the construct. Survey and data documentation provided by NCES offer no information about item development or tests for scale reliability or validity (Wine, Natasha, Wheeless, & Hunt-White, 2011). Producing an index from the four relevant items also does not offer adequate internal consistency, resulting in a cronbach alpha, a measure of how closely related a set of items are, of .49—well below the acceptable standard of .7. Additionally, for my purposes, using composite measures may mask which engagement experiences drive observed effects. Therefore, I used the four engagement measures as distinct “treatments” in order to evaluate selection into each experience and its effect on outcomes.

The survey items capturing engagement experiences in the BPS included three response categories: never, sometimes, and often. Ideally, items would be scaled using five- or seven-points in order to create the variance necessary to examine the relationships among items and obtain adequate reliability (internal consistency) estimates (Bagozzi, Yi, & Phillips, 1991; Carmines & Zeller, 1979; Lissitz & Green, 1975). The items in the BPS were not designed in line with this recommendation, which may contribute to the lack of internal consistency across items. Furthermore, among the community college students, the “often” response option was rarely selected (across the four experiences, 7 percent of the sample, on average, reported “often” participating).

Ultimately, I combined the “sometimes” and “often” categories to consider the effect of ever participating in an engagement experience during the first year compared to never participating. This decision was practical—the results are more meaningful when comparing someone who engages in an activity to someone who does not. It is difficult to meaningfully compare “sometimes” engaging to “often” engaging when the label is self-ascribed. But the decision also aligns with theory, given the challenges that arise from using an inadequate number of response options (Carmine & Zeller, 1979). Given the low rate of “often” responses among members of the sample, there is minimal threat of bias from combining the two categories. To ensure results were not sensitive to combining the two categories, I ran supplemental analyses without students who “often” engage in each experience. The results were robust to their inclusion.

Outcomes.

In order to evaluate the effects of the engagement experiences on first-year achievement, retention into the second year, associate degree completion, and transfer to a four-year college, I used BPS survey data and data from the BPS Postsecondary Education Transcripts (PETS). Retention into the second year was determined from self-reported enrollment items from the survey data, while the remaining outcomes—first-year GPA, earning a degree, and transferring to a baccalaureate college—were drawn from transcript data.

While I evaluate the impact on several outcomes in order to capture the varied impacts of engagement, estimating the impact on each outcome separately is not without its limitations. The approach offers greater depth than prior research, but oversimplifies the complex pathways taken by students, especially in the community college context. The long-term outcomes (associates degree and transfer) are not mutually exclusive. It would be potentially feasible to create one outcome with more categories (e.g., an alternative long-term outcome could capture: did not earn an associate, did not transfer; did not earn an associate and transferred; earned an associate and didn’t transfer; earned an associate and did transfer). Using a multinomial outcome would require a different method—multinomial logistic regression instead of logistic regression for the long-term outcomes. What might be gained in nuance would also create for more difficult interpretation. For ease of interpretation, I use separate outcomes for degree attainment and transfer, which still represents a contribution to the literature as the first long-term examination of engagement’s effects in community colleges.

Control measures.

Analyses included independent variables expected to impact engagement experiences and outcomes, controlling for measures of demographic background, academic preparation, and college contexts. The literature reviewed above informed the inclusion of individual characteristics that predict community college student engagement and outcomes. The literature on institutional factors influencing student engagement, though largely performed on four-year student samples, informed my choice of institutional characteristics (Kezar, 2006; Pike & Kuh, 2006; Porter, 2006). Table 1 describes the engagement experience measures, outcomes, and control variables.

Table 1.

Description of Variables

Variable	Description	Mean (SD)
Dependent variables:
First-Year GPA	Cumulative grade point average from first year	2.662 (1.049)
Retention into Second Year	Remained enrolled throughout first year and re-enrolled in year 2	0.178 (0.383)
Associate Degree	Earned an associate degree within six years of initial enrollment	0.236 (0.425)
Transfer to Four-Year	Transferred to a four-year college within six years of initial enrollment	0.169 (0.375)
Independent variables:
Engagement Experiences
Faculty Social	Had informal/social contact with faculty outside of classrooms and the office	0.294 (0.456)
Faculty Academic	Spoke with faculty members about academic matters outside of class	0.680 (0.467)
Study Group	Attended study groups	0.393 (0.489)
Clubs	Participated in school clubs	0.134 (0.340)
Background
Logged Income	Household 2002 income in dollars, transformed into log(income)	10.005 (2.071)
Pell Grant	2003-04 Pell Grant award amount in $1000s	0.710 (1.299)
Parents’ Education:
No College (reference)	Neither parent attended college	0.426 (0.494)
Up to Two Years	At least one parent attended college for up to 2 years, including earning an Associate’s degree	0.226 (0.418)
Up to Four Years	At least one parent attended college for more than 2 years, but did not earn a Bachelor’s degree	0.061 (0.240)
Bachelor’s Degree	At least one parent earned Bachelor’s degree	0.166 (0.372)
Advanced Degree	At least one parent earned an advanced degree (master’s, professional, or beyond)	0.112 (0.315)
Parents Own Home	Indicates whether parents of respondent own their home in 2003-2004	0.761 (0.427)
Single Parent	Indicates whether parent is divorced, separated, or widowed	0.521 (0.499)
Race:
White (reference)	Identifies as white	0.650 (0.477)
Black	Identifies as black	0.161 (0.367)
Asian	Identifies as Asian	0.039 (0.194)
Other Race	Identifies as another race, including pacific islander and Native American	0.019 (0.324)
Hispanic	Identifies as Hispanic	0.148 (0.355)
Female	Identifies as female	0.571 (0.495)
Age	Age at initial enrollment	22.107 (7.342)
Parental Support	Indicator of receiving at least some financial support from parents (e.g. pay housing, living expense, other expense, and or tuition/fees)	0.497 (0.500)
Married	Married at time of initial enrollment	0.163 (0.370)
Children	Has children	0.158 (0.365)
Dependent	Financially dependent on parents (claimed as dependent on tax returns)	0.631 (0.483)
Educational Information
GED	Earned General Education Development certificate instead of high school diploma	0.132 (0.338)
High School GPA	Continuous measure of cumulative Grade Point Average in high school, created by assigning the average value of the categorical measure (1=.5-.9, 2=1.0-1.4, 3=1.5-1.9, 4=2.0-2.4, 5=2.5-2.9, 6=3.0-3.4, 7=3.5-4.0) for a more interpretable metric	2.846 (0.623)
AP Credit	Earned Advanced Placement credit in high school	0.072 (0.259)
Years Math	Years of high school coursework the respondent completed in mathematics, according to self-report on standardized test questionnaire	2.561 (1.415)
Public High School	Public high school	0.914 (0.280)
College Context
Lives on campus	Lives on campus in fall semester of first year	0.057 (0.232)
Full-Time	Initially enrolls as full-time student	0.665 (0.472)
Distance Home	Miles from college to home (in high school)	21.066 (36.553)
Enrollment Size	Enrollment size of institution	8266.3 (7195.6)
Percent Minority	Percent minority enrollment at college (2003-04)	31.860 (21.883)
Percent Federal Grant	Percent students at college receiving federal grants (2003-04)	38.531 (18.841)
Experiences During College a
Major:	Major (2003-2004), 12 categories
Undeclared (reference)	Student undeclared / no major selected	0.331 (0.471)
Humanities	Humanities major	0.066 (0.245)
Social/Behavioral	Social/behavioral science major	0.027 (0.163)
Life Science	Life science major	0.019 (0.138)
Physical Science	Physical science major	0.003 (0.057)
Math	Mathematics major	0.002 (0.046)
Computer/Information	Computer or information science major	0.041 (0.197)
Engineering	Engineering major	0.032 (0.176)
Education	Education major	0.057 (0.233)
Business/Management	Business or management-related major	0.117 (0.321)
Health	Health major	0.167 (0.373)
Vocational/Technical	Vocational or technical major	0.050 (0.200)
Other Professional	Other professional major	0.095 (0.292)
Hours Worked	Hours the respondent worked for pay per week (2003-04), excluding work study	20.869 (15.629)
Job Related to Major	Employed at job related to college program/major	0.108 (0.311)
Remedial Coursework	Number of remedial courses taken	1.808 (2.350)

Notes: Notes: N= 7,010.

^aPropensity score matching models selection into college experiences using pre-treatment measures, making it inappropriate to include measures capturing other college experiences in the model predicting engagement during the first-year of college (Morgan & Winship, 2007). For this reason, the “experiences during college” variables are included in the final regression model, but not in the propensity score model. This captures the influence of the measures in estimating the effect of engagement on each outcome without potentially “controlling away” part of the treatment effect of first-year engagement.

Analysis

My analysis proceeded in two steps. First, I estimated the probability an individual engaged in each experience. To do so, I specified separate selection models for each of the four engagement experiences. I estimated each student’s propensity to engage using a probit regression of the engagement experience on demographic, academic, and institutional measures,

$$P=\Phi ({\beta }_0+\sum _{\mathrm{k}}{\beta }_{\mathrm{k}}{X}_{\text{ik}}).$$

P represents the propensity score, an estimate of individual likelihoods of engagement relative to each control state. ${\beta }_0$ represents the estimated intercept. ${\beta }_{\mathrm{k}}$ represents each estimated parameter (1 to $k$ for covariates ($X_{\text{ik}}$) included in the propensity model. Because $\Phi$ represents the cumulative normal distribution, the βs are z scores that represent the expected change in standard deviation units of the latent dependent variable for covariate $\mathrm{k}$.

The resulting propensity score summed up the probability of participating in the activity into one number (Morgan & Winship, 2007). Study participants with the same calculated propensity score are essentially comparable with respect to their “propensity to engage” in an experience, even if they differ in values of specific covariates (Guo & Fraiser, 2010). I invoked the “ignorability” assumption that, conditional on the rich set of pre-treatment covariates, there were no additional confounders between students who participated (treatment) and those who did not (control) (Morgan & Winship, 2007). Of course, propensity score matching—which, in this case, compares students with similar propensities to engage who demonstrated different actual engagement behavior—is not a cure-all for selection concerns. It can only account for observed differences between treatment and control groups. Still, it is a valuable technique that aligns with the principles of causal inference when used with a rich set of observed characteristics, like those available in the BPS (Morgan & Winship, 2007).

The propensity model included background characteristics indicating race, gender, family composition, parental income, and educational attainment. In addition, I included identifiers of academic preparedness and institutional characteristics. Ideally, I would match students within colleges, as college context is linked to student engagement (Hu & Kuh, 2002). Unfortunately, this method was not feasible given the small sample size within each institution. Instead I included measures of college characteristics to match students by college type.

In this study, the propensity score model served two important purposes. First, it enabled me to capture how community college students sort into each engagement experience by modeling the selection mechanism, as described above. Second, I leveraged the propensity scores to obtain a more conservative (i.e., less biased) estimate of the average effects of engaging with faculty and peers on student outcomes, described next.

In the second step of analysis, I estimated effects of the engagement experiences on first-year GPA, retention into the second year, associate degree attainment, and transfer to a four-year college. I used regression models that control for the independent variables listed in Table 1, weighted by the estimated propensity scores. Performing regression with covariates on propensity score-processed data reduces remaining covariate imbalance across treatment and control groups left behind after matching and is thus preferable to relying solely on the propensity scores to match students and obtain average treatment effects (Ho, Imai, King, & Stuart, 2007). To be prudent, I also specified the final regression models without students’ propensity scores. The results showed that the pre-processed data (matched on students’ propensity to engage) generally offered more conservative estimates of the average effects.

Pre-processing the data based on propensity to engage decreased the difference in covariate means across participants and non-participants (treatment and control groups). Table 2 demonstrates that the differences in covariate means between groups were largely reduced by accounting for propensity scores. Matching failed to completely eliminate bias on observables. For instance, while matching greatly diminished the differences in Pell award amount between treatment and control groups for all engagement experiences, the treatment groups still had higher Pell awards than the control groups after matching. This confirmed the need for regression on the pre-processed data to eliminate remaining bias.

Table 2.

Comparison of Treatment and Control in Unmatched and Matched Data: Covariate Means

	Engagement Experience
	Faculty Social		Clubs		Study Group		Clubs
Covariate	T	C	T	C	T	C	T	C
Logged Income	9.962	10.032	9.989	10.058	9.943	10.054*	9.941	10.022
	9.967	9.977	9.991	10.010	9.946	9.956	9.943	9.930
Pell Grant ($1000)	0.834	0.672***	0.814	0.519***	0.833	0.648***	0.922	0.689***
	0.828	0.803***	0.814	0.752***	0.829	0.808***	0.919	0.913**
Parent: No College	0.396	0.450***	0.414	0.478***	0.397	0.458***	0.390	0.441**
	0.397	0.396**	0.414	0.415***	0.398	0.400***	0.391	0.388
Parent: Up to Two Years	0.231	0.227	0.233	0.216	0.228	0.228	0.224	0.229
	0.231	0.230	0.233	0.235	0.228	0.229	0.224	0.227
Parent: Up to Four Years	0.065	0.059	0.064	0.054	0.068	0.056*	0.057	0.061
	0.065	0.067	0.064	0.063	0.067	0.067	0.057	0.062
Parent: Bachelor’s Degree	0.175	0.162	0.171	0.154	0.180	0.157*	0.190	0.162*
	0.175	0.178	0.171	0.170	0.180	0.181*	0.190	0.185
Parent: Advanced Degree	0.133	0.102***	0.118	0.097*	0.127	0.101**	0.138	0.107**
	0.132	0.129**	0.118	0.118*	0.126	0.124*	0.137	0.138
Parents Own Home	0.778	0.754*	0.761	0.762	0.761	0.761	0.787	0.757*
	0.779	0.774	0.761	0.764	0.761	0.761	0.788	0.787
Single Parent	0.485	0.567***	0.527	0.577***	0.545	0.541	0.412	0.563***
	0.486	0.489***	0.527	0.526***	0.546	0.545	0.412	0.423***
White	0.622	0.658**	0.650	0.641	0.618	0.666***	0.624	0.651
	0.622	0.628	0.651	0.657	0.619	0.626**	0.624	0.619
Black	0.174	0.152*	0.162	0.152	0.177	0.147**	0.168	0.157
	0.174	0.172	0.162	0.160	0.177	0.174	0.168	0.171
Asian	0.044	0.036	0.040	0.035	0.042	0.036	0.071	0.033***
	0.044	0.044	0.040	0.040	0.042	0.042	0.071	0.068***
Other Race	0.128	0.122	0.115	0.142**	0.129	0.120	0.110	0.126
	0.128	0.126	0.115	0.11**	0.129	0.128	0.110	0.110
Hispanic	0.144	0.150	0.138	0.17**	0.149	0.148	0.136	0.150
	0.144	0.143	0.138	0.137**	0.149	0.150	0.135	0.137
Female	0.542	0.585**	0.577	0.563	0.592	0.560*	0.553	0.575
	0.542	0.543**	0.577	0.567	0.591	0.591*	0.554	0.552
Age	21.025	22.638***	21.774	22.984***	21.822	22.376**	20.174	22.466***
	21.032	21.015***	21.775	21.782***	21.826	21.897**	20.176	20.348***
Parental Support	0.567	0.469***	0.530	0.432***	0.535	0.475***	0.623	0.479***
	0.566	0.563***	0.529	0.529***	0.534	0.531***	0.623	0.618***
Married	0.114	0.187***	0.149	0.200***	0.148	0.177**	0.093	0.177***
	0.114	0.115***	0.149	0.146***	0.148	0.151**	0.093	0.098***
Children	0.126	0.175***	0.157	0.168	0.155	0.164	0.100	0.170***
	0.126	0.128***	0.157	0.156	0.155	0.156	0.100	0.104**
Financially Dependent	0.702	0.602***	0.662	0.568***	0.650	0.620*	0.748	0.614***
	0.701	0.699***	0.662	0.661***	0.650	0.647*	0.748	0.739***
GED	0.108	0.141***	0.125	0.147*	0.131	0.132	0.108	0.135*
	0.109	0.111**	0.125	0.126*	0.131	0.134	0.108	0.111*
High School GPA	2.859	2.811**	2.840	2.795**	2.862	2.802***	2.931	2.809***
	2.858	2.854*	2.840	2.836*	2.861	2.857**	2.930	2.917***
AP Credit	0.102	0.060***	0.080	0.057**	0.084	0.065**	0.118	0.065***
	0.101	0.096***	0.080	0.078**	0.083	0.080*	0.118	0.110***
High School Math	2.688	2.453***	2.574	2.413***	2.643	2.446***	2.883	2.467***
	2.686	2.678***	2.574	2.564***	2.641	2.628***	2.882	2.837***
Public High School	0.904	0.917	0.910	0.921	0.918	0.910	0.882	0.918***
	0.904	0.906	0.910	0.917	0.918	0.918	0.883	0.894**
Live on Campus	0.103	0.040***	0.069	0.038***	0.083	0.044***	0.155	0.044***
	0.101	0.095***	0.069	0.062***	0.082	0.069***	0.154	0.145***
Full-Time	0.735	0.637***	0.706	0.580***	0.731	0.624***	0.845	0.639***
	0.735	0.728***	0.706	0.705***	0.731	0.732***	0.845	0.833***
Distance Home	24.594	20.247***	22.279	19.918*	23.376	20.347**	27.387	20.641***
	24.594	23.831*	22.221	21.236	23.227	22.005*	27.387	26.933*
Enrollment Size	7886.5	8414.7**	8147.3	8495.3	8241.1	8269.4	7460.4	8380.9***
	7895.9	7962.6*	8145.6	8199.3	8229.7	8213.1	7461.8	7677.7**
Percent Minority	30.826	32.368**	31.027	33.799***	32.439	31.575	30.325	32.155*
	30.865	30.977*	31.030	31.262***	32.407	32.387	30.332	30.878*
Percent Federal Funding	38.924	38.444	38.579	38.601	38.643	38.550	39.514	38.444
	38.906	38.771	38.581	38.504	38.641	38.659	39.526	39.460
Undeclared	0.310	0.349**	0.315	0.386***	0.317	0.350**	0.283	0.346***
	0.310	0.341	0.315	0.366**	0.317	0.334	0.283	0.332
Humanities	0.063	0.061	0.066	0.053*	0.060	0.063	0.098	0.056***
	0.063	0.068	0.066	0.058	0.060	0.067	0.098	0.061**
Social/Behavioral	0.027	0.027	0.030	0.020*	0.032	0.024	0.036	0.025
	0.027	0.029	0.030	0.022	0.031	0.025	0.036	0.029
Life Science	0.024	0.017*	0.020	0.016	0.024	0.016*	0.029	0.018*
	0.024	0.019	0.020	0.017	0.024	0.018	0.029	0.024
Physical Science	0.004	0.003	0.004	0.003	0.005	0.002	0.006	0.003
	0.004	0.005	0.004	0.004	0.005	0.002	0.006	0.005
Math	0.003	0.002	0.003	0.000	0.002	0.002	0.003	0.002
	0.003	0.002	0.003	0.000	0.002	0.002	0.003	0.003
Computer/Information	0.042	0.041	0.041	0.042	0.042	0.040	0.046	0.040
	0.042	0.042	0.041	0.043	0.042	0.039	0.046	0.041
Engineering	0.027	0.034	0.030	0.038	0.030	0.034	0.034	0.032
	0.027	0.036*	0.030	0.039	0.030	0.034	0.034	0.035
Education	0.068	0.052**	0.064	0.040***	0.061	0.053	0.073	0.054*
	0.068	0.052*	0.064	0.043**	0.061	0.056	0.073	0.053
Business/Management	0.116	0.117	0.116	0.117	0.112	0.119	0.108	0.118
	0.116	0.119	0.116	0.118	0.112	0.120	0.108	0.122
Health	0.166	0.167	0.174	0.152*	0.186	0.155**	0.138	0.171*
	0.166	0.157	0.174	0.157	0.186	0.164	0.139	0.164
Vocational/Technical	0.044	0.040	0.039	0.046	0.034	0.046*	0.029	0.043
	0.043	0.041	0.039	0.044	0.034	0.041	0.029	0.040
Other Professional	0.108	0.089*	0.098	0.087	0.095	0.095	0.116	0.092*
	0.108	0.088*	0.098	0.089	0.095	0.095	0.116	0.092*
Hours Worked	19.128	21.535***	19.911	22.769***	19.762	21.500***	17.143	21.388***
	19.133	20.408	19.914	21.324	19.783	20.223	17.123	19.009
Job Related to Major	0.111	0.108***	0.112	0.102	0.109	0.109	0.103	0.110
	0.111	0.101	0.112	0.096	0.109	0.105	0.104	0.099
Remedial Courses	1.894	1.782	1.907	1.619***	1.856	1.789	1.796	1.818
	1.896	1.762	1.907	1.669**	1.856	1.850	1.798	1.763
N	2,070	4,940	4,770	2,240	4,260	2,750	940	6,070

Notes: N= 7,010. For each covariate, the table displays the means prior to matching over means after matching.

^aN denotes the pre-matching sample size. A small number of observations were off the common support: Faculty social: 15 cases (10 control, 5 treatment), Faculty Academic: 3 (2 control, 1 treatment), Study Groups: 12 (8 control, 4 treatment), Clubs: 60 (59 control, 1 treatment).

^*p<.05

^**p<.01

^***p<.001

Results

Descriptive Statistics

Table 1 presents means and standard deviations of the outcomes and independent variables. The average student in the analytic sample came from a family with an income of $45,230. For 43 percent of the sample, neither parent attended college. Sixty-five percent of the sample identified as white, 16 as Black, 4 as Asian, and nearly 15 percent as Hispanic. The average age at college entry was 22.1, with a standard deviation of 7.3, which indicates a fairly wide range of age compared to the more traditional college-going population covered in the early student development literature (e.g., Astin, 1993; Tinto, 1993). Still, almost two thirds of students were financially dependent on their parents, for financial aid purposes. However, less than half of the sample reported receiving financial assistance from their parents during college. The average student worked approximately 21 hours a week for pay. Two-thirds of the sample initially enrolled full time and one-third did not enter college with a major.

Most students engaged with faculty and peers, but there was variation in participation rates across the four engagement experiences. Academic contact with faculty was the most common experience: 68 percent of students reported doing so. Thirty-nine percent of students reported participating in study groups, 29 percent of students engaging in social contact with faculty, and only 13 percent of the sample reported participating in school clubs. In contrast, among BPS students who initially enrolled at four-year colleges, 84 percent engaged in academic contact with faculty, 69 participated in study groups, 46 percent in social contact with faculty, and 45 percent in school clubs (author’s calculations). These descriptive patterns suggest that community college students are indeed less likely to engage with members of the campus community outside of the classroom than their four-year counterparts. This finding is a function of the measures, which focus on engagement external to the classroom, and is therefore anticipated by the literature (e.g., Deil-Amen, 2011).

Selection into College Experiences

Table 3 presents the coefficients from probit models predicting participation in each college experience. It provides insight into the factors that predict participation in specific engagement experiences after controlling for other student background and institutional characteristics. For ease of interpretability, I describe the substantively important, statistically significant relationships in terms of predicted probabilities.

Table 3.

Probit Models Predicting Propensity to Engage

	Engagement Experience (Treatment)
Variable	Faculty Social		Faculty Academic		Study Group		Clubs
Logged Income	−0.018*	(0.009)	−0.020*	(0.009)	−0.013	(0.009)	−0.022**	(0.011)
Pell Grant ($1000)	0.055***	(0.014)	0.103***	(0.015)	0.048***	(0.014)	0.070***	(0.016)
Parent College: Two Years	0.036	(0.044)	0.086*	(0.043)	0.083*	(0.042)	−0.019	(0.054)
Parent College: Four Years	0.111	(0.071)	0.175*	(0.072)	0.213**	(0.068)	0.010	(0.089)
Parent College: Bachelor’s	0.052	(0.049)	0.103*	(0.049)	0.174***	(0.047)	0.039	(0.060)
Parent College: Advanced	0.164*	(0.056)	0.166**	(0.058)	0.230***	(0.055)	0.094	(0.068)
Parents Own Home	0.014	(0.043)	−0.051	(0.042)	0.022	(0.040)	0.017	(0.053)
Single Parent	−0.099**	(0.041)	−0.040	(0.041)	0.065	(0.040)	−0.238***	(0.051)
Black	0.154*	(0.051)	0.048	(0.051)	0.121*	(0.049)	0.124**	(0.063)
Asian	0.127	(0.088)	0.041	(0.090)	0.111	(0.085)	0.420***	(0.096)
Other Race	0.164*	(0.068)	−0.043	(0.067)	0.128	(0.065)	0.021	(0.085)
Hispanic/Latino	0.000	(0.064)	−0.016	(0.063)	−0.005	(0.062)	0.080	(0.079)
Female	−0.075**	(0.034)	0.043	(0.034)	0.090**	(0.033)	−0.014	(0.042)
Age	−0.007**	(0.003)	−0.001	(0.003)	0.003	(0.003)	−0.016***	(0.005)
Parental Support	0.061	(0.049)	0.144**	(0.049)	0.165**	(0.048)	0.096	(0.060)
Married	−0.079	(0.061)	0.029	(0.056)	−0.004	(0.056)	0.030	(0.080)
Children	−0.031	(0.061)	0.125*	(0.057)	0.009	(0.056)	−0.057	(0.080)
Financially Dependent	−0.031	(0.065)	0.104	(0.062)	−0.079	(0.062)	−0.129	(0.081)
GED	−0.021	(0.054)	−0.017	(0.051)	0.088	(0.050)	0.144**	(0.067)
High School GPA	0.023	(0.028)	0.025	(0.028)	0.059*	(0.027)	0.086**	(0.035)
AP Credit	0.211*	(0.062)	0.088	(0.067)	0.060	(0.062)	0.159**	(0.071)
Years Mathematics	0.021	(0.013)	0.011	(0.013)	0.041**	(0.013)	0.055***	(0.017)
Public High School	−0.091	(0.059)	−0.099	(0.060)	0.113*	(0.058)	−0.159**	(0.069)
Lives on Campus	0.446***	(0.070)	0.182*	(0.078)	0.337***	(0.069)	0.557***	(0.073)
Full-Time Student	0.130**	(0.038)	0.224***	(0.036)	0.250***	(0.036)	0.408***	(0.051)
Distance to Home	0.000	(0.000)	0.000	(0.001)	0.001	(0.000)	0.000	(0.001)
Enrollment Size	0.000**	(0.000)	0.000	(0.000)	0.000	(0.000)	−0.000***	(0.000)
Percent Minority Enrollment	−0.002*	(0.001)	−0.003***	(0.001)	0.001	(0.001)	−0.001	(0.001)
Percent Federal Funding	0.001	(0.001)	0.000	(0.001)	−0.002	(0.001)	0.001	(0.001)
Constant	−0.304	(0.176)	0.400*	(0.173)	−1.071***	(0.169)	−1.067***	(0.223)

Notes: N= 7,010. Standard errors are in parentheses.

^*p<.05

^**p<.01

^***p<.001

It appears that financial support positively influenced community college student engagement, even after controlling for family income and other background characteristics. Pell Grant award amount was positively related to each engagement behavior even after controlling for other factors (faculty social: β=0.055, $p<.001$; faculty academic: β=0.103, $p<.001$; study groups: β=0.048, $p<.001$; clubs: β=0.070, $p<.001$). For example, among students who did not receive a Pell Grant, 66 percent engaged with faculty about academic matters, compared to 73 of students who received a grant of $2000, and 79 percent of students who received $4000. Parental financial support positively predicted engagement in academic contact with faculty and study groups (faculty academic: β=0.144, $p<.01$; study groups: β=0.165, $p<.01$). Students who received financial support from their parents were 5 percentage points more likely to interact with faculty regarding academic matters than their peers who did not. Pell dollars and financial support from parents may offset students’ need to work for pay, increasing time available to spend on campus.

After controlling for other background characteristics, high school GPA and years of math positively predict participation in study groups and school clubs (study groups: =0.059, $p<.05$; =0.041, $p<.01$; clubs: =0.086, $p<.01$; =0.055, $p<.001$). The PS model also confirms prior literature that Black students are more likely than white students to socially engage in the campus community even after controlling for background and prior experiences (faculty social: β=0.154, $p<.05$; study groups: β=0.121, $p<.05$; clubs: β=0.124, $p<.01$).

Full-time enrollment and living on campus positively predict engagement. Full-time students were more likely to interact socially with faculty (by 4 percentage points), academically with faculty (by 8 percentage points), and participate in study groups and clubs (by 9 and 7 percentage points, respectively) than part-time students (faculty social: β=0.130, $p<.01$; faculty academic: β=0.224, $p<.001$; study groups: β=0.25, $p<.001$; clubs: β=0.408, $p<.001$). Likewise, living on campus—which is relatively rare (only 5.7 percent of the sample reported doing so)—appeared to greatly increase student engagement in social and academic contact with faculty (by 16 and 6 percentage points, respectively) and participation in study groups and school clubs (by 13 and 14 points each) (faculty social: β=0.446, $p<.001$; faculty academic: β=0.182, $p<.05$; study groups: β=0.337, $p<.001$; clubs: β=0.557, $p<.001$).

Effects of Engagement Experiences on Outcomes

Table 4 presents the average treatment effects of each college experience on short-term outcomes (first-year cumulative GPA and retention into the second year of college) and long-term outcomes (earning an associate degree and transferring to a four-year college within six years) as predicted probabilities and effect sizes (Cohens’ d⁴) (Cohen, 1988, 1992). The table presents results from the unweighted regression models (not controlling for students’ propensity to engage), in addition to the final propensity-score weighted results. I interpret the results of the weighted regression models, which generally offer more conservative estimates. For a full set of regression coefficients for the final propensity score weighted models, please see Table A1-A4 in Appendix A. Weighting the regression by propensity scores diminishes the size of the effect of study groups and club participation on achievement and retention.

Table 4.

Short- and Long-Term Effects of Engagement Experiences: Predicted Probabilities^A and Effect Sizes^B

	Engagement Experience (Treatment)
Outcome	Faculty Social			Faculty Academic			Study Group			Clubs
Unweighted Regressions:
First-Year GPA	0.057	(0.027)	0.054	0.069**	(0.027)	0.065	0.043†	(0.025)	0.041	0.084*	(0.037)	0.081
Retained into Second Year	0.018†	(0.010)	0.299	0.047***	(0.011)	0.916	0.038***	(0.009)	0.745	0.045***	(0.012)	0.854
Associate Degree	0.006	(0.011)	0.092	0.066***	(0.011)	0.962	0.019†	(0.010)	0.236	0.011	(0.014)	0.143
Transfer to Four-Year College	−0.013	(0.010)	−0.256	0.019†	(0.010)	0.353	0.006	(0.009)	0.108	−0.001	(0.012)	−0.014
Propensity Score Weighted Regressions:
First-Year GPA	0.048†	(0.026)	0.046	0.079**	(0.028)	0.076	0.042†	(0.024)	0.041	0.065†	(0.035)	0.062
Retained into Second Year	0.012	(0.014)	0.149	0.046**	(0.014)	0.723	0.031*	(0.013)	0.441	0.026	(0.018)	0.363
Associate Degree	0.006	(0.011)	0.089	0.065***	(0.010)	1.042	0.015	(0.010)	0.217	0.011	(0.015)	0.162
Transfer to Four-Year College	−0.013	(0.009)	−0.274	0.019*	(0.009)	0.375	0.007	(0.009)	0.125	0.005	(0.014)	0.104

Notes: N= 7,010. The table presents the average treatment effect in predicted probabilities with standard errors in parentheses, followed by the effect size. The effect of each engagement experience was obtained through separate analytic models on distinct treatment and control groups. A complete set of regression coefficients are available in Appendix A.

^aTo interpret the impact in terms of predicted probabilities, it is useful to see control means for each outcomes: First-year GPA: Faculty Social= 2.664, Faculty Academic=2.644, Study Groups=2.661, Clubs=2.658; Retention into second year of college: Faculty Social= 0.164, Faculty Academic=0.126, Study Groups=0.158, Clubs=0.166; Associate degree: Faculty Social= 0.222, Faculty Academic= 0.167, Study Groups= 0.221, Clubs= 0.225; Transferring to a four-year college: Faculty Social= 0.163, Faculty Academic= 0.135, Study Groups= 0.157, Clubs= 0.161.

^bEffect size was calculated using a pooled standard deviation (Cohen, 1988); see Appendix B for formula.

^†p<.1

^*p<.05

^**p<.01

^***p<.001

Academic engagement with faculty outside of class offers positive returns for both short- and long-term college outcomes. Meeting with an instructor about academic matters improves first-year GPA by .08-percentage points, a small, but statistically significant impact that would take the mean student from the control group from a 2.644 first-year GPA to a 2.723 (p<.01). While statistically significant, the effect size is very small—0.07 standard deviation units. The impact still may be meaningful to students near cutoffs for academic standards, such as institutional academic probation standards or satisfactory academic progress standards for need-based aid (Schudde & Scott-Clayton, 2016; Scott-Clayton & Schudde, 2016).

The effect of faculty academic interactions on student persistence and attainment may have greater practical significance. Academic contact with faculty improves retention into the second year of college by almost 5 percentage points (increasing the probability of persisting from 12.6 to 17.1) (p<.01), or about 0.72 standard deviation units (a medium to large effect size, according to Cohen, 1988), and associate degree attainment by almost 7 percentage points (increasing the probability of earning a degree from 16.7 to 23.2) (p<.001), an entire standard deviation increase, which is very large. It also improves the probability of transferring to a four-year college by about 2 percentage points (p<.05).

Engagement with peers shows more modest impacts on student outcomes. Study group participation positively impacts student retention and degree attainment, increasing the probability of remaining enrolled into the second year of college by 3 percentage points (p<.05) (0.44 standard deviation units). To put this into context, participating in study groups increased the probability of remaining enrolled by the second year of college from 15.8 to 17.6. Engaging in study groups and school clubs increased first-year GPA by .04- and .07-percentage points, respectively, but the statistically significant impacts (p<.1) are very small, with effect sizes well under .1 standard deviation units.

Discussion

In this study, I tested the impact of four engagement experiences at community colleges on achievement, retention, degree attainment, and transfer to a four-year college. To determine whether engagement experiences impact student outcomes, I used nationally representative data and leveraged students’ propensity to engage in each engagement experience to control for selection bias. The results show that engagement with faculty regarding academic matters offers both short- and long-term positive impacts on college outcomes. Other engagement experiences, like study group and club participation, also positively impact outcomes, but the results appear more sensitive to propensity score matching. The diminishing size of the effects after weighting the regression with propensity scores, as in the impact of study groups on retention and of clubs on retention and first-year GPA, bolsters support for the need to control for selection into engagement. Studies that indicate strong correlations between engagement experiences and outcomes but do not adequately control for student background may suffer from selection bias, presenting artificially inflated impacts.

The findings of the propensity score models align well with recent research examining selection into engagement, including work by Saenz et al. (2011) and the Community College Center for Student Engagement (2005, 2013a, 2013b), but contribute additional information on the influence of detailed background factors on engagement behavior among community college students. For instance, it appears that financial support, whether from parents or the federal government, improves students’ propensity to engage in most experiences. Unlike the four-year literature, which largely shows that students from low-income families are less likely to engage with faculty and peers than affluent students (Schudde, 2013; Stuber, 2011), Pell recipients at community colleges appear more likely to engage in the campus environment than their peers. This study cannot demonstrate why that is the case, but theory suggests that aid offsets the need to work for pay, potentially freeing up time to spend on campus (Boatman & Long, 2016; Broton et al., 2016; Nora et al., 2006). Students without financial support—who likely need to take on more hours working for pay off campus—may miss out. This is a particularly important point because approximately 40 percent of community college students do not apply for financial aid, and certain populations, like undocumented immigrants, do not qualify for federal aid (and may not qualify for any aid in some states) (McKinney & Novak, 2013; NCES, 2016; Novak & McKinney, 2011). It is possible that in the four-year college context, where tuition is more expensive, federal grant aid does not offset enough of the financial burden for students to significantly increase their participation on campus. The lower tuition at community colleges means that need-based grants cover a greater proportion, potentially offsetting financial constraints and allowing students to work less. However, given the rapidly increasing costs of college, the Pell Grant does not have much buying power in either context. It seems possible that increasing aid to cover more college costs could further offset working for pay and improve engagement in all postsecondary settings.

This study contributes new information to our understanding of the impact of engagement on academic outcomes for community college students. Engagement experiences, particularly meeting with faculty about academics, improve student outcomes in the community college context, not just at four-year colleges. Whereas prior research, such as Barnett’s (2011) study on faculty validation, suggested an impact on student’s intent to persist, this study tests whether engagement experiences actually impact early college persistence, along with other outcomes. The results show that engaging academically with faculty outside of class positively impacts student outcomes, including early achievement (though the effect sizes are likely too small to have practical significance) and retention, as well as degree attainment and vertical transfer. Results also demonstrate that study group and school club participation also improve persistence, though the effects are smaller than that of faculty academic engagement. Given the findings, efforts to get community college students more involved in academic interactions with faculty have the potential to move the dial in terms of successful outcomes.

Of course, only students who participate in these experiences can reap the rewards. McClenney and Greene (2005) argue that colleges can increase participation in out-of-class activities, like visiting faculty during office hours, by using “intentionality and focus” to promote particular engagement experiences on campus (p. 6). Research suggests that well-designed curricular activities can encourage student engagement in and out of the classroom. Coursework that emphasizes performing and applying knowledge leads to higher engagement (Saenz et al., 2011). Of course, providing greater extracurricular faculty engagement opportunities may be a challenge. Many faculty at community college campuses are overextended, making it difficult to meet the demand for one-on-one faculty-student meetings.

While the results suggest that engagement with campus community members may guide students toward persistence, more evidence is necessary to establish what effective faculty-student and student-student academic interactions look like. Measures available in the BPS capture four different dimensions of faculty and peer engagement during the first year of college, but do not offer specific information on what the experience looked like. For instance, while meeting with faculty about academic matters improves a variety of desirable outcomes, colleges need information they can use to encourage productive faculty-student interactions outside of class. Data from sources like CCSSE offer detailed measures of campus experiences, but are cross-sectional rather than longitudinal, making it difficult to examine the effect of those experiences on outcomes. Incorporating similar fine-grained measures into longitudinal datasets could improve our ability to pinpoint the most effective engagement practices.

Examining patterns of participation and effects at the institutional level using similar methods to this study—controlling for student characteristics to illuminate the effects of different engagement experiences—could help colleges understand the specific needs of their student body. Colleges should examine the rates of participation among specific subgroups to pinpoint groups who may be missing out. Examining variation in effects may also be useful at the institutional level, as specific college contexts may be contribute to heterogeneous effects across subgroups (Arum & Roksa, 2011; Braxton et al., 2004).⁵ Administrators and practitioners could use this information to implement changes in educational practice based on average treatment effects, rates of participation, and differential effects across subgroups.

Conclusion

The results bolster support for ongoing research on engagement and integration in the community college setting by demonstrating that engagement with faculty and peers positively impacts student outcomes among a nationally representative sample of community college students. Efforts to increase engagement with faculty and peers, especially in relation to academic matters, may serve as vital interventions to improve persistence. Additional research is necessary to provide college administrators and practitioners with a blue print they can use for promoting effective educational practices.

Capturing clear evidence on how colleges can intervene to improve student outcomes requires research that may use similar methodological techniques as this study, but with granular engagement measures. There is a dearth of longitudinal data linking finely measured engagement experiences to short- and long-term outcomes. Yet there are increasing calls for community colleges to absorb the growing demand for postsecondary education in the United States, despite stagnant budgets. In a time of resource constraints and increasing accountability, community college leaders need strong evidence to inform educational practices. Building longitudinal data sets to evaluate the impacts of campus engagement experiences is necessary to facilitate rigorous statistical analyses on which types of in-class and out-of-class engagement experiences offer the greatest impact.

Biography

Lauren Schudde is a sociologist of education who examines inequality in postsecondary educational outcomes and returns, particularly across socioeconomic status. Much of her work evaluates the impact of educational policies and practices on student outcomes at community colleges. Her research has been published in the Review of Research in Education, Research in Higher Education, Review of Higher Education, and Community College Review and funded by the Greater Texas Foundation and the Spencer Foundation.

Appendix A

Table A1.

Regression Coefficients for Effect of Engagement Experiences on First-Year Grade Point Average

	Engagement Experience
Variable	Faculty Social		Clubs		Study Group		Clubs
Engagement	0.048*	(0.027)	0.080***	(0.028)	0.044*	(0.025)	0.065*	(0.035)
Logged Income	0.028***	(0.007)	0.020***	(0.007)	0.020***	(0.007)	0.013**	(0.006)
Pell Grant ($)	0.028***	(0.011)	0.031***	(0.012)	0.023**	(0.011)	0.030***	(0.010)
Parent: Up to Two Years	0.022	(0.032)	0.036	(0.033)	0.018	(0.032)	0.115***	(0.031)
Parent: Up to Four Years	−0.141***	(0.052)	−0.022	(0.056)	−0.049	(0.053)	−0.001	(0.049)
Parent: Bachelor’s degree	−0.046	(0.036)	−0.044	(0.038)	−0.018	(0.036)	0.056	(0.035)
Parent: Advanced degree	−0.049	(0.042)	−0.033	(0.045)	−0.036	(0.042)	−0.142***	(0.040)
Parents Own Home	−0.013	(0.031)	−0.058*	(0.033)	−0.032	(0.031)	−0.000	(0.030)
Single Parent	−0.100***	(0.030)	−0.106***	(0.032)	−0.139***	(0.030)	−0.106***	(0.029)
Black	−0.396***	(0.038)	−0.429***	(0.040)	−0.389***	(0.038)	−0.317***	(0.036)
Asian	0.101	(0.065)	−0.055	(0.068)	0.019	(0.065)	0.125**	(0.062)
Other Minority	−0.196***	(0.049)	−0.196***	(0.053)	−0.142***	(0.050)	−0.264***	(0.047)
Hispanic/Latino	−0.018	(0.047)	−0.029	(0.051)	−0.079*	(0.048)	0.140***	(0.044)
Female	0.176***	(0.027)	0.235***	(0.029)	0.217***	(0.027)	0.120***	(0.027)
Age	0.016***	(0.002)	0.014***	(0.002)	0.017***	(0.002)	0.026***	(0.002)
Parental Support	−0.025	(0.036)	−0.043	(0.039)	−0.078**	(0.037)	−0.175***	(0.035)
Married	0.251***	(0.044)	0.176***	(0.045)	0.176***	(0.043)	0.225***	(0.044)
Children	−0.065	(0.044)	−0.078*	(0.045)	−0.074*	(0.043)	−0.038	(0.043)
Financially Dependent	−0.288***	(0.047)	−0.287***	(0.049)	−0.278***	(0.047)	−0.100**	(0.042)
GED	−0.178***	(0.039)	−0.137***	(0.040)	−0.185***	(0.038)	−0.283***	(0.038)
High School GPA	0.226***	(0.020)	0.194***	(0.022)	0.210***	(0.021)	0.242***	(0.020)
AP Credit	0.068	(0.047)	0.084*	(0.050)	0.049	(0.048)	−0.052	(0.045)
Years Mathematics	0.025***	(0.010)	0.012	(0.010)	0.008	(0.010)	0.040***	(0.009)
Public High School	−0.186***	(0.044)	−0.246***	(0.048)	−0.263***	(0.044)	−0.085*	(0.045)
Lives On Campus	−0.193***	(0.058)	−0.035	(0.061)	−0.194***	(0.059)	−0.273***	(0.055)
Full-Time Student	−0.000	(0.027)	−0.029	(0.029)	0.057**	(0.028)	0.244***	(0.027)
Distance to Home	0.000	(0.000)	0.000	(0.000)	0.000	(0.000)	−0.000	(0.000)
Enrollment Size	−0.000	(0.000)	−0.000***	(0.000)	−0.000***	(0.000)	0.000	(0.000)
Percent Minority Enrollment	0.000	(0.001)	0.001	(0.001)	0.001	(0.001)	0.001	(0.001)
Percent Federal Funding	−0.002**	(0.001)	−0.001	(0.001)	−0.001*	(0.001)	−0.002**	(0.001)
Humanities	0.073	(0.055)	−0.010	(0.059)	0.009	(0.055)	−0.009	(0.047)
Social/Behavioral	0.193**	(0.076)	0.002	(0.084)	0.073	(0.076)	0.153**	(0.061)
Life Science	0.052	(0.091)	−0.317***	(0.098)	−0.149*	(0.088)	−0.220***	(0.078)
Physical Science	0.092	(0.213)	0.237	(0.220)	0.361*	(0.206)	−0.040	(0.167)
Math	−0.873***	(0.254)	−0.222	(0.437)	0.075	(0.285)	−0.076	(0.251)
Computer/Information	−0.117*	(0.063)	−0.126*	(0.065)	−0.076	(0.063)	−0.153***	(0.057)
Engineering	0.231***	(0.075)	0.251***	(0.070)	0.222***	(0.069)	0.509***	(0.068)
Education	0.091*	(0.054)	0.062	(0.063)	0.050	(0.055)	0.281***	(0.054)
Business/Management	−0.032	(0.042)	−0.047	(0.044)	−0.003	(0.042)	0.104**	(0.043)
Health	0.059	(0.038)	0.039	(0.040)	0.028	(0.037)	0.212***	(0.040)
Vocational/Technical	0.179***	(0.065)	0.202***	(0.067)	0.136**	(0.067)	0.209***	(0.071)
Other Professional	0.081*	(0.044)	0.045	(0.048)	0.064	(0.045)	0.116***	(0.042)
Hours Worked	−0.005***	(0.001)	−0.003***	(0.001)	−0.005***	(0.001)	−0.005***	(0.001)
Job Related to Major	−0.007	(0.041)	−0.003	(0.045)	0.066	(0.042)	0.091**	(0.039)
Remedial Courses	−0.023***	(0.005)	−0.035***	(0.006)	−0.028***	(0.005)	−0.076***	(0.005)
Constant	1.922***	(0.132)	2.195***	(0.140)	2.132***	(0.135)	1.436***	(0.136)
R2	0.136		0.127		0.141		0.210

N otes: : N= 7,010. The “Engagement” coefficient corresponds to the measure the dataset is matched on. Table displays standard errors in parentheses.

^*p<.1

^**p<.05

^***p<.01

Table A2.

Regression Coefficients for Effect of Engagement Experiences on Persistence to Second Year

	Engagement Experience
Variable	Faculty Social		Faculty Academic		Study Group		Clubs
Engagement	0.057	(0.076)	0.276***	(0.081)	0.168**	(0.071)	0.139	(0.097)
Logged Income	−0.019	(0.021)	−0.016	(0.020)	−0.001	(0.020)	0.027	(0.021)
Pell Grant ($)	0.277***	(0.027)	0.294***	(0.029)	0.295***	(0.027)	0.293***	(0.027)
Parent: Up to Two Years	0.118	(0.091)	−0.151	(0.097)	−0.035	(0.092)	0.053	(0.089)
Parent: Up to Four Years	−0.034	(0.161)	−0.188	(0.173)	−0.044	(0.158)	−0.118	(0.159)
Parent: Bachelor’s degree	0.086	(0.104)	0.096	(0.109)	0.118	(0.102)	0.218**	(0.101)
Parent: Advanced degree	0.124	(0.121)	−0.109	(0.130)	0.035	(0.121)	0.071	(0.122)
Parents Own Home	0.087	(0.094)	0.067	(0.097)	0.035	(0.091)	0.182**	(0.090)
Single Parent	−0.022	(0.086)	−0.209**	(0.091)	−0.150*	(0.087)	0.148*	(0.086)
Black	−0.225**	(0.114)	−0.192	(0.119)	−0.181	(0.112)	−0.198*	(0.112)
Asian	−0.658***	(0.214)	−0.625***	(0.221)	−0.529**	(0.207)	−0.679***	(0.241)
Other Minority	0.070	(0.146)	0.287*	(0.150)	−0.005	(0.146)	0.266*	(0.140)
Hispanic/Latino	−0.138	(0.140)	−0.142	(0.147)	0.034	(0.137)	0.282**	(0.133)
Female	0.067	(0.080)	0.039	(0.085)	0.030	(0.079)	0.004	(0.079)
Age	−0.007	(0.008)	−0.005	(0.008)	−0.001	(0.008)	0.004	(0.007)
Parental Support	0.115	(0.102)	0.264**	(0.110)	0.103	(0.102)	0.338***	(0.102)
Married	−0.056	(0.154)	0.007	(0.154)	0.017	(0.144)	0.280**	(0.134)
Children	−0.058	(0.151)	0.114	(0.151)	−0.027	(0.145)	−0.249*	(0.136)
Financially Dependent	0.212	(0.148)	0.041	(0.155)	0.122	(0.147)	0.082	(0.136)
GED	−0.467***	(0.132)	−0.419***	(0.134)	−0.271**	(0.125)	0.216*	(0.118)
High School GPA	−0.049	(0.061)	−0.019	(0.063)	0.012	(0.061)	−0.143**	(0.059)
AP Credit	−0.086	(0.129)	−0.027	(0.136)	−0.150	(0.130)	−0.117	(0.132)
Years Mathematics	0.032	(0.029)	0.004	(0.030)	0.045	(0.029)	0.092***	(0.029)
Public High School	−0.046	(0.130)	−0.101	(0.140)	−0.223*	(0.124)	−0.132	(0.135)
Lives On Campus	−0.142	(0.139)	−0.340**	(0.152)	−0.146	(0.140)	0.000	(0.132)
Distance to Home	−0.000	(0.001)	0.000	(0.001)	−0.000	(0.001)	0.001	(0.001)
Enrollment Size	0.000	(0.000)	0.000**	(0.000)	0.000	(0.000)	0.000**	(0.000)
Percent Minority Enrollment	0.000	(0.000)	0.000***	(0.000)	0.000	(0.000)	−0.000	(0.000)
Percent Federal Funding	−0.003	(0.002)	−0.003	(0.002)	−0.001	(0.002)	−0.005**	(0.002)
Humanities	−0.001	(0.002)	0.002	(0.002)	0.001	(0.002)	0.001	(0.002)
Social/Behavioral	0.134	(0.153)	0.234	(0.160)	0.309**	(0.149)	0.146	(0.133)
Life Science	0.375*	(0.210)	0.429*	(0.234)	0.439**	(0.210)	0.659***	(0.190)
Physical Science	0.148	(0.239)	0.718***	(0.254)	0.453**	(0.230)	0.298	(0.232)
Math	0.273	(0.572)	−0.763	(0.739)	0.459	(0.507)	−0.976	(0.770)
Engineering	0.812***	(0.171)	0.496***	(0.181)	0.602***	(0.168)	1.047***	(0.161)
Education	0.307	(0.201)	0.717***	(0.180)	0.448**	(0.184)	0.209	(0.193)
Business/Management	0.015	(0.157)	0.042	(0.188)	0.128	(0.159)	0.451***	(0.145)
Health	0.159	(0.125)	0.314**	(0.127)	0.199	(0.125)	−0.063	(0.135)
Vocational/Technical	0.088	(0.114)	0.076	(0.120)	0.092	(0.112)	0.108	(0.120)
Other Professional	0.589***	(0.178)	0.742***	(0.178)	0.407**	(0.189)	0.327*	(0.196)
Hours Worked	0.374***	(0.124)	0.473***	(0.134)	0.420***	(0.125)	0.602***	(0.119)
Job Related to Major	−0.011***	(0.003)	−0.007**	(0.003)	−0.005*	(0.003)	−0.009***	(0.003)
Remedial Courses	0.236**	(0.119)	−0.281**	(0.140)	−0.047	(0.123)	0.033	(0.121)
Constant	0.056***	(0.015)	0.081***	(0.015)	0.060***	(0.015)	0.041***	(0.014)

Notes: : N= 7,010. The “Engagement” coefficient corresponds to the measure the dataset is matched on. Table displays standard errors in parentheses.

^*p<.1

^**p<.05

^***p<.01

Table A3.

Regression Coefficients for Effect of Engagement Experiences on Earning an Associate Degree within Six Years of Entry

	Engagement Experience
Variable	Faculty Social		Faculty Academic		Study Group		Clubs
Engagement	0.038	(0.068)	0.441***	(0.069)	0.104*	(0.063)	0.069	(0.089)
Logged Income	0.094***	(0.023)	0.113***	(0.025)	0.083***	(0.021)	0.092***	(0.022)
Pell Grant ($)	0.191***	(0.027)	0.283***	(0.029)	0.200***	(0.026)	0.311***	(0.027)
Parent: Up to Two Years	0.195**	(0.081)	−0.004	(0.088)	0.179**	(0.081)	0.180**	(0.081)
Parent: Up to Four Years	0.017	(0.137)	0.263*	(0.141)	0.080	(0.137)	−0.121	(0.143)
Parent: Bachelor’s degree	0.213**	(0.090)	0.272***	(0.094)	0.288***	(0.088)	0.247***	(0.092)
Parent: Advanced degree	0.340***	(0.102)	0.590***	(0.104)	0.350***	(0.101)	0.371***	(0.102)
Parents Own Home	−0.007	(0.084)	0.182**	(0.090)	0.059	(0.082)	−0.016	(0.081)
Single Parent	−0.224***	(0.075)	−0.114	(0.079)	−0.256***	(0.075)	−0.160**	(0.074)
Black	−0.550***	(0.108)	−0.645***	(0.118)	−0.546***	(0.107)	−0.147	(0.102)
Asian	0.307**	(0.153)	0.298*	(0.155)	0.152	(0.153)	0.428***	(0.154)
Other Minority	−0.087	(0.131)	−0.275*	(0.142)	−0.306**	(0.131)	−0.262**	(0.127)
Hispanic/Latino	−0.063	(0.123)	0.050	(0.133)	0.117	(0.121)	0.357***	(0.116)
Female	0.181***	(0.070)	0.074	(0.072)	0.106	(0.069)	0.268***	(0.070)
Age	−0.006	(0.007)	−0.001	(0.007)	−0.002	(0.006)	0.023***	(0.006)
Parental Support	0.321***	(0.091)	0.303***	(0.097)	0.331***	(0.092)	0.383***	(0.091)
Married	−0.084	(0.133)	0.121	(0.134)	0.329***	(0.122)	0.418***	(0.129)
Children	0.123	(0.133)	−0.164	(0.135)	0.045	(0.123)	−0.329**	(0.131)
Financially Dependent	−0.042	(0.129)	0.011	(0.134)	0.028	(0.127)	0.293**	(0.119)
GED	−0.715***	(0.127)	−0.511***	(0.126)	−0.422***	(0.115)	−0.471***	(0.117)
High School GPA	0.393***	(0.054)	0.223***	(0.056)	0.281***	(0.054)	0.557***	(0.057)
AP Credit	0.130	(0.110)	0.180	(0.113)	0.280***	(0.108)	−0.170	(0.110)
Years Mathematics	0.165***	(0.026)	0.123***`	(0.027)	0.142***	(0.026)	0.178***	(0.026)
Public High School	−0.277***	(0.106)	−0.281**	(0.114)	−0.402***	(0.104)	−0.344***	(0.112)
Lives On Campus	0.120	(0.127)	0.543***	(0.126)	0.215*	(0.125)	0.206*	(0.124)
Full-Time Student	0.410***	(0.075)	0.504***	(0.080)	0.422***	(0.074)	0.150**	(0.075)
Distance to Home	−0.000	(0.001)	−0.005***	(0.001)	−0.000	(0.001)	−0.004***	(0.001)
Enrollment Size	0.000**	(0.000)	0.000**	(0.000)	0.000**	(0.000)	0.000***	(0.000)
Percent Minority Enrollment	−0.010***	(0.002)	−0.010***	(0.002)	−0.010***	(0.002)	−0.014***	(0.002)
Percent Federal Funding	−0.003	(0.002)	−0.002	(0.002)	−0.003	(0.002)	−0.007***	(0.002)
Humanities	0.396***	(0.127)	−0.045	(0.138)	0.165	(0.130)	0.076	(0.120)
Social/Behavioral	0.583***	(0.184)	0.095	(0.206)	0.337*	(0.181)	0.668***	(0.149)
Life Science	0.334	(0.213)	0.307	(0.231)	−0.040	(0.217)	0.131	(0.211)
Physical Science	1.015**	(0.476)	1.692***	(0.505)	1.271***	(0.453)	3.342***	(0.617)
Math	1.242**	(0.610)	0.876	(1.056)	1.261*	(0.688)	−0.971	(0.642)
Computer/Information	0.683***	(0.157)	0.434***	(0.158)	0.366**	(0.156)	0.881***	(0.148)
Engineering	0.927***	(0.181)	0.529***	(0.166)	0.610***	(0.165)	0.460***	(0.175)
Education	0.767***	(0.128)	0.199	(0.153)	0.438***	(0.130)	0.817***	(0.130)
Business/Management	0.474***	(0.107)	0.273**	(0.109)	0.252**	(0.107)	0.456***	(0.112)
Health	0.119	(0.102)	−0.066	(0.107)	0.075	(0.097)	0.120	(0.109)
Vocational/Technical	0.089	(0.184)	−0.412**	(0.194)	−0.081	(0.187)	0.419**	(0.186)
Other Professional	0.419***	(0.111)	0.162	(0.119)	0.260**	(0.111)	0.440***	(0.112)
Hours Worked	−0.012***	(0.002)	−0.011***	(0.002)	−0.012***	(0.002)	−0.005**	(0.002)
Job Related to Major	0.006	(0.103)	0.276**	(0.110)	0.266***	(0.101)	−0.416***	(0.107)
Remedial Courses	−0.009	(0.014)	−0.026*	(0.016)	0.013	(0.014)	−0.082***	(0.015)
Constant	−3.625***	(0.390)	−3.715***	(0.414)	−3.165***	(0.379)	−4.663***	(0.401)

Notes: : N= 7,010. The “Engagement” coefficient corresponds to the measure the dataset is matched on. Table displays standard errors in parentheses.

^*p<.1

^**p<.05

^***p<.01

Table A4.

Regression Coefficients for Effect of Engagement Experiences on Transfer to a Four-Year College within Six Years of Entry

	Engagement Experience
Variable	Faculty Social		Faculty Academic		Study Group		Clubs
Engagement	−0.103	(0.075)	0.146*	(0.078)	0.092	(0.063)	0.039	(0.096)
Logged Income	0.094***	(0.027)	0.112***	(0.027)	0.085***	(0.025)	0.134***	(0.028)
Pell Grant ($)	0.157***	(0.029)	0.223***	(0.031)	0.179***	(0.029)	0.169***	(0.028)
Parent: Up to Two Years	0.292***	(0.095)	0.223**	(0.098)	0.300***	(0.092)	0.589***	(0.088)
Parent: Up to Four Years	0.033	(0.161)	0.167	(0.162)	0.238	(0.152)	−0.312*	(0.172)
Parent: Bachelor’s degree	0.657***	(0.099)	0.589***	(0.102)	0.553***	(0.098)	0.478***	(0.098)
Parent: Advanced degree	0.524***	(0.114)	0.734***	(0.114)	0.463***	(0.113)	0.431***	(0.110)
Parents Own Home	0.197**	(0.099)	0.268***	(0.102)	0.098	(0.095)	0.097	(0.092)
Single Parent	−0.128	(0.084)	−0.098	(0.087)	−0.185**	(0.084)	0.127	(0.082)
Black	0.006	(0.113)	−0.122	(0.119)	−0.008	(0.112)	−0.183*	(0.109)
Asian	0.267	(0.168)	0.514***	(0.164)	0.279*	(0.166)	0.071	(0.165)
Other Minority	0.297**	(0.148)	−0.040	(0.155)	−0.079	(0.147)	−0.368**	(0.149)
Hispanic/Latino	−0.375***	(0.144)	0.029	(0.145)	−0.077	(0.138)	−0.351***	(0.134)
Female	0.080	(0.078)	0.176**	(0.079)	0.042	(0.076)	0.011	(0.076)
Age	−0.013	(0.010)	−0.041***	(0.011)	−0.033***	(0.010)	0.009	(0.009)
Parental Support	0.022	(0.100)	0.084	(0.103)	0.148	(0.099)	0.457***	(0.098)
Married	−0.192	(0.169)	−0.254	(0.169)	−0.338**	(0.165)	−0.166	(0.170)
Children	0.380**	(0.165)	0.181	(0.161)	0.319**	(0.159)	0.140	(0.161)
Financially Dependent	0.382**	(0.151)	0.076	(0.148)	0.198	(0.146)	0.607***	(0.133)
GED	−0.481***	(0.150)	−0.258*	(0.143)	−0.251*	(0.138)	−0.140	(0.132)
High School GPA	0.246***	(0.062)	0.142**	(0.062)	0.249***	(0.061)	0.204***	(0.061)
AP Credit	0.153	(0.118)	0.327***	(0.117)	0.139	(0.116)	0.159	(0.114)
Years Mathematics	0.275***	(0.032)	0.193***	(0.032)	0.197***	(0.031)	0.223***	(0.030)
Public High School	−0.212*	(0.118)	−0.237*	(0.125)	−0.115	(0.118)	−0.067	(0.124)
Lives On Campus	0.072	(0.137)	0.487***	(0.132)	0.079	(0.135)	−0.122	(0.137)
Full-Time Student	0.395***	(0.086)	0.448***	(0.089)	0.323***	(0.085)	0.084	(0.082)
Distance to Home	−0.003**	(0.001)	−0.006***	(0.001)	−0.002	(0.001)	−0.000	(0.001)
Enrollment Size	0.000***	(0.000)	0.000***	(0.000)	0.000***	(0.000)	0.000***	(0.000)
Percent Minority Enrollment	−0.002	(0.002)	(0.002)	−0.001	−0.004*	(0.002)	−0.001	(0.002)
Percent Federal Funding	0.006***	(0.002)	0.004*	(0.002)	0.006***	(0.002)	0.005**	(0.002)
Humanities	0.410***	(0.136)	−0.201	(0.146)	0.332**	(0.135)	0.381***	(0.122)
Social/Behavioral	0.521***	(0.195)	0.068	(0.209)	0.309	(0.190)	0.445***	(0.164)
Life Science	0.750***	(0.211)	0.017	(0.248)	0.225	(0.218)	0.997***	(0.189)
Physical Science	0.173	(0.529)	1.479***	(0.481)	1.097**	(0.442)	0.929**	(0.425)
Math	1.001	(0.642)	1.170	(1.094)	1.289*	(0.658)	0.481	(0.599)
Computer/Information	0.185	(0.186)	0.004	(0.182)	−0.100	(0.190)	0.197	(0.191)
Engineering	0.373*	(0.201)	0.122	(0.182)	0.185	(0.187)	0.795***	(0.181)
Education	0.412***	(0.143)	0.173	(0.160)	0.344**	(0.143)	0.808***	(0.138)
Business/Management	0.171	(0.120)	−0.085	(0.119)	0.159	(0.117)	0.429***	(0.122)
Health	−0.228*	(0.119)	−0.516***	(0.122)	−0.279**	(0.115)	0.301**	(0.119)
Vocational/Technical	−1.237***	(0.315)	−1.796***	(0.348)	−1.360***	(0.321)	−0.403	(0.258)
Other Professional	0.081	(0.128)	−0.347**	(0.138)	−0.029	(0.128)	0.225*	(0.128)
Hours Worked	−0.013***	(0.003)	−0.012***	(0.003)	−0.010***	(0.003)	0.001	(0.002)
Job Related to Major	0.324***	(0.115)	0.373***	(0.123)	0.299***	(0.114)	0.162	(0.107)
Remedial Courses	−0.023	(0.017)	−0.016	(0.017)	0.001	(0.016)	−0.022	(0.016)
Constant	−4.839***	(0.467)	−3.685***	(0.488)	−3.920***	(0.463)	−6.094***	(0.485)

Notes: : N= 7,010. The “Engagement” coefficient corresponds to the measure the dataset is matched on. Table displays standard errors in parentheses.

^*p<.1

^**p<.05

^***p<.01

Appendix B

Calculating Cohen’s d

To report effect size, a standardized difference in the impact across the treatment and control group, in Table 4, I calculated Cohen’s d (Cohen, 1988) using the coefficients (the adjusted mean difference between treatment (T) and control (C): ${\mu }_T{\mu }_C$) from the full regression tables reported in Appendix A above and the pooled standard deviation (obtained using the standard deviation of the treatment and control groups, weighted by the sample size of each group). The formula is as follows:

$${}^{{\mu }_T-{\mu }_C}∕_{\sqrt{\frac{(N_T-1)S{D}_T^2+(N_C-1)S{D}_C^2}{N_T+N_C-2}}}$$

According to Cohen (1988), an effect size of less than .20 is small, between .20 and .50 is medium, and over .80 is large, though even Cohen warned against too strictly using these as cutoff for interpretation.