A customized belonging intervention improves retention of socially disadvantaged students at a broad-access university

Bolstering belonging at a broad-access university improves persistence.

Abstract

Broad-access institutions play a democratizing role in American society, opening doors to many who might not otherwise pursue college. Yet these institutions struggle with persistence and completion. Do feelings of nonbelonging play a role, particularly for students from groups historically disadvantaged in higher education? Is belonging relevant to students’ persistence—even when they form the numerical majority, as at many broad-access institutions? We evaluated a randomized intervention aimed at bolstering first-year students’ sense of belonging at a broad-access university (N = 1,063). The intervention increased the likelihood that racial-ethnic minority and first-generation students maintained continuous enrollment over the next two academic years relative to multiple control groups. This two-year gain in persistence was mediated by greater feelings of social and academic fit one-year post-intervention. Results suggest that efforts to address belonging concerns at broad-access, majority-minority institutions can improve core academic outcomes for historically disadvantaged students at institutions designed to increase college accessibility.

INTRODUCTION

To succeed in the modern economy, it is increasingly necessary to earn a postsecondary degree (1). The vast majority of aspiring college graduates, including first-generation and racial-ethnic minority college students, attend broad-access public institutions that offer lower barriers to entry (accepting more than 75% of applicants) and greater affordability than more selective institutions. Broad-access institutions can play a democratizing role in society by spurring upward mobility among economically and socially disadvantaged groups (2). Yet, they also struggle with low graduation rates, especially among the many racial-ethnic minority and first-generation college students they serve (3, 4). It is imperative to understand the factors that cause students from disadvantaged social backgrounds to leave broad-access institutions without a degree and to develop appropriate remedies.

Students face many structural barriers in broad-access institutions, including financial constraints (5, 6) and lower levels of academic preparation (7). Here, we ask whether a social-psychological process—persistent worries about belonging—also plays a role. In selective university contexts, worries about belonging among racial-minority and first-generation college students can arise from awareness of the historic exclusion of their groups or families from American higher education and of cultural stereotypes that impugn their intellectual abilities (8–13). Such worries can lead students to perceive common challenges in college—such as struggles making friends or receiving a poor grade early in the term—as signs that they do not belong, promoting disengagement. Field experiments show that, in highly selective contexts, allaying worries about belonging through targeted exercises can improve core outcomes for disadvantaged students (9, 12). These social-belonging interventions use stories from older students to represent academic and social challenges as normal and temporary in the college transition—not as evidence of a global or permanent lack of belonging (9, 12). The seminal social-belonging intervention represented the challenges to belonging that many students who attend highly selective institutions contend with (e.g., feeling intimidated by renowned professors). In that study, reported in Science, an hour-long reading-and-writing exercise in the first year at an elite, predominantly white college in New England raised African American students’ grades over the next 3 years, cutting the racial achievement gap in half (9).

The policy implications of prior research, however, are limited by the fact that most racial-ethnic minority and first-generation college students do not attend elite institutions (2). Can this approach be effective in less selective postsecondary contexts where most students—and most students from backgrounds disadvantaged in higher education—attend college? Could an intervention that targets the belonging challenges these students face—including both concerns shared with other college students, such as making friends or feeling overwhelmed in class; and context-distinct concerns, such as commuting long distances and juggling work and family obligations that can compromise participation in campus life—effectively bolster students’ academic outcomes? To test this question, we adapted the social-belonging intervention for a broad-access institution (see Materials and Methods), integrated it into existing university practice (a first-year writing course) to evaluate it in a way that colleges and universities could practically use to reach entire cohorts, and tested its effect on college persistence.

THE SOCIAL CONTEXT OF BROAD-ACCESS INSTITUTIONS

In focusing on the role of social context, we address an urgent need in the literature. Even as studies have repeatedly demonstrated that brief, social-psychological interventions can improve postsecondary outcomes (14–16), it is unclear where and for whom these interventions may be effective or ineffective. For both theory and practice, it is essential to understand how aspects of the social context (e.g., fewer structural resources and a minority-majority student body) may facilitate or foreclose the ability of students to benefit from psychological treatments (17–20), and especially to test interventions with a strong evidentiary base in contexts where the needs are acute.

It is possible that exercises to address belonging may be less effective or less relevant in broad-access institutions. For instance, if structural barriers are prohibitive, addressing any psychological factor should not help. Belonging, in particular, could be less relevant to students at broad-access institutions that often lack extensive residential facilities and where many students commute to school. Perhaps in such settings, students place less emphasis on the college community. Furthermore, when first-generation and racial-minority students are numerically more well represented, they may be less likely to doubt their belonging (21–23). However, insofar as college is a seminal experience for all students, including students at less selective institutions (1), it is also possible that a sense of belonging in the college community is important generally, a supposition supported by pilot qualitative research conducted for this project (see the Supplementary Materials). In support, correlational data link uncertainty about belonging to lower persistence for first-generation and racial-minority students at diverse mostly less-selective colleges and universities (12). Thus, we hypothesized that an intervention to help students think about belonging productively, as a process in which doubts are normal yet people’s experience improves over time, would benefit persistence at a broad-access institution, especially for first-generation and racial-minority students. We also explored the possibility that benefits would be greatest for academically struggling students from socially disadvantaged backgrounds (17, 24), as academic struggles can undermine students’ sense of belonging (25, 26).

This double-blind, randomized intervention was delivered to first-year students (N = 1063) as a reading-and-writing exercise in all required first-year writing courses at a large, broad-access, Hispanic-Serving Institution in the Midwest with a racially and economically diverse body. Starting with intervention materials that had effectively bolstered students’ sense of belonging and fit at highly selective universities (9, 12), our research team engaged in an extensive customization process in partnership with upper-year students and university administrators to redesign materials that would (i) address the specific barriers and obstacles to belonging that students in this broad-access context face and (ii) model coping strategies that were available and effective in that context (see Materials and Methods and table S20 for details). Students in the treatment condition read stories from upper-year students that highlighted common academic and social challenges to belonging in this context and represented these challenges as normal and temporary. Students were then asked to complete writing exercises to facilitate the personalization and internalization of the core belonging message. The randomized control group completed similar activities, but the content did not focus on belonging (see Materials and Methods).

Our primary interest was among students who face social disadvantage in higher education (N = 606), defined as African American, Latino, and Native American students as well as first-generation college students of any racial-ethnic background, consistent with past theory and research (9, 12). The primary outcomes were persistence through the second year of college and the third year of college, assessed as continuous enrollment. These outcomes are highly predictive of graduation rates (4) and were identified by university administrators as their priority in this college context—and more so than full-time enrollment—because many students enroll part-time during some terms to maintain progress toward degree completion while working or caring for family, for example, as at many broad-access institutions. Thus, institutional processes were designed to support this flexibility, and the university uses continuous enrollment as its index of student retention. Grade point average (GPA) served as a secondary outcome (9, 12), and we largely report these results in the Supplementary Materials, with important highlights in the main text. To track mediating psychological processes, we examined whether the treatment forestalled global interpretations of academic and social adversities experienced in the days following the intervention and assessed students’ feelings of social and academic fit on campus via daily experience sampling surveys (ESS) immediately following the intervention’s delivery and in a 1-year follow-up survey. Last, we were also able to collect anonymized academic records of all students enrolled in the first-year writing course in the year before and the year following the intervention’s implementation; these students formed additional campus-wide control groups and allowed us to examine institutional gains.

RESULTS

College persistence through 1 year after intervention

Examining the primary outcome—college persistence—we found that 76% of socially disadvantaged students in the control condition maintained continuous enrollment over 1 year after the intervention through the second year of college. This rate rose considerably to 86% with treatment [logistic regression odds ratio (OR) = 2.09, Z = 3.21, P = 0.001, d = 0.26]. Socially advantaged students showed a trend in the same direction, but this did not reach statistical significance (control: 77%; treatment: 81%; OR = 1.22, Z = 0.79, P = 0.43, d = 0.10), consistent with prior research (9, 12). Thus, the group × condition interaction was not statistically significant (OR = 1.65, Z = 1.48, P = 0.138) (see Fig. 1).

Fig. 1. Percentage of students who maintained continuous enrollment over 2 years after intervention by term, student group, and condition.

Note: Percentages are unadjusted for baseline covariates. Sample size by group and condition: socially advantaged students, control condition (N = 243); socially advantaged students, treatment condition (N = 226); socially disadvantaged students, control condition (N = 299); socially disadvantaged students, treatment condition (N = 295).

College persistence through 2 years after intervention

In the control condition, we find that 64% of disadvantaged students maintained continuous enrollment over 2 years after intervention. This rate was again higher—at 73%—with treatment (logistic regression OR = 1.53, Z = 2.26, P = 0.024, d = 0.19). Socially advantaged students showed a trend in the same direction, but this did not reach statistical significance (control: 64%; treatment: 69%; OR = 1.22, Z = 0.94, P = 0.35, d = 0.11). Thus, the group × condition interaction was not statistically significant (OR = 1.30, Z = 0.93, P = 0.354) (see Fig. 1).

College GPA

A secondary outcome was students’ GPA. We found that the intervention boosted disadvantaged students’ GPA in the semester immediately following treatment by 0.19 GPA points (B = 0.19, t = 2.81, P = 0.005, d = 0.25); this difference narrowed subsequently such that, over the full 2-year assessment period, the gain from treatment was a marginal 0.11 points (B = 0.11, t = 1.79, P = 0.075, d = 0.16). The Supplementary Materials include full reporting and robustness checks on the GPA results.

Institutional gains

Did the intervention contribute to institutional gains? Analyses that examined the sophomore-year outcomes of all students enrolled in the first-year writing course the year before and the year after the randomized cohort (N = 4094) suggest so (see Table 1; only sophomore-year outcomes were available for these cohorts). As compared with disadvantaged students in these campus-wide comparison groups, disadvantaged students who received the treatment were more likely to maintain continuous enrollment over the next year (86% versus 78%) (logistic regression OR = 1.82, Z = 3.18, P = 0.001). They also earned higher GPAs over this period (mean = 2.75 versus 2.62) (B = 0.14, t = 2.74, P = 0.006).

Table 1. Percentage of first-year students who maintained continuous enrollment through the end of the sophomore year by cohort, student group, and condition.

Continuous enrollment: End of second year	Socially advantaged students	Socially disadvantaged students
Prior cohort (no intervention; N = 1984)	81%	82%
Randomized cohort, control condition (N = 542)	78%	77%
Randomized cohort, treatment condition (N = 521)	81%	88%***
Subsequent cohort (no intervention; N = 2110)	79%	76%

***P < 0.001, statistically different from untreated socially disadvantaged students in the randomized control condition as well as those in the two campus-wide no-treatment cohorts.

Moderation by prior performance

Whom did the intervention benefit most—socially disadvantaged students who began college struggling or those who began performing well? With regard to persistence, it appears that all socially disadvantaged students benefited regardless of their prior performance (condition × first-semester GPA, OR = 1.17, Z = 0.53, P = 0.595; OR = 1.24, Z = 0.80, P = 0.427). Analysis of students’ GPA in the first semester after intervention showed, however, that the greatest gains arose among students who struggled most in their first semester of college (condition × first-semester GPA, B = −0.20, t = −2.13, P = 0.034; see the Supplementary Materials). Additional analyses revealed that the intervention reduced the percentage of socially disadvantaged students in the bottom 10% of the class (see the Supplementary Materials for quantile regression and class rank analysis results).

Psychological process analyses

How did our brief reading-and-writing exercise in a single class session improve disadvantaged students’ academic trajectories over the next 2 years? Past research finds that the social-belonging intervention can prevent disadvantaged students in selective university contexts from drawing global negative interpretations from day-to-day adversities (9, 10, 27). We found a similar pattern in this broad-access context. Daily ESS completed over 9 days immediately after treatment revealed that disadvantaged students in the control condition evidenced a statistically significant negative relationship between daily adversity and daily feelings of social and academic fit in college: the greater adversity disadvantaged students experienced on a day, the lower their sense of social and academic fit (B = −0.022, P < 0.001). The treatment eliminated this association (B = −0.005, P = 0.35; see table S4 for more details), as if daily adversities no longer implied to disadvantaged students a lack of social and academic fit each day on campus.

With a more secure sense of social and academic fit on a daily basis, did disadvantaged students also become more confident about their fit in college more generally over time? They did. One year after intervention, socially disadvantaged students reported greater feelings of social and academic fit on campus in the treatment as compared with the control condition (B = 0.19, t = 2.20, P = 0.029).

Did improved fit contribute to the higher rates of continuous enrollment over the subsequent year (students’ third year of college)? Our ability to examine this question was somewhat constrained by the smaller sample size for the 1-year follow-up survey. (Response rates did not vary by condition or baseline student characteristics; see table S31.) Even so, the mediation pattern was statistically significant [average causal mediated effect (28) B = 0.02, 95% confidence interval = 0.001 to 0.06; Fig. 2]. Although these results are correlational and should be interpreted with caution, they are consistent with the hypothesis that the intervention instigated a positive recursive cycle over the 2-year assessment period, in which disadvantaged students were able to develop a secure sense of social and academic fit in college, which improved their persistence (12).

Fig. 2. The effect of the social-belonging treatment on continuous enrollment among socially disadvantaged students through the third year of college is mediated by increased feelings of social and academic fit reported at the end of the second year.

Note: Sample size from follow-up survey administered end of the second year: socially disadvantaged students, control condition (N = 83); socially disadvantaged students, intervention condition (N = 80). *P < 0.05.

DISCUSSION

Broad-access institutions represent the frontlines of efforts to improve outcomes in American higher education, but this engine of social mobility is constrained by high levels of dropout. We tested a brief, customized reading-and-writing exercise designed to bolster feelings of belonging. It raised continuous enrollment over 2 years by 9 percentage points among socially disadvantaged students enrolled in a broad-access institution. This exercise was integrated into the existing university curriculum—in all required first-year writing classes. It was tested in a manner that universities could realistically implement immediately at scale to reach full institutional cohorts.

The effects observed (ds = 0.2 to 0.3) are commensurate with past efforts to scale social-belonging interventions online (12), and they have practical, economic, and social justice implications. When college students drop out, they each forfeit between $500,000 and $1 million dollars in lifetime wages (29), disadvantaging themselves and their communities, families, and children. Given that most students, particularly students from first-generation and racial-minority backgrounds, attend broad-access institutions, it is especially important to develop theory-based ways to increase persistence in those contexts.

In the present study, we tested the belonging intervention using a within-classroom randomized experimental design, which provides maximal statistical power and precision (30, 31). However, an aspect of this design is that it permits contagion effects, for instance, if treated students communicate the key message to untreated students. Such effects would make it less likely to detect treatment effects; if so, our findings may be conservative estimates. That said, in full-scale implementation, students may talk about their experience. Thus, for both applied and theoretical reasons, it would be valuable for future research to explore contagion effects and their potential benefits to the transmission and dissemination of the intervention message directly. There is some evidence that contagion and related processes can help bolster and sustain effects of psychological interventions in educational settings (32, 33). Ultimately, the goal of full-scale implementation (e.g., no control condition) is to create cascading benefits beyond the individual to engender a whole-school climate of greater belonging and support for all students.

Disproportionate attention to elite institutions is common in education research (34). This can lead to assumptions and policy recommendations that result from a few highly atypical schools. Given variability in social contexts (35), researchers have cautioned against widespread rote implementation of psychological interventions as generic tasks to be carried out (16). At a minimum, two kinds of challenges must be met in moving effectively to new contexts: (i) understanding whether a psychological barrier (e.g., uncertainty about belonging) is at play in a new setting, and thus subject to potential intervention, and (ii) ensuring that intervention materials address this barrier in a way that is authentic and appropriate for the context. An important contribution of the present research is that it provides evidence that concerns about belonging can impede the success of students at broad-access institutions, and it shows that this can be addressed to improve outcomes. To do so, we used a customized design process to identify both local barriers to belonging and local solutions, and we modified materials accordingly [see also (12, 27, 36)].

While the present research demonstrates that a social-belonging intervention can improve outcomes at a broad-access institution, this is no guarantee it always will. Certain challenges, like commuting and balancing school and work, are likely common across broad-access universities, but other challenges might be idiosyncratic to specific institutions. Thus, it is necessary to further understand the educational contexts in which this, and other promising psychological approaches, can be effective. For instance, it is not known whether there is something distinctive about Hispanic-Serving Institutions like the present university or whether the intervention would be effective at other kinds of broad-access institutions. For example, one kind of constraint involves objective affordances: If students lack learning opportunities, if bureaucratic processes are prohibitive, or if needed financial aid is lacking, psychological interventions might not work. Although it is essential to continue structural reforms to support students (37–39), the present research suggests that at least this broad-access institution offered students opportunities to achieve better outcomes. Societal investments can mean that even relatively low-performing schools offer untapped opportunity (17). A second kind of constraint involves psychological affordances (18–20). An essential precondition for the success of belonging interventions may be that a school affords the opportunity for students from disadvantaged backgrounds to belong. If not, even when students enter school with an adaptive mindset about belonging, they may not reap the benefit of these beliefs. It is essential that schools anticipate the worries about belonging that students commonly experience as they come to college, especially from positions of social disadvantage, and then help students to both adopt adaptive mindsets about belonging as a process and create contexts in which students can use this mindset to legitimately and authentically develop their belonging in the institutional context with time.

In conclusion, our experimental delivery of a customized social-belonging intervention at a highly diverse, broad-access university had remarkable effects—increasing students’ persistence 2 years into the future. It did so because it increased students’ feelings of academic and social fit over time. We hope to see such customized interventions implemented and tested at more institutions of higher education in the future, especially within the broad-access institutions that serve America’s historically disadvantaged students—that is, the colleges and universities whose mission it is to create a more egalitarian American society.

MATERIALS AND METHODS

Study site and study design

Our study site is a large, broad-access, Hispanic-Serving Institution in the Midwest that enrolls a diverse student body. Underrepresented racial-ethnic minorities and first-generation college students constitute more than 50% and 35% of the first-year cohort, respectively. More than 85% of students receive federal financial aid, and the vast majority supplement this aid with part- or full-time work during college. Like many broad-access institutions, this university is classified as a “commuter school” with over 85% of the nearly 30,000 students commuting to campus.

Customization of study materials

Replication studies conducted in new environments suggest the importance of customizing materials for the local context (12, 27, 36). Belonging intervention materials should address the specific barriers and obstacles to belonging that people face in the context, and they should model coping strategies available and effective in that context. Using design-thinking and a problem-centered approach (27, 36), we conducted extensive qualitative research (student surveys, focus groups, conversations with university administrators, and review of university reports) to surface barriers to belonging experienced by students at this broad-access institution and the supports available to them. Several themes emerged. First, students valued the campus community and wanted to belong in it. Second, students reported that the need to commute long distances to school, and/or to work multiple jobs, not only reduced their time to study but also compromised their ability to participate in student groups and campus events, which undermined their sense of belonging. We therefore adapted materials to acknowledge these barriers to belonging, to represent them as normal challenges, and to model ways students could overcome them to develop a sense of belonging in college (see the Supplementary Materials).

Study procedure

The intervention was implemented in the spring term of students’ first year of college during an hour-long class meeting in all required first-year writing courses. Students (N = 1063) were randomly assigned within classes to the treatment or control condition. The exercise was described to students as a university-wide effort to learn about students’ transition to college to better serve future students. Students were asked to read nine stories from diverse upper-year students. Treatment condition stories highlighted common academic and social challenges to belonging and represented these challenges as normal and temporary (see the Supplementary Materials). As in previous implementations (9, 12), students were then asked to write about how their own experiences in college reflected the same themes the upper-year students described. Last, students were asked to write a letter to a future student at their university who doubted their belonging during the transition to college. These “saying-is-believing” exercises place students in the role of benefactors not beneficiaries and encourage them to connect the core message to their lives and to internalize it, amplifying its impact (40–43). The randomized control group completed similar activities, but the content focused on study skills not belonging [see the Supplementary Materials; (26)].

Random assignment was effective for prior achievement, preintervention GPA, and disadvantaged-group membership (see table S1); however, slightly more women were assigned to the treatment than the control condition. Therefore, all analyses control for student gender. Manipulation-check measures and completion rates suggest high implementation fidelity (see the Supplementary Materials). Following the intervention, a randomly selected subsample was invited to complete online ESS during the 9 days immediately following the intervention (N = 559 participated). ESS assessed students’ daily experiences of social and academic adversity on campus and their daily sense of social and academic fit, adapted from prior studies (e.g., “Right now, I feel like I belong at <school>,” “Right now, I feel other students at <school> accept me in college”; 1 = strongly disagree, 7 = strongly agree) (Supplementary Materials provide all measures).

One year after intervention, all students were invited to complete a follow-up survey, which examined students’ sense of social and academic fit in college as both a theorized outcome of interest (9, 10) and a potential mediator of longer-term treatment effects (e.g., “I belong at <school>,” “Other students at <school> accept me”; 1 = strongly disagree, 7 = strongly agree); 294 responded (see the Supplementary Materials for all measures). Official student academic records were retrieved for all students from the university for each semester for 2 years after intervention, through the end of students’ third year of college, including whether they enrolled each term. All estimates are raw percentages or means, unadjusted for covariates. All analyses are intent-to-treat (ITT). Statistical tests for treatment effects are from regression models with baseline covariates [preintervention (i.e., first-semester) GPA and gender; see the Supplementary Materials for additional robustness checks and longitudinal GPA results].

As noted above, our primary interest was among students who face disadvantage in higher education (N = 606), defined as African American, Latino, and Native American students as well as first-generation college students of any racial-ethnic background, consistent with past research. All other students were classified as socially advantaged (N = 457). These subgroups were identified before data collection, and the definitions match that used in past research on the social-belonging intervention with broad student samples [experiment 2; (12)]. They also reflect theory and literature, demonstrating that socially disadvantaged students have benefited from social-belonging interventions in more selective university contexts (9, 10, 12). Notably, as a longitudinal field experiment with outcomes over years, this study was conducted well before preregistration was normative and was thus not preregistered. However, the theory, analytic approach, and hypotheses follow directly from past research (see the Supplementary Materials for more details). Robustness checks and disaggregated analyses are also provided in the Supplementary Materials.

SUPPLEMENTARY MATERIALS

Supplementary material for this article is available at http://advances.sciencemag.org/cgi/content/full/6/29/eaba4677/DC1