Research with youth of color in low-income communities: Strategies for recruiting and retaining participants

Abstract

Background

Youth of color from low-income urban communities are crucial participants in research, as their involvement can shape effective, culturally responsive interventions and policy to promote youth health and well-being. These young people, however, are an often-neglected research population, due in part to perceived challenges associated with their inclusion as well as marginalized communities’ justifiable mistrust of research.

Objectives

Based on our experience conducting a school-based randomized intervention trial in Baltimore, Maryland, we present strategies for conducting research with low-income, urban youth of color. We discuss strategies in three domains: university-community partnership development, participant recruitment, and participant retention.

Methods

We reviewed partnership building and recruitment strategies employed by our team across four years of trial implementation and evaluated success of participant retention at our final survey timepoint.

Results

Partnership building was facilitated by selection of a study design that maximized benefits for all participants, promotion of capacity building at partner institutions, and attention to research staff hiring and training practices. Effective study recruitment strategies included personal contact with parents and close cooperation between school personnel and study staff. Providing incentives and collecting multiple types of participant contact information contributed to increased retention rates. On average, those who participated in the final survey timepoint were less likely to be male and Latinx and exhibited more favorable baseline mental health than those who did not, suggesting differential attrition based on youth characteristics.

Conclusions

Lessons learned from this school-based trial can be applied more broadly to research with low-income urban youth of color. Researchers should strive to maximize scientific rigor, minimize harm to vulnerable adolescents and their communities, promote positive research experiences for young people, and provide concrete benefits to those who participate.

Introduction

Nearly 30 million children under age 18 live in low-income households, with African American (61%) and Latinx (59%) children twice as likely to be in this group as white children (28%).¹ Youth living in poverty face ongoing emotional and physical adversities, such as food and housing insecurity and loss of loved ones to violence. Youth of color in these contexts additionally experience stress, trauma, and barriers to academic and workforce opportunity as a result of structural and interpersonal racism, heightening their risk for long-term social, emotional, and physical health problems.^{2 , 3} The resulting population-level inequities in health and life expectancy for low-income youth of color are stark and persistent.^{4 , 5 , 6} It is critical that these young people's experiences and outcomes are represented in research to ensure development of culturally sensitive evidence-based interventions and policies to address their needs. However, socially disadvantaged groups—including low-income youth of color--are often not included in research.⁷

This underrepresentation is due, in part, to the unique challenges and opportunities associated with involving low-income urban youth in research. For instance, low-income adolescents have been found to participate in research at lower rates compared to their peers in middle- and upper-middle class communities.⁸ This type of statistic has led to labeling low-income youth as “hard-to-survey,” a term that may reinforce classist and racist stereotypes of (primarily white) researchers, including assumptions that young people of color in low-income communities are “non-compliant,” “cause trouble,” or are deficient in some way. Yet, the perspectives of communities of color--including youth--regarding research have been shaped by a long history of past injustices perpetrated by researchers. Abuses of power through research, such as the Tuskegee syphilis study and the testing of birth control in the African American community before it was shown to be safe, have bred justified suspicion and mistrust and have resulted in lower participation in research.⁹ Lower participation rates and higher rates of study dropout among urban youth of color are also a result of life stressors associated with poverty. For instance, hardships such as financial strain and housing insecurity are associated with school absences, frequent changes of address, and disconnected phone numbers, which make it more difficult to contact and maintain connections with participants.^{10 , 11}

Researchers have highlighted these issues, emphasized the importance of including racial and ethnic minority populations in research, and shared general strategies for promoting their participation.^{12 , 13} Distinct subpopulations, age groups, and research designs, however, involve different considerations, challenges, and opportunities. Relatively limited information has been disseminated regarding specific strategies for facilitating participation of urban African American and Latinx adolescents in school-based intervention trials, particularly those with long-term follow-up assessments, which may be intimidating to some participants due to the time commitment and study duration. Grape and colleagues described procedures that enhanced recruitment and retention of urban adolescents in a multi-site intervention trial of asthma self-management¹¹; however, their project did not have a primary focus on schools as a partner or research site. Using a case study approach, some research has emphasized the importance of using multi-component, culturally sensitive, relationship-based approaches for recruiting research participants from ethnic minority backgrounds.¹⁴ Additionally, Ezell and colleagues described strategies for reconnecting with urban youth overdue for a 12-month follow-up survey in a school-based study of an asthma education intervention¹⁰ but did not address strategies for initial partnership and recruitment. In all, these studies provide some guidance and insights into challenges of participant recruitment, intervention, and retention, but much more can be learned.

This paper presents strategies for the recruitment and retention of urban, low-income adolescents of color in school-based research. Although our approaches are discussed in the context of school-based intervention research, we believe these methods can be adapted for use with vulnerable youth across multiple research contexts (e.g., school, community, survey, and medical research). Specifically, we draw from our experience implementing a school-based randomized controlled intervention trial (RCT) with adolescents from low-income communities of color, which followed rigorous design and assessment protocols while also attempting to minimize potential harms of participation and provide positive research experiences for youth. In this paper, we describe procedures implemented by our team to promote positive partnerships with study stakeholders, and we summarize strategies to facilitate participant recruitment and participation. Our goal is to offer useful approaches that have been tested in school-based, intervention research to the broader research community who are conducting studies with urban youth of color. We hope to stimulate further reflection and progress in how to balance scientific rigor in research with sensitivity and respect for youth in historically oppressed communities.

Methods

The Project POWER trial and its local context

The goal of Project POWER (Promoting Options for Wellness and Emotion Regulation) was to assess whether a 12-session trauma-informed universal group intervention for 8th graders called RAP (Relax, be Aware, and do a Personal rating) Club enhanced student emotional, behavioral, and academic outcomes as compared with a 12-session active control condition called Healthy Topics. Using community-based participatory methods,^{15 , 16} our team initially adapted RAP Club as a preventive intervention for urban young people from a clinical treatment program called Structured Psychotherapy for Adolescent Responding to Chronic Stress.¹⁷ We subsequently further adapted RAP Club as a school-based universal prevention intervention for middle school students using an iterative process, which indicated the program had the best developmental “fit” with 8th graders, and evaluated initial effectiveness in a pilot randomized study.¹⁸ The final adapted intervention and current trial are described more fully elsewhere.¹⁹ Participants completed a baseline self-report survey about emotional and behavioral functioning at the start of their 8th grade year, then were randomized within schools to receive either RAP Club or Healthy Topics twice per week over a six-week period. Participants completed additional self-report surveys at three further timepoints (post-intervention, four-month follow up, and 9th grade follow up). Teachers assessed participants' emotional, behavioral, and academic functioning at baseline, post-intervention, and four-month timepoints. Academic data (i.e., attendance, disciplinary sanctions, grades, test scores) were also obtained for participants’ 7th, 8th, and 9th grade years.

The RCT was conducted by researchers at the Johns Hopkins Bloomberg School of Public Health, which is a part of the Johns Hopkins University system. Many Baltimore residents--particularly those in the African American community--understandably regard the university and hospital system with feelings of mistrust. Johns Hopkins has pioneered significant advances in medicine, and the university and health system have launched a number of initiatives to promote growth and wellbeing in Baltimore. However, university policies and practices have also often resulted in harms to Baltimore's communities of color. Examples include racially segregated healthcare delivery and research practices harmful to individuals and communities of color, including the unethical use of Henrietta Lacks' cells and a controversial lead paint study conducted in the 1990s.^{20 , 21 , 22} Similarly, Hopkins faculty members frequently conduct studies in low-income communities in Baltimore, but many of those studies have not brought tangible or lasting benefits to community members, raising justifiable community concerns regarding exploitation by researchers. This context was the backdrop in which we interacted with schools and families to engage them in the project.

Project POWER study procedures

University-community partnerships. K-12 co-educational Baltimore City Public Schools were recruited for Project POWER by the research team. Partnerships were initiated if schools did not already administer programs similar to RAP Club or Healthy Topics to 8th graders and were willing and able to: schedule study programs for 8th graders during the school day that did not interfere with core academic classes; identify school counselors, social workers, psychologists, or teachers to receive training in RAP Club and teachers to receive training in Healthy Topics; and provide adequate space for study activities. The study team partnered with each participating school for one year, during which time we recruited and randomized 8th grade students, delivered the intervention and active control programs, and conducted assessments.

School and participant recruitment. Each January, the team's project coordinator emailed all principals of co-educational K-8 Baltimore City Public Schools who had not yet participated to invite them to join the project in the upcoming school year. For principals who expressed interest, we scheduled meetings to describe the project in detail, determine whether a partnership would be feasible, and assess if the program would meet the needs of the school.

We aimed to recruit a volunteer sample of approximately 20–25 8th graders at each participating school. Eligible students were enrolled in 8th grade at a participating school, could speak, read, and understand English, and were not members of self-contained special education classrooms. Except in special circumstances, youth under age 18 require parental permission to participate in research; they are considered a vulnerable population as they are not yet fully mature cognitively, emotionally, or legally, and thus require special protections. In the current study, the school district and university Institutional Review Boards required that each participant provide signed consent from a parent or guardian, as well as their own signed assent.

Participant retention and survey participation. The RCT involved three assessment points during participants’ 8th grade year (baseline, post-intervention, spring follow up) and one assessment during 9th grade approximately one year following program completion to assess program impact on the transition into high school. At each of these points, students completed a self-report survey assessing mental health, coping, and stress, which took approximately 45 min. These assessments were delivered at each school by research staff, and when possible, were administered to all participants at the school as a group to limit the amount of disruption to classes; snacks were provided. The 9th grade survey was the most challenging from a retention perspective: participants could not be surveyed as a group at school because they had graduated from the partner K-12 schools and were scattered across different local high schools. Instead, the study team contacted each participant individually to complete the survey virtually or by phone outside of school time, with the option for the research team to do a home visit and bring a tablet or hard copy of the survey if the participant preferred. We provided a $25 gift card to participants who completed the 9th grade survey.

Identifying effective practices and lessons learned

Partnership building, recruitment, and retention strategies employed across four years of trial implementation were identified and explored for the current study through review of study documents (e.g., research protocols, study team meetings, discussions with stakeholders), discussion with staff members regarding their experiences implementing different strategies, and calculation of school and student recruitment and retention numbers by cohort. We also analyzed 9th grade follow-up survey participation to assess whether participant retention varied as a function of youth baseline demographic and mental health characteristics.

Results

Promoting effective university-community partnerships

Discussion with study investigators and staff identified three factors critical to establishing and maintaining effective partnerships between the research team and participating schools: (1) inclusion of an active control condition, (2) building capacity at partner schools, and (3) attention to hiring and training culturally competent staff. These factors are discussed below.

Inclusion of an active control condition. A study design that included an active control condition--the health education program, Healthy Topics--was selected both to enhance study rigor and to ensure that all participants would receive programming designed to provide new knowledge and skills. With respect to rigor, we compared RAP Club with a program matched in duration, frequency, and extent of contact with caring facilitators but designed to provide skills distinct from those offered in the intervention (health education versus emotion regulation and coping), allowing for testing core components in a more targeted fashion. While it is difficult to fully “blind” stakeholders and participants regarding study conditions in a behavioral intervention trial, inclusion of Healthy Topics also enabled us to frame the study to principals, parents, students, and our own study staff as a comparison of two interventions promoting different types of health and wellness without flagging RAP Club as the primary intervention of interest. From a community perspective, we believe that inclusion of two programs, each designed to be fun and educational, helped promote stakeholder buy-in and facilitated participant recruitment and retention.

Building capacity at partner schools. A key component of the Project POWER trial was to equip schools with the training needed to continue offering programming after the research study ended. At the start of the partnership, the principal identified 1–2 school mental health personnel (e.g., psychologists, social workers, or counselors) to be trained in RAP Club and 1–2 teachers (e.g., gym or health teachers) to be trained in Healthy Topics. These school personnel attended 1–2 days of curriculum training over the summer, attended and assisted with program delivery in the fall, and participated in weekly supervision calls. They were paid for their time and received all materials needed for continued program delivery. The study team also offered schools the option for staff to participate for free in program training in subsequent summers and to receive free consultation in continued use of programming if they wished. These steps to build schools’ capacity for continued program delivery were intended to avoid the common situation in which, once research studies end, interventions cease to be available, and the community receives no ongoing benefits. Our interactions with principals suggested that this capacity building was positively received and facilitated partnership building.

Hiring and training culturally sensitive research staff. When hiring research staff, we gave priority to applicants who had experience and enjoyed working with young people, ideally in Baltimore City or another urban context. When possible, we also hired team members who were culturally similar to participants, which is often beneficial for promoting trust. Research staff members were informed about the impact of structural racism on the interactions of Hopkins with low-income communities of color in Baltimore City to provide context for understanding community perspectives on the university and its research. This also helped prepare staff for common questions and concerns of the community, and phone recruitment scripts were written with these potential concerns in mind. Staff members were also trained in how to interact with school personnel and families respectfully, which in our experience was critically important in establishing and sustaining positive partnerships with schools.

Recruitment of schools and students

School recruitment. School recruitment rates. We identified 92 Baltimore City Public Schools that potentially met study eligibility criteria. We contacted each of these schools via email during our January recruitment periods preceding each new cohort to explore their interest in study participation. Our records indicated that 17 principals declined participation during the initial outreach attempt, and 39 principals did not respond to our communications. Principals declined because of similar partnerships already established at the school, a lack of staffing to assist in carrying out program needs, and a desire for all 8th graders to participate in the programming. Of those schools who expressed interest and met with the study team, seven school partnerships did not progress due to logistical or resource issues (e.g., too much 8th grade programming already in place, scheduling barriers). Our team's capacity to partner effectively with schools increased over the course of the trial as indicated by the increasing number of schools we were able to recruit and retain. We partnered with 6 schools in Cohort 1 (after 2 schools withdrew from the study), 7 schools in Cohort 2 (1 school withdrew), 7 schools in Cohort 3 (3 schools withdrew), and 9 schools in Cohort 4 (1 school withdrew).

Lessons learned. Establishing a solid partnership with each school was fundamental to the success of all subsequent study procedures. Partnership building involved not only conveying potential benefits of participation but agreeing on school resources needed for study success, including adequate space for programming, feasibility of delivering programming during the school day, identifying school personnel who could be trained in programming, and identifying a school “champion” to liaise across the research team, school personnel, and students.

In our experience, principals were often interested in our project because their schools served students with unmet emotional and behavioral needs. However, schools whose capacities or infrastructure were severely limited were less able to effectively support the research. As several partnerships did not progress for this reason, we became more skilled at assessing the school's capacity for partnership during the initial meetings. Over time, we developed a list of screening questions to assess schools' resources and capacities (see Appendix). Although undoubtably schools with fewer resources need interventions as much or more than schools prepared to build research partnerships, we found that attempting project implementation in a school without a basic level of support led to a frustrating experience for all involved.

Student recruitment. Student recruitment rates. We recruited an average of 22 students per school (range: 11–36) across 29 schools for a total of 635 participants out of 1399 eligible students (46%). As shown in Table 1 , within schools an average of 4 students or parents declined participation (8%) and 22 students or parents did not respond to recruitment outreach (46%). We enrolled slightly more than 30 students at several schools as we wanted to accommodate all students who had submitted parent permission and assent forms when possible, and we did not feel that this increase would compromise intervention quality. At other schools, enrollment was lower than anticipated but still adequate for delivery of study programs. In all but one school, we were able to enroll all students who submitted permission forms into the study. At one school, we were unable to enroll 9 students due to limitations in space or instructor capacity; thus, we selected randomly from among students who had submitted permission forms. Our recruitment rate increased each year (34% of eligible students recruited in Cohort 1; 44% in Cohort 2; 53% in Cohort 3; 56% in Cohort 4), indicating that our team's recruitment strategies improved over time.

Table 1.

Participant recruitment results by school and cohort.

Cohort	School	Possible	Declined		No Response		Consented
		n	n	%	n	%	n	%
1	1	89	0	0	67	75	22	25
	2	31	0	0	14	45	17	55
	3a	59	0	0	25	42	34	58
	4	64	0	0	47	73	17	27
	5	53	4	8	28	53	21	40
	6	94	0	0	74	79	20	21
	Subtotal	390	4	1	255	65	131	34
2	7	54	13	24	18	33	23	43
	8	49	6	12	26	53	17	35
	9	65	12	18	35	54	18	28
	10	47	4	9	19	40	24	51
	11	35	5	14	9	26	21	60
	12	34	3	9	5	15	26	76
	13	56	6	11	31	55	19	34
	Subtotal	340	49	14	143	42	148	44
3	14	51	6	12	11	22	34	67
	15	39	4	10	11	28	24	62
	16	69	4	6	43	62	22	32
	17	41	12	29	10	24	19	46
	18	64	10	16	23	36	31	48
	19	23	0	0	0	0	23	100
	20	20	5	25	4	20	11	55
	Subtotal	307	41	13	102	33	164	53
4	21	22	2	9	4	18	16	73
	22	33	1	3	11	33	21	64
	23	46	4	9	23	50	19	41
	24	25	2	8	12	48	11	44
	25	53	4	8	13	25	36	68
	26	63	2	3	36	57	25	40
	27	28	3	11	4	14	21	75
	28	45	1	2	14	31	30	67
	29	47	4	9	21	45	22	47
	Subtotal	362	23	6	138	38	201	56
Total		1399	117	8	638	46	644	46

^a9 students who turned in consents were not enrolled due to limitations in space or instructor capacity at this school.

Lessons learned. Obtaining parental permission can be challenging in our target population because financial pressures result in families’ frequent changes of address and inactivated phone numbers. We developed a multi-pronged recruitment strategy to address this challenge: study descriptions and permission forms were mailed to 8th grade families; study staff visited schools to inform students about the project and send additional forms home with students; and study staff contacted families by phone to follow up.¹⁹ In addition, we periodically re-tried disconnected phone numbers, as it is common for phones to be turned on and off.

We worked closely with each school to identify and honor their preferences regarding parent contact. Some schools preferred for their own offices to mail our forms in their back-to-school packets; others requested we mail our forms separately. Some schools--particularly those with large proportions of Spanish-speaking parents--preferred to make initial follow-up phone calls themselves so that parents would interact with school staff members they already trusted, whereas others requested that our team follow up with parents by phone after forms were mailed.

Contacting households to request parental permission for a child's participation in research is an important interaction between researchers and community members, with potential to produce either positive or negative experiences for families. We took several steps to increase the likelihood of positive contacts. First, we included a brief letter from the school principal on school letterhead with our permission forms to highlight the principal's support for the project, as well as a note from the principal investigator with a simple project summary, as the permission forms are formal legal documents and can be off-putting without additional context. Second, we attempted to reach each family by phone to initiate a personal connection with parents, talk through the consent process, and give them an opportunity to ask questions. This step was crucial not only to allow parents to learn more about the study but also to facilitate recruitment of a diverse selection of students at each school. Restricting recruitment to families who provided permission based solely on receiving a form without further communication would likely result in a biased sample (e.g., more involved, less stressed).

When phoning parents, study team members first highlighted their partnership with the child's school and principal before stating their Hopkins affiliation. Team members explained that school leadership scheduled intervention sessions so as not to disrupt core academic courses, asked if parents had questions, and gave parents additional time to make the decision if needed. They were trained to listen carefully for signs of discomfort or annoyance and to “back off” as needed. If a parent declined a child's participation or requested not to be contacted again, team members accepted the decision without pressuring the parent. Staff training also provided guidance to address common questions and misconceptions. For example, several parents believed that research conducted by Hopkins would entail the collection of their children's genetic material, a reasonable concern given the history of medical research in the community.

We kept detailed logs of each call made, call time of day, messages left, and family members spoken with so as not to burden families with unnecessary repeat calls. In the relatively rare event that we were unsuccessful in reaching a family by phone and were unsure whether they had received the permission form, we attempted a home visit. Relatively early in the study, we received a complaint from a parent who felt threatened when two study team members with whom he was not familiar knocked on his door and did not have identification to prove their affiliation with the study. Following the complaint, we ensured that all study team members carried both Baltimore City Public School and Hopkins ID badges with photo identification when interacting in person with parents or guardians, and we made home visits sparingly.

Additional strategies helped incentivize and streamline the process of obtaining signed parent permission forms. For instance, at each school study staff attended back-to-school nights to describe the project to 8th grade parents. This was a successful strategy for recruitment, although often the number of parents at these events were small. The intervention instructors also visited each 8th grade classroom to present the program and offer a pizza party to classes in which a majority of the students turned in signed parental permission forms by a given date, regardless of whether the parent permitted or declined the child's study participation. We provided pizza parties for all participating schools, as return rates for parent permission forms were high. Names of students who submitted forms were not made public to the class to avoid shaming students whose caretakers did not return a signed form. Research assistants made daily school visits to answer student questions about the study, remind them to have parents sign the permission form, and collect signed forms. Teachers, office staff, and other school “study champions” played a key role in prompting students to share the forms with their parents and collecting signed forms. We provided small honoraria for school personnel who offered extra assistance in these areas.

Parents could return a signed form in several ways: ask their child return the form to their teacher, mail the form to the study team using an enclosed stamped envelope, take a photo of the signed form and text the photo to a secure study team Gmail address, or use an Adobe app to securely sign and email the form (see Fig. 1 ). Adding texting and Adobe app options in Cohorts 3–4 not only decreased participant response times but also increased our recruitment rate by almost 25% compared to using only the mailed consent or return to teacher option in previous cohorts. Finally, study staff reviewed the assent forms in person with students at school, gave them an opportunity to ask questions, and collected signed forms from the students.

Fig. 1.

Consent method by cohort.

Participant retention in the 9th grade survey

Retention rates. As shown in Table 2 , average survey participation rates by cohort ranged between 85 and 100% for assessments conducted in school during the 8th grade (baseline, post-intervention, and four-month follow up). (The lower rate of 41% for Cohort 4 at the four-month follow up resulted from remote survey administration due to COVID-19.) As noted in the Methods section, the 9th grade follow-up survey timepoint posed greater participation challenges as students had to be individually contacted to complete the survey outside the school context, resulting in average participation rates of 36–65%.

Table 2.

Project POWER survey participation rates.

Cohort	Intervention	Enrollment	Assessment Timepoint
			Baseline		Post		4-Month Follow-Up		9th Grade Follow-Upa
		N	N	%	n	%	n	%	n	%
1	RAP Club	63	61	97	59	94	55	87	24	38
	Healthy Topics	59	59	100	53	90	51	86	20	34
	Total	122	120	98	112	92	106	87	44	36
2	RAP Club	71	71	100	64	90	68	96	43	61
	Healthy Topics	77	77	100	62	81	72	94	47	61
	Total	148	148	100	126	85	140	95	90	61
3	RAP Club	84	82	98	80	95	77	92	56	67
	Healthy Topics	80	80	100	76	95	70	88	51	64
	Total	164	162	99	156	95	147	90	107	65
4	RAP Club	101	101	100	97	96	42	42b	January 2021
	Healthy Topics	100	100	100	94	94	40	40b
	Total	201	201	100	191	95	82	41b
All	RAP Club	319	315	99	300	94	242	76	123	56
	Healthy Topics	316	316	100	285	90	233	74	118	55
	Total	635	631	99	585	92	475	75	241	38

Note. Survey participation rates did not significantly differ (p > 0.05) by intervention group at any timepoint.

^aCohorts 1–3 only.

^bLow response rates resulting from remote survey administration due to COVID-19.

Twelve-month survey participation rates increased from 36% in Cohort 1 to 61% in Cohort 2 and 65% in Cohort 3 (Cohort 4 has not yet completed the 9th grade survey). On average, participants who completed the 9th grade survey received 4.4 contact attempts, while those who did not received 6.3 attempts. Contact attempts decreased from 5.98 attempts per participant in Cohort 1 to 3.57 in Cohort 2, indicating that our survey administration methods became more efficient. Ninth grade follow-up data collection with Cohort 1 posed particular challenges because funding to conduct that assessment was received after the initial year of the trial, so that activity was not included in the original consent documents. Therefore, the research team re-contacted participant households a year after the original consent to obtain another signed parent permission and youth assent, without updated participant contact information.

As displayed in Table 2, rates of 9th grade survey completion did not differ by intervention group (RAP Club: 56.4%, Healthy Topics: 54.6%, p = 0.749). However, survey completers versus non-completers did differ on some demographic and mental health characteristics. As shown in Table 3 , 62.7% of students who completed the 9th grade follow-up survey were female versus 53.0% of those who did not complete the survey (p < 0.05). Only 10.0% of survey completers reported being Latinx compared to 17.3% of non-completers (p < 0.05). As shown in Table 4 , participants who completed the follow-up survey reported lower levels of baseline post-traumatic stress disorder (PTSD) avoidance symptoms (p < 0.05) and functional impairment (p < 0.01) and better peer relationships (p < 0.01) as compared with those who did not complete the survey. Baseline self-reported use of alcohol (p = 0.09), tobacco (p = 0.97), and marijuana (p = 0.15) did not significantly differ between those who completed the follow-up survey and those who did not (data not shown).

Table 3.

Demographic characteristics of participants who did and did not complete the 9th grade survey.

Characteristic	Completed Survey
	Yes (n = 241)		No (n = 181)
	n (mean)	%	n (mean)	%	p
Age	13.18		13.26		0.133
Race					0.417
Black	164	68.0	114	63.0
White	26	10.8	16	8.8
Other	12	5.0	16	8.8
Multiple	33	13.7	17	9.4
Unknown	6	2.5	18	9.9
Ethnicity					0.019*
Latinx	24	10.0	32	17.7
Non-Latinx	216	89.6	147	81.2
Unknown	1	0.4	2	1.1
Sex					0.047*
Male	90	37.3	85	47.0
Female	151	62.7	96	53.0

*p < 0.05.

Table 4.

Comparison of baseline mental health scores for participants who did and did not complete the 9th grade survey.

Participated	Yes	No	p	Possible Range
Self-Efficacya	47.59	46.64	0.275	12.2–70.7
Anxietyb	9.52	9.27	0.548	4–20
PTSD Severityc	15.58	18.03	0.062	0–51
PTSD Re-experiencingc	4.72	5.17	0.285	0–12
PTSD Avoidancec	5.82	7.09	0.021*	0–18
PTSD Hyperarousalc	5.11	5.42	0.451	0–15
PTSD Functionalc	1.63	2.16	0.006**	0–6
Depressiond	3.06	3.82	0.062	0–20
Total Distresse	51.93	58.39	0.157	−16–220
Peer Relationshipsf	15.75	14.77	0.007**	4–20
Adverse Childhood Experiencesg	1.78	1.95	0.327	0–8

*p < 0.05. **p < 0.01.

^aNIH Toolbox Self-Efficacy Scale.

^bPROMIS Anxiety Symptoms.

^cCPSS – Revised: Child PTSD Symptom Scale.

^dChildren's Depression Inventory Short Form.

^eYouth Outcomes Questionnaire – Self-Report.

^fPROMIS Pediatric Peer Relationships.

^g2011/12 National Survey of Children's Health.

Lessons learned. We made several changes to improve participation in the 9th grade follow-up survey for Cohort 2. In addition to including the 9th grade follow up within the initial consent documents, a key modification was to obtain updated and expanded contact information for participants at the four-month follow-up survey in the spring of 8th grade. Participants completed a contact information sheet which requested a current address, student and parent phone numbers and emails, which high school the student would be attending, social media handles (i.e., Instagram, Facebook, and “other”), and two additional contacts who would know how to reach the student. Phone numbers and addresses often change, but social media handles typically remain constant, as adolescents seek to preserve their connections and previous posts. Adolescents frequently have more than one social media account on the same platform (e.g., two or more Instagram accounts); encouraging them to provide all account information was helpful for follow-up in case some accounts were more frequently used than others.

Over time, the research team also improved the efficiency of procedures for conducting participant outreach. The outreach protocol developed for the 9th grade follow-up assessment specified the order in which contact information should be used, beginning with sending the survey link through email, then following up with the student over text and social media handles, followed by outreach to parents, then alternate contacts, and finally the participant's high school. Visits to participants' home addresses were made if all other avenues for contacting the participant had not been successful. Whether reaching out to participants directly or to other individuals they suggested, we preserved confidentiality surrounding research participation by not revealing on a voice mail or to another person that the individual was part of a research study. Research personnel used a Microsoft ACCESS database to store and update participant contact information and outreach attempts, as well as participants' program attendance, interventionists they worked with, survey participation, and consent information. This combination of data enabled study staff to personalize their interactions with each participant. Finally, the use of gift card incentives was key to recruiting the students throughout the follow-up assessments.

Despite our attempts to reach all participants, those who did not complete the 9th grade survey were more likely to be male and Latinx, more likely to report PTSD avoidance symptoms and functional impairment, and less likely to report positive peer relationships. These data suggest that participants with certain characteristics may be more difficult to engage in survey activities at follow up and may require additional retention and engagement strategies.

Discussion

This paper described strategies for enhancing research success in urban communities of color with a specific focus on promoting positive university-community partnerships, facilitating participant recruitment, and maximizing survey completion. Key strategies are summarized in Table 5 . The approaches outlined in the paper are by no means comprehensive, nor will they address all practical or ethical issues that may arise in this context. Rather, they were presented in the spirit of sharing lessons learned and generating reflection. While our experiences involved school-based intervention research, the principles and strategies presented likely have broader relevance for conducting research with low-income urban youth of color in a variety of settings (e.g., health clinics, recreation centers, community contexts, phone surveys). In our experience, a key theme is balancing rigor with sensitivity in all aspects of design and implementation.

Table 5.

Lessons learned.

Strategy Type	Strategy	Rationale
General strategies to enhance success of study activities	Include active control program in RCT design	Provide potential benefits to all participants
	Build capacity of partner schools to continue offering programming after the study ends	Sustain long-term intervention benefits
	Hire staff with experience working with adolescents and/or schools in urban contexts	Maximize likelihood that staff will be skilled in interacting with stakeholders and participants
	Educate staff regarding local university-community history and context	Sensitize staff to issues of structural racism in the study context
	Train staff to be friendly and respectful in all interactions with parents, students, and school staff	Promote positive research team-community interactions
Strategies to facilitate school recruitment	Meet individually with interested principals to assess potential for school partnership in detail Use a screener or checklist to assess school/partner readiness for collaboration	Realistically assess partnership potential to avoid failed alliances Reduces the likelihood of schools dropping out right before/during programming
Strategies to facilitate student recruitment	Use multiple methods to provide consent forms from parents (i.e., mailings, sending home with child) and send forms multiple times	Increase likelihood of parents receiving consent forms even if addresses are not current or parents misplace initial forms
	Attempt to follow up with all parents by phone	Develop personal connections, with families, maximize likelihood of broad participation across different types of families
	Provide parents multiple ways to return signed forms (mailing, child returns to teacher, photograph and text signed form, Adobe sign)	Facilitate ease and return speed
	Provide pizza party incentive for classrooms in which most parents return a signed (yes or no) permission form	Motivate student and parent return of forms in timely manner
Strategies to facilitate follow up survey participation	Collect multiple types of contact information for participants (i.e., phone, email, social media handles, multiple parent contacts, two additional contacts, high school name, address)	Enhance likelihood of reaching participants at follow up
	Use consistent protocol for contacting participants, with detailed log of contact attempts to avoid duplicating prior attempts	Maximize efficiency and minimize annoyance to participants
	Provide $25 honorarium for completed surveys	Enhance student motivation to complete the follow-up survey

RCT designs, often viewed as the “gold standard” for scientifically rigorous tests of an intervention, can be problematic from a community perspective. If they are not conducted with care and respect, traditional RCTs have potential to cause harm to young people, including the reinforcement of negative perceptions about research among youth in marginalized communities. As displayed in Table 5, several general strategies can help lay the groundwork for more positive community experiences. Researchers should attempt whenever possible to provide potential benefits to participants in the control arm of the study, such as active control programs if feasible. In addition, building capacity of community partners (e.g., schools, recreation centers, clinics) to sustain interventions after the research period is over is a key component in enabling those partners to continue to benefit from the research. Research staff members should be selected with an eye toward their comfort with, interest in, and prior experience working with urban youth of color and should also be provided with training regarding structural racism in the local context and the role that research may have played in perpetuating racist injustices. Staff training should explicitly address how to speak with parents, caretakers, and young people in a respectful way, adequately address their concerns, and identify signs of potential discomfort and respond sensitively.

With respect to recruitment strategies, partnership development with recruitment sites--in this trial, individual schools--is the initial step and sets the stage for how participant recruitment is likely to unfold. Clear discussions with the principal and other school leaders regarding mutual expectations were essential to successful partnerships in Project POWER. In our experience, discussing and agreeing on specifics early on (i.e. 6-9 months before intervention implementation) was important for identifying problems before substantial time was invested by the research team or the community partner. A brief readable study summary and list of screener questions are helpful tools for guiding initial meetings to explore partnerships, whether it is with a school, community, or health partner. Availability of a “champion” to liaise between each site and the research team is an important item to assess in the context of screening in order to facilitate consistent communication. If significant logistical or personnel barriers emerge in initial meetings, it is often wise to postpone or decide against partnership. Therefore, we recommend that for a study to move forward with a community, health, or survey partner, sites should be screened for a basic level of supports, including a top-level staff or coordinator (i.e., a “champion”) who is invested in the study.

In our experience, reaching parents to obtain permission for their child's participation is the most challenging aspect of recruiting youth into a research study. Given parents' busy lives and frequent changes of address and phone numbers, flexibility and persistence are key to this process. Success is facilitated by using more than one method to send permission forms home (i.e., mail, teachers distribute forms at school, email), multiple attempts to reach parents by phone and at different times of the day, and provision of multiple options for how parents can sign and return forms. For research teams with limited funds, it may be possible to eliminate forms mailed to the home, which can be costly in term of materials and staff time, as our experience suggests emailing forms to parents and having students bring forms home from school is at least as effective as mailing forms. In other contexts, such as youth community centers, health centers, or other in-person groups, identifying a contact who has daily or frequent interaction with the youth could greatly increase consent returns. We also found that providing parents with the option to sign and return forms using the Adobe app or by photographing and texting the signed forms to a secure study email account significantly improved response times and return rates, suggesting these are successful strategies for future use.

Our experience indicates that phone discussions with parents significantly facilitate obtaining parent permission for youth study participation. Study team members reported that the phone conversations were critical because they established trust and rapport with parents, ensured parents understood and approved of the study goals, and addressed parent questions. We found that recruitment staff with the highest recruitment rates were comfortable staying on the phone with parents as long as necessary to mitigate any concerns, guide them through the Adobe sign process, and help them problem-solve how to get the consent form back to us, if needed. For instance, staff were trained to guide parents through the Adobe sign process over the phone in real time, which both provided support to parents and streamlined the consent process. If staff resources are limited, phone calls to parents in the mid to late afternoon on weekdays was the most effective in both finding someone at home and not disturbing them at an inconvenient time (e.g., in the morning or evening when they might be getting ready for work). Weekends and school holidays were also generally a good time to reach both parents and adolescents.

At the same time, persistence must be balanced with respect for parents' privacy (e.g., too many contact attempts can be intrusive), and parent refusals must be honored. Provision of an incentive, such as a pizza party, for return of parent permission forms can be an effective way to boost student motivation and speed the process of collecting forms. Incentives, however, must be carefully planned to avoid ethical problems. For instance, incentives cannot be provided only for students whose parents agree to their child's participation, as this would be a coercive approach. Provision of a small incentive to parents is another option (e.g., a $5 gift card for returning the form). In our experience on various studies, student incentives such as pizza parties are more effective than the small parent incentives, but future research should explore this further.

Obtaining high rates of participation in follow up surveys is particularly challenging when participants cannot be surveyed in groups in the school setting and when substantial time has passed since last contact with the research team. Consistent with recommendations of other researchers,¹¹ we found maintaining contact with participants is facilitated by asking for multiple methods of contact (e.g., email, phone, social media handles), as well as for names and numbers of family or friends that the research team can contact if the families’ personal contact information changes. Similar to Ezell and colleagues,¹⁰ we had success reaching young people via social media, although in our experience, Instagram and Snapchat were most popular platforms, whereas the paper by Ezell and colleagues only highlighted Facebook. This distinction underscores the importance of keeping current with the rapidly changing landscape of which social media platforms are popular among adolescents. Provision of different options for how to complete follow up surveys is also helpful, including online platforms that can be accessed via computer or phone, a hard copy that can be mailed or delivered to the home, or a research team member who administers items over the phone. Maintaining a detailed database to track study outreach increases efficiency and facilitates tailoring communications with each participant.

While we were able to improve our retention rates over time, there was evidence for differential attrition such that males, Latinx adolescents, and adolescents with PTSD symptoms and peer relationship problems were less likely to be retained at the 9th grade follow up. Implementation of additional strategies may reduce differential attrition at longer-term follow up. A “check in” contact via email and social media could be added between the 8th grade and 9th grade follow-up surveys to remind participants about the study, provide them with a study update, update their contact information, and offer a friendly reminder to keep using program skills. Contact information for a study team member who can provide links to different emotional and behavioral health services could also be included, in recognition of the fact that youth who suffer from PTSD symptoms and relationship problems may require additional supports. The “check in” strategy may serve to maintain a more active positive connection between participants and study team members in addition to providing updated contact information. Future qualitative research with study participants may also be useful for clarifying reasons why adolescents choose to continue or stop participating in survey research.

Researchers have identified a continuum of community engagement in research, ranging from no community involvement on the one hand to fully community driven/community-led research on the other.²³ Researchers whose work is intended to benefit a community should familiarize themselves with the options along this continuum and should strive for the highest level of community engagement that is feasible given the parameters of the issue or project. A growing literature on community-engaged research (CEnR) indicates the benefits of CEnR approaches and the increasing availability of evaluative tools and models to guide their use.²⁴ Research involving children and adolescents raises particularly complex questions regarding sharing of power, as the power differential between adults and minors is layered onto the other power hierarchies inherent in most university-community collaborations. Regardless of the type of research conducted, both researchers and community members should be clear and transparent with one another regarding the levels of engagement and power of each partner to avoid false expectations and ruptured trust.

Although initial stages of RAP Club adaptation and testing were conducted in a more community-engaged manner using community-based participatory research, the RCT described in this paper was largely a researcher-driven study, in which the community and its stakeholders (school leadership, parents, students) had relatively little input into project aims, design, or activities. While we advocate the use of greater levels of community engagement even in large trials, we also believe that it is possible to minimize harm associated with researcher-driven study designs. Minimizing harms necessitates transparency with community partners, including the extent to which partners do (or do not) have power to shape study questions and procedures, as well as use of the strategies described above to maximize both rigor and sensitivity.

Conclusion

Inclusion of vulnerable youth populations in intervention research poses potential risks to participants and communities, but we believe that not including these populations is even riskier. Young people experiencing chronic stress and trauma deserve societal structural reforms as well as school-, family-, and community-based interventions that can enhance skills for navigating adversity. We hope that researchers will continue to explore ways to more deeply involve and empower stakeholders in research endeavors, including adolescents, parents, and teachers and share effective strategies that can benefit us all.

Funding

Funding for this research was provided by the Institute of Education Sciences, U.S. Department of Education (R305A160082; PI: Mendelson) and the Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health (1R01HD090022; PI: Mendelson). The opinions expressed are those of the authors and do not represent the views of the U.S. Department of Education or the National Institutes of Health.

CRediT authorship contribution statement

Tamar Mendelson: Conceptualization, Resources, Writing - original draft, Writing - review & editing, Supervision, Project administration, Funding acquisition. Steven C. Sheridan: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Data curation, Writing - original draft, Writing - review & editing, Visualization. Laura K. Clary: Conceptualization, Methodology, Writing - original draft, Writing - review & editing, Visualization, Supervision, Project administration.

Declaration of competing interest

The authors declare that they have no conflict of interest.