Community-based implementation of a paraprofessional-delivered cognitive behavioral therapy program for youth involved with the criminal justice system

Abstract

Psychosocial interventions for common emotional and behavioral difficulties have been developed for use in correctional facilities, yet these programs are largely unavailable upon community re-entry due to a shortage of trained mental health specialists. In this study, we developed and piloted a youth worker-delivered cognitive behavioral therapy (CBT) program for young men at high risk for incarceration receiving services at a youth development organization. We formed a community-academic partnership to support the conduct of research across all phases of this project (2014–2016). We analyzed data gathered through focus groups and individual interviews with program staff, administrators, and young men. This study reports on implementation process and outcomes across phases. Main findings indicate the preliminary feasibility and acceptability of the youth worker-delivered CBT curriculum. We discuss strengths and limitations of our approach and provide suggestions for future studies that aim to implement paraprofessional-delivered CBT programs within community-based organizations.

An estimated 66% of young men in detention meet criteria for at least one mental health or substance use disorder.¹ The most common disorders among youth involved with the justice system are depressive disorders,² attention-deficit hyperactivity disorder (ADHD),³ and posttraumatic stress disorder (PTSD),⁴ with prevalence estimates for these disorders as high as 30% (each). Cognitive behavioral therapy (CBT) programs have been developed to address behavioral and emotional difficulties commonly experienced by young men at risk for (re)incarceration, and are among the most promising evidence-based treatments (EBTs) in the field.^5–8 Yet, young men face significant challenges to accessing EBTs in the community due to (1) a shortage of trained mental health specialists,^9,10 and (2) young men’s ambivalence about engaging in traditional mental health services. Existing EBTs require highly specialized training for therapists and may pose additional burdens in under-resourced settings where providers have high workloads and little time to commit to training.¹¹ Retention in programming remains a significant challenge outside of controlled correctional settings.¹²

Shifting initial engagement in mental health support to community-based settings could extend the reach of EBTs. For example, CBT programs that are educational (rather than clinical) can be implemented within existing employment, education, and youth development service organizations.^13–17 These programs may be the first opportunities to teach youth effective strategies for regulating negative affect and solving interpersonal problems.^{14, 16} Further, educational interventions can be delivered by a wider range of providers, including paraprofessional frontline staff who have access to hard-to-reach populations. Existing programs of this nature are often categorized as “life skills” programs that teach coping skills and focus on promoting well-being rather than treating psychopathology, per se. A focus on well-being may help to reduce mental illness stigma—a known barrier to help-seeking.¹⁸

The present study focuses on the iterative development and feasibility testing of a CBT “life skills” program delivered by youth workers (paraprofessionals) at a community-based youth development organization that serves very high-risk young men who are ambivalent to change. True to community-based participatory research (CBPR) principles, the project relied on a stable workgroup to oversee the design, pilot-testing, and implementation of the intervention. Methodology was guided by Intervention Mapping (IM),¹⁹ staged formative and summative evaluation,²⁰ and Proctor’s implementation outcomes (feasibility, acceptability, satisfaction).²¹

Methods

Community partner and participants.

Roca, Inc. is a youth development organization with a Young Men’s Program that serves high-risk men ages 17–24 across four sites in Massachusetts. Roca defines young men as high-risk if they (a) have a history of arrest or incarceration, are gang-affiliated, struggle with substance use, and/or have dropped out of high school; and (b) are not ready, willing, or able to participate in programming and/or gainful employment on a consistent basis. Roca, Inc. is known statewide as the youth development organization that accepts young men who may not be eligible for services through other youth development organizations due to their high-risk status (e.g., risk to self or others, active substance use, noncompliance with organizations’ rules). Roca, Inc. provides educational, employment, and life-skills programming with the long-term goals of reducing recidivism and increasing employment.

In the present study, participants were Roca staff (i.e., community working group members, youth workers, educational/vocational instructors, crew supervisors) and young men enrolled in Roca’s program. A description of each of these participant groups can be found in Table 1. The study received expedited approval from the Institutional Review Board at Massachusetts General Hospital. Inclusion criteria for staff participants were as follows: (a) at least 18 years of age; (b) employed by Roca; and (c) willing to have the focus group or interview audio-taped. Inclusion criteria for young men were as follows: (a) between the ages of 18 and 24; and (b) willing to have the focus group or interview audio-taped. Study information sheets were distributed in person and spoken consent was obtained from each participant.

Table 1.

Participants

Participant type	Description
Community working group (CWG)	A group comprised of investigators, organization leadership, site directors and supervisors, and additional key stakeholders within the organization. The CWG assisted with development, design, and packaging of the CBT program, interpretation of findings, and selection of implementation strategies.
Youth worker supervisors	Depending on the site, supervisors were either in a middle management position between the youth workers and site director, or, the site director. Supervisors were responsible for supervision of youth workers, which involves implementation of all aspects of Roca’s programming (outreach, case management, engagement in education, employment, and life skills programming). For this study, supervisors were pivotal in the training and ongoing supervision of youth workers in CBT at their site.
Youth workers	Youth workers are responsible for engaging a caseload of 25 high risk young men in programming as well as working with community partners (police, probation officers). Youth workers are also the primary staff members responsible for delivering life skills programming. For this study, youth workers received training and delivered CBT to young men using both classroom and outreach format. Youth workers are non-clinical frontline staff who are diverse in terms of race/ethnicity, education, and lived experiences; some of whom have their own history of street/gang involvement, incarceration, or substance abuse.
Educational/Vocational instructors	Instructors work with young men to develop the skills they need to succeed at school and work. They provide training in job preparedness courses (e.g., preparing resumes, interview skills), financial management courses, and prepare young men to take the high school equivalency test. For this study, instructors received training as part of the CBT milieu.
Crew supervisors	Crew supervisors are charged with coordinating and supervising young men who participate in the transitional employment programming. They serve in the role of manager to young men on work crews (subsidized employment), yielding up to 26 hours of contact with young men per week. For this study, crew supervisors received training as part of the CBT milieu.
Young men	Ages 18–24, engaged in Roca programming. Inclusion: have a history of arrest, a history of incarceration, are gang-affiliated, struggle with substance use, and/or dropped out of high school. The young men in the current study had received at least one “dose” of CBT programming.

Intervention component selection: CBT life skills.

A meta-analysis of 58 studies that investigated core components of existing CBT programs for adult and juvenile offenders⁸ found that programs that focused on skills training in anger control (i.e., emotion dysregulation) and interpersonal problem-solving had the strongest effect on outcomes (emotional, behavioral, and recidivism-related outcomes). From the extant mental health literature, emotion dysregulation has emerged an underlying mechanism for a host of behavioral and emotional problems, including problematic substance use, aggressive or criminal behaviors, and general mental health.²² Studies on the psychophysiology of violence suggest that anger and emotional aggression (which is often impulsive, uncontrolled, and reactive) may be due to an individual’s inability to regulate emotions (or physiological arousal).^{23, 24} The cumulative effect of stressors such as interpersonal violence, bereavement, discrimination, and incarceration in this population are theorized to contribute to emotional dysregulation. Thus, resilience factors such as effective coping skills and engagement in social supports may help to mitigate the association between cumulative stressors and emotional and behavioral dysregulation (see Figure 1).

Figure 1.

Logic model: Relations between stressors, resilience factors, and emotional dysregulation

Our intervention aims to equip young men with effective emotional regulation and interpersonal problem-solving strategies. Given that engaging and retaining high-risk young men in behavioral health programming is a monumental challenge involved in implementation of EBTs in community settings, our program was developed iteratively using feedback from the organization (staff and administrators) as well as young men, to ensure the fit of the program to the population and setting, and to promote engagement. The curriculum and training program provide detail on how CBT can be implemented through formal practice (i.e., 30–45 minute meetings, drop-in group format, traditional classroom setting) and informal practice (i.e., individual/outreach format).

Interventionists: Youth workers.

The curriculum was designed and packaged to accommodate flexible delivery by paraprofessionals—in this case, Roca’s youth workers, who are non-clinicians with a diverse range of educational and lived experiences (described in Table 1). Youth workers are racially and ethnically diverse, and some have their own experiences of incarceration or gang/street involvement. Most young men (87%) and youth workers (82%) identify as racial and ethnic minorities. Youth workers are slightly older than the young men enrolled in the program (median: 29 v. 21 years). Youth workers and young men reported high exposure to potentially traumatic events. Young men who completed an anonymous survey (N=104) reported an average of four types of potentially traumatic events (M=4.2, SD=3.5), with the most common being sudden unexpected death of someone close to you (52%), physical assault (50%), and assault with a weapon (50%). In comparison, youth workers (N=23) anonymously reported exposure to an average of seven types of potentially traumatic events (M=6.6, SD=3.9), with the most common being physical assault (78%), transportation accident (70%), sudden unexpected death of someone close to you (70%), assault with a weapon (52%), and sexual assault (35%).

Data collection and procedure.

Our implementation-focused approach to curriculum development and feasibility testing was guided by IM (See Table 2 for description of steps).¹⁹

Table 2.

Intervention Mapping Steps by Study Phase

Formative Evaluation	Y1:Q4	Y2:Q1	Y2:Q2	Y2:Q3	Y2:Q4	Y3:Q1	Y3:Q2	Y3:Q3	Y3:Q4
Step 1: Logic Model of the Problem
Establish and work with planning group	X
Conduct needs assessment to create logic model	X
Describe the context for the intervention	X
State program goals	X
Step 2: Program Outcomes and Objectives
State expected outcomes for behavior and environment		X
Specify performance objectives		X
Create logic model of change		X
Step 3: Program Design
Generate program themes, components, scope, sequence		X
Choose theory and evidence-based change methods		X
Select/design practical applications to delivery change methods		X
Step 4: Program Production
Refine program structure and organization			X
Prepare plans for program materials			X
Draft messages, materials, and protocols			X
Pretest, refine, and produce materials			X		X		X
Pilot 1				X
Pilot 2					X	X
Summative Evaluation	Y1:Q4	Y2:Q1	Y2:Q2	Y2:Q3	Y2:Q4	Y3:Q1	Y3:Q2	Y3:Q3	Y3:Q4
Step 5: Program Implementation Plan
Identify program users, adopters, and maintainers							X
State outcomes and performance objectives for program use							X
Design implementation interventions							X
Step 6: Evaluation Plan
Write effect and process evaluation questions							X
Develop indicators and measures of assessment							X
Specify the evaluation plan							X
Implementation & Evaluation
Feasibility trial								X	X
Assess feasibility and acceptability									X
Summative evaluation									X

Y=Year; Q=Quarter; Y1=2014; Y2=2015; Y3=2016.

IM Steps 1 and 2: Needs assessment and development of logic model.

A series of semi-structured interviews and focus groups were conducted to assess the needs of the organization, staff, and young men. Focus groups and interviews were planned to identify anticipated barriers to the implementation of a new life-skills program. We asked staff and young men to identify topics, problem behaviors, and emotional difficulties that they believed would be relevant to address within the life-skills program. Individual interviews (n=17) and 13 focus groups (n=73) were conducted with staff and young men. All interviews and focus groups were audio-recorded. Investigators directly observed educational and employment programming and shadowed youth workers on outreach visits. Observations were documented in audio-recorded field notes.

IM Step 3: Curriculum development.

The initial draft of the curriculum was guided by the logic model (Figure 1), the extant literature on CBT programs for juvenile and adult offenders,⁵ and the needs assessment (Steps 1 and 2). We selected 10 common CBT components (“skills”) for emotion regulation and problem-solving (see Figure 2). The working group convened for five consecutive weeks to review the proposed CBT skills and offer detailed feedback on fit for young men at Roca (e.g., literacy level), as well as fit for youth worker delivery.

Figure 2.

CBT skills categorized by CBT cycle components: thoughts, emotions, and behaviors

IM Step 4: Iterative pilot testing.

Based on feedback from the workgroup and our aim to reduce provider burden in delivering the intervention, we developed a variety of educational materials, a concise manual, PowerPoint presentations, posters, and pocket-sized cards on key rings. Training and technical assistance shifted from researcher-driven (external) to blended (dual) facilitation over time.²⁵

Pilot 1: Supervisors and youth workers.

Psychologist researchers (LM and SV) trained supervisors, who then trained their youth worker supervisees. Youth workers delivered CBT skills in both formal (class-based) and informal settings (street outreach, during educational or employment programming). Training was experiential rather than didactic,²⁶ and proceeded on a weekly basis, whereby youth workers learned one skill each week, then practiced delivering that skill over the course of the following week. We selected this training approach in light of the needs assessment and workgroup feedback that suggested the need for a self-practice. Youth workers and young men participated in focus groups in week 5 (to review Skills 1–5) and week 10 (to review Skills 6–10). In total, we conducted seven focus groups with youth workers (n=50 total, with n=25 in each round) and eight focus groups with young men (n=61 total). Feedback from these focus groups informed revisions to the CBT curriculum and implementation plan.

Pilot 2: All staff.

The aim of the second pilot was to integrate CBT skills practice into all phases of programming, thus creating a CBT milieu (“culture of CBT”). In this connection, psychologists (LM and SV) and Roca’s Chief Programming Officer (AC-M) trained and provided consultation to all frontline staff in the CBT curriculum across each program at Roca (see Table 1 for description of various frontline staff). Training progressed with the introduction of one skill weekly, which staff then practiced over the course of the following week. Staff received ongoing coaching and supervision from their respective Roca supervisors. Training groups for staff were split by job description at two sites (i.e., one training with educational/vocational staff, and a separate training with youth workers), and one site held a combined staff training (five training groups total). Staff and young men participated in two rounds of focus groups (mid- and end-point; 18 staff focus groups [n=97 staff total]; 8 young men focus groups [n=39 young men total]). These focus groups informed revisions to the curriculum. This step resulted in the final version of the CBT program, including a manual and an implementation plan, to be tested in the feasibility trial.

IM Steps 5 – 6: Feasibility trial.

A series of four focus groups with youth workers (n=23) and individual interviews with young men (n=17) were conducted to assess the feasibility and acceptability of the implementation package, and young men’s satisfaction with the intervention. Focus groups and interviews were audio-recorded. Youth workers were asked to describe the feasibility (i.e., the degree to which a new treatment or intervention could successfully be used within a given organizational setting)²¹ and acceptability of the intervention, and the perception among stakeholders that the proposed treatment or service is suitable, fitting, or satisfactory²¹ to the curriculum (e.g., “Thinking about the final CBT program, what would you say were the greatest successes or strengths?”). Youth workers were also asked about the feasibility and acceptability of implementation strategies (e.g., “What do you think needs to happen in order for the CBT program at Roca to keep running?”). Young men were asked about their satisfaction with the program (e.g., “What did you like/dislike about CBT? What would you change about the CBT program at Roca?”).

Fidelity and Competency.

We developed checklists for each of the 10 CBT formal classes. The checklists were used to gather both self-reported adherence and observer (researcher) reported adherence (Dates Administered: July 1, 2016- January 1, 2017). Adherence to the manual (% of content delivered [yes/no]) was high for both self-report (94.7%) and researcher reported assessments (81.3%). Additionally, first and senior author conducted one-one-one interviews (“CBT Drills”) with all youth workers to assess competency. Youth Workers were rated for their understanding of and ability to explain the teaching points associated with each of the 10 skills. Competency was rated on a scale from 0–6, with 0 representing “not at all, or executed so poorly it may have had a negative impact on a young person,” up to 6 representing “excellent, textbook quality (or very close), would show it to a class.” Youth worker were also assessed on their knowledge of the main components of a CBT class, and their ability to describe how they would apply two randomly chosen skills to a specific young person on their caseload. Across all sites, the mean competency was 3.76 (SD=1.2; Range: 1.9–5.8), suggesting adequate competency for the purposes of this study (3 is minimum standard). This rating falls between a 3 “Did okay, but many opportunities for improvement” and a 4 “Good skill, minor feedback still needed.”

Data analysis.

Focus groups, interviews, and researcher field notes from Steps 1–4 were analyzed using rapid coding methods.²⁷ Per Neal et al.’s guidelines,²⁷ the following steps were taken in the present study: 1) the key research focus was identified (i.e., feedback on the intervention and its implementation); 2) key themes were identified based on both previous experience and by reviewing the audio, generating a codebook; 3) a coding form was created to serve as a tool during rapid coding; 4) data elements were coded using the coding form; 5) coded data were analyzed for further emergent patterns and themes. Focus groups and interviews from Steps 5–6 were thematically analyzed using methods suggested by Braun and Clarke,²⁸ whereby researchers familiarized themselves with the data; generated initial codes; searched for, reviewed, and defined themes; and then described themes following analysis. For all phases, we used consensus (team-based) approach to coding, whereby two research assistants and the first author independently reviewed transcripts to generate an initial coding scheme. This process was repeated until no new themes were identified in the data. Once the coding scheme was finalized, the two research assistants double-coded all transcripts. Then, the team of three met to review codes, discuss, and resolve discrepancies.

Results

Findings from rapid coding in Step 1–3 (needs assessment, logic model, and curriculum development) are summarized in text. Findings from Step 4 (iterative pilot testing) formative evaluation are summarized in text. Data-driven adaptations to the implementation package based on the formative evaluation are detailed in Table 3. We report Steps 5 and 6 (feasibility testing and summative evaluation) findings regarding feasibility, acceptability, and satisfaction in text.

Table 3.

Curriculum and Implementation Strategy Refinement based on Pilot 1 & 2

Pilot 1 Refinements

Pilot 2 Refinements

Engagement. Youth workers need to link CBT to young men’s goals Youth workers need to spend more time providing rationale for each skill Create more visuals for the class-based curriculum Create a list of activities to help young men bring attention to the lesson, and to demonstrate the skill Create a positive reinforcement system for engagement* Create a plan for keeping young men engaged for the long-term (after receiving all 10 skills)* Live updating schedule of classes (available online)* CBT materials and setting. Create posters of CBT skills to place in common areas Ensure each site has access to necessary equipment Ensure Youth workers prepare teaching materials in advance of classes CBT classes should be conducted in closed rooms rather than open spaces Create “toolkit” to make the skills more portable (key ring with cue cards for skills) Training & Coaching Needs. Provide “CBT 101” module for training new staff Training should include basics of teaching and running a group How to handle disruptive behaviors in group Adjusting groups to participant’s skill level* Implementing a closed-door policy after a certain time Time management: Adhering to 30 or 45-minute plan Language Fit. Adaptations to enhance cultural/age fit and clarity (e.g., replace “mindfulness” language with “being present”)

Supervision. Develop CBT supervision model that fits Roca Implement weekly CBT-style Supervision* Implement system to monitor CBT supervision fidelity* Benchmarks for Dosage. Set clear expectations for “dosage” differing by job description* Set organizational standards regarding structure of formal CBT delivery (criteria for formal vs. informal delivery style in regard to dosage) Fidelity. Develop fidelity monitoring checklists for CBT (to be reviewed by supervisors) Implement fidelity monitoring tool in supervision* Competency. Set an organizational expectation of staff CBT competency* Create a standard CBT assessment (knowledge) Assess youth worker competency Assess supervisor mastery of CBT* Training & Coaching. Field coaching on group management and CBT style Pair master CBT instructors with staff who are struggling Invite non-youth worker staff to shadow classes Create a workshop on applying CBT to specific scenarios (domestic violence, substance abuse)* Develop training videos for each skill* Hold “case conference” staff meetings Communication. Conduct weekly CBT instructor meetings to plan and prepare for CBT class Improve networking and communication across the organization (e.g., relay CBT goals and progress across all points of contact with the agency) Expand library of example handouts that apply to difficult, but common, issues that young men face Minor changes to Curriculum. Re-arrange sequence of session to maintain attention (i.e., demonstrations and activities first) Create catchy titles for skills Add/remove specific activities Refine presentation slides, scripts, and handouts Create a “cheat sheet” that includes teaching points for each skill Develop questions to assess young men’s knowledge of CBT teaching points (for advanced groups)

^*Recommendations that had not been implemented as of 12/31/2016.

IM Steps 1–2 (needs assessment and logic model).

Interviews and focus group responses provided information on strengths and challenges of life-skills programs that existed at Roca prior to the research collaboration, as well as the specific emotional and behavioral needs of young men.

Existing life-skills curricula.

Based on their experiences with manualized CBT programs at Roca, both staff and supervisors described training needs that, if addressed, could facilitate the implementation of a new CBT program at Roca. During interviews, staff members described the following challenges of previous curricula: 1) programs were too bulky and burdensome, and involved too much jargon and complexity, and too high a level of literacy; 2) the delivery burden of the program was high, and restricted to formal delivery only; and 3) the training burden of the program was too high. Manuals used by Roca prior to this project were intended for professional delivery in controlled environments, and required highly specialized training and intensive interventionist time to deliver (up to one year). Staff reported marked difficulty translating life-skills training to practice, including difficulty with identifying opportunities to use skills with young men conversationally. Staff reported that there was a need for additional training and consultation, including more opportunities to behaviorally rehearse and receive feedback on their competency in delivering life skills.

Young men’s emotional and behavioral needs.

Staff and young men identified the emotional and behavioral needs of young men that could be addressed through CBT, including poor emotion regulation, interpersonal problem solving difficulties, rigid thinking patterns, avoidant coping styles, and distress related to trauma or loss. Staff and young men described young men’s symptoms of depression, anxiety, posttraumatic stress, and substance abuse.

IM Step 3 (curriculum development).

Working groups offered specific feedback on the proposed curriculum in several areas including content as well as design choices to make a simple and easy-to-learn curriculum, with culturally- and developmentally-relevant metaphors and activities to allow young men to learn and apply CBT skills. Working group members suggested design changes to make CBT more accessible to young men. For example, they suggested the inclusion of “mantras” or “catchy slogans” related to each skill that would be easily remembered. Feedback on the presentation of individual skills, including ideas for imagery or metaphors that would most effectively explain CBT concepts, included suggestions for the use of the imagery of a speedometer to help explain the concept of emotional intensity, and its relationship to “feeling out of control.” Working group members also helped to brainstorm interactive activities that could help young men engage with and learn CBT skills, such as YouTube videos, games, breathing exercises, obstacle courses, and mindfulness playlists.

IM Step 4 (iterative pilot testing).

Feedback and recommendations for refinement of curriculum and implementation strategy are presented in Table 3. A summary of recommendations from each pilot test are summarized below.

Pilot 1 feedback.

Youth workers and young men requested adaptations to increase the accessibility and relevance of CBT curriculum content, and requested supplemental visual and multi-media material to make classes more engaging. Youth workers requested group management training (e.g., setting and keeping an agenda, managing disruptive behaviors), allocated time to prepare for CBT classes, and a cap on class size/late arrivals. Youth workers also expressed a desire for coaching on how to use self-disclosure within the CBT model.

Pilot 2 feedback.

Staff and young men requested additional visual tools to use in CBT classes (slide decks, posters, key rings with main teaching points), and suggested that a printed CBT manual (rather than electronic library of materials) could be beneficial during training (so that youth workers could take notes). Staff reported a need for CBT-specific supervision. Staff reported unclear organizational expectations regarding CBT delivery benchmarks, resulting in sub-optimal adoption of CBT skills among staff. Staff described the inconsistent delivery structure of CBT classes, and a need for more organizational standards for CBT classes. Staff also reported a need for improved communication and coordination between CBT instructors, to reduce redundancy of CBT class content (e.g., repeated activities).

IM Steps 5–6 (feasibility trial).

Data were collected from youth workers and young men during this phase about the feasibility and acceptability of the CBT curriculum at Roca. Example themes and quotations from findings are outlined below. Potentially identifying details have been removed or changed to protect the identity of individuals who participated in these interviews.

Youth workers’ feasibility and acceptability of the intervention. CBT provides concrete skills and a common language. Youth workers observed that young men were talking positively about CBT to others. As one youth worker reported,

I think the greatest success about CBT, [for] me personally I’ve been noticing the young participant always talking about it. …, the more they talk about it, the more [CBT] spreads to other participants. Not only that, but I’ve also heard some young participants telling outsiders about CBT: that’s a success because words are going around and I think it’s working… I was in the [music recording] studio with a young participant and he spoke about CBT in his lyrics, and how it helped him, that’s a success.

Young men find CBT relevant to their lives. Youth workers described how the young men were able to relate CBT lessons to their own lives. They also noted that as time progressed, they no longer had to “sell” CBT to get young men engaged. One youth worker explained,

I don’t believe that I have to sell it [CBT]. …when we start talking about the teaching points in the classes…they feel it. They feel that maybe this can work. I just did Flex your Thinking (cognitive reappraisal), with the gentlemen, and everybody was going through something. …it was a good class because a lot of them are in the extreme [thinking]. If their mind is set on the thing, it’s that and nobody can change their mind. But during the handout, guys were able to actually look at what they are thinking and maybe [think through how] ‘this thought ain’t really the best thought for me…maybe this [behavior I want to engage in] isn’t what I’m really supposed to do…’

Young men are more open to addressing larger problems. Youth workers reported that young men were more aware of and interested in taking steps to address larger issues in their lives (e.g., domestic violence, accessing professional treatment) than they had been previously. As one youth worker said,

[CBT] can be used as a stepping stone for them recognizing maybe they need more help in certain areas of their lives, whether it’s like domestic violence or anything like that, it kind of opens their eyes to ‘Oh, do I have control over my behaviors?’.

Young men are using CBT in their relationships. Youth workers observed that young men were discussing CBT with people outside of Roca. As one youth worker explained,

I thought… [for] a lot of these guys…it will be difficult to implement the whole process, but … it’s been a lot easier than I thought. As a matter of fact, I was in [participant]’s school yesterday and he had a friend there…asking about the program, about Roca, like ‘I want to get in’ and [participant] was telling [him about] the program. Then I sat down to do CBT with [participant] and I had the friend do CBT as well. … [we did] Being Present (mindfulness) and [the friend]…was like ‘I think about the past a lot, man, you know, I need to focus on right now!’ and I was like ‘That’s what this is all about!’.

Young men are having fewer behavioral problems. Youth workers reported that young men were having fewer and less severe behavioral problems. As one youth worker described,

Today… my participant who has incredible anger problems, he destroys property, that’s what he does… So he had a fight with his girl, she kicked him out, he was still there yelling and he calls me, he’s like “I’m riding the wave, I’m riding wave” (Feel Your Feelings; distress tolerance). And then he left and obviously he’s still [going to] have a lot of emotional problems with that, but no physical action was taken. He didn’t bust up anything in her house, he didn’t break any windows… I didn’t think I was ever going to get him to stop doing that. So for me that’s like ‘Alright, yes, this stuff is working.’

Young men are showing shifts from short- to long-term thinking. Youth workers observed young men opting for long-term [vs. short-term] solutions to their problems. For example,

They [young men] are making better decisions outside, making better decisions on the street. Slowing down, thinking about consequences, noticing when they are feeling elevated…[Getting] them to… [use] all of [the skills] that they got… all together [helps them to make] a better decision and therefore [have] better consequences, better results.

CBT improves engagement in other programming. Youth workers observed young men using CBT to help them better engage in other areas of programming, such as employment and education. A youth worker described how one young man signed up to take a high school equivalency test that he had been anxiously avoiding, by applying the Approach! Don’t Avoid (exposures and behavioral experiments) skill. As one youth worker explained,

I had this kid who I have been struggling with for a long time, probably because he is really young. But we’ve done a lot of CBT and yesterday he comes to me… and he goes ‘You know what I did?’ What did you do? ‘I approached!’ I asked him what he approached and he was like ‘I approached, gave him [educator] 50 bucks, I’m doing my test!’ … I told everyone, I was proud. It was heartwarming.

Young men are using CBT independently. Youth workers expressed seeing young men using CBT skills on their own, without prompting. As one youth worker reported,

We had a young man who… [just received news that about] a job that he didn’t get…so we had a conversation, and we were talking… and he was like ‘You know what, I don’t really want to hear that, I already thought about all my options, I thought about my pros, I thought about my cons [problem solving], and I’m going to sit with you and I’m going to do applications.’ So it just kind of hit me because he was really pissed off that he [didn’t] get that… job, believe me, so when he said that I was like, wow.

CBT is helpful for staff. Youth workers reported that CBT was a useful practice for not only their young men, but also themselves. As one youth worker described it,

To be honest, in the beginning even I didn’t like it [CBT], but the more you teach it, the more you learn, it’s like you always find something like, ‘This applies to me,’ and I can also apply this in my personal life.

Young men are engaged by CBT visual materials. Youth workers reported that they and the young men were satisfied with the supplemental visual materials. One youth worker noted,

When you all came out with the flashcards, that they put out in the offices now and the big white sheets, guys can actually see those, as we are speaking to them, they were able to visualize what we were teaching them, with the [main teaching] points.

The CBT curriculum should be expanded. Youth workers discussed the need for a “deeper” CBT class that involved more specific targeting of major life issues, including other cumulative stressors (e.g., racism, discrimination, poverty, familial conflict, victimization), and required more self-disclosure from young men. As one youth worker reported,

I think CBT … works because it hits a process to get them to pause and think about things, I think it should have more layers to the onion, though....We have some situational issues and we have maturation issues, some things are just right now at a moment and time we are dealing with, and some are so deep-rooted, like no father, no mom, no nothing, no belief, they drop out of all the systems around them, how do we look at that, how do you view that? You have to deal with the lack of trust for people because of other folks in the city, and I think those are blockades [when you] deal with this population.

Refreshing CBT curriculum content may improve engagement. Youth workers recommended regularly refreshing activities and demonstrations in the curriculum. As one youth worker explained, “All I hear is ‘Oh I did this one already,’ ‘Oh we are doing this one again?’ and ‘Oh I did all of them.’ I have sat in and I hear a lot of people say that.”

Youth workers’ feasibility and acceptability of the implementation procedures. CBT is attractive to new employees. Youth workers reported that Roca had started using CBT as a point of interest for new hires. As one youth worker noted,

When I got hired here, [a youth worker] was the one who told me that before CBT, this place was a madhouse, participants [were stealing] chairs, breaking computers, smashing stuff, fighting everywhere, and [the youth worker] told me once CBT came out here… [there was] silence [meaning the situation got better]. That was one of his selling points about me applying.

Youth workers need additional CBT training and consultation. Due to staff turnover over the course of this multi-year study, youth workers who were involved during the final phases reported a wide range of exposure to CBT training provided by the research team, in which some had received 30 or more hours of training/consultation while others had experienced only two to three hours of initial condensed training. Youth workers suggested the need for a sustainable training model, and application of real-time observations and feedback. As one youth worker described it,

Training definitely helps, [but] I think you just got to start teaching, listen to some other youth worker teach it, and kind of like draw what you need and put it into your own words and start doing it, you just need to start gradual and put [your own] examples into it.

Additional CBT supervision is needed. Youth workers reflected on the importance of monthly training sessions and tailored one-on-one coaching, and unanimously reported the need for more CBT-focused supervision to facilitate fluency in applying CBT intentionally with young men. Youth workers noted that their scheduled weekly time for supervision was insufficient to address CBT skills coaching adequately in addition to other programmatic content. As one youth worker stated, “I think we should have [Roca program] supervision alone and CBT [supervision] alone. Because the supervision we have [has]… tons of stuff, it’s too much [to include CBT in the current time allotted for supervision].”

Sustainability of CBT is a concern. Youth workers worried that staff turnover would create drift in the competency and fidelity to CBT. As one youth worker reported,

…There’s so much turnover [in staff] I’m worried that when the older generation who was trained by MGH leaves, there’s not going to be a lot to [keep it going]. It needs to be more institutionalized. I think the idea of a CBT specialist who’s…on Roca’s payroll and that’s at least 50% of their job, I think that’s crucial to CBT remaining part of this [Roca].

Peer-to-peer teaching may improve engagement. Youth workers also recommended having young men who are “experts” in CBT help with teaching. One youth worker explained,

I have three guys on my caseload that do CBT, I asked them to teach me instead of me teaching them, and when they go out of line I pull them back. And then I show them the system we have, like you said, it’s… better than what they used to have.

Young men’s satisfaction with the intervention. CBT helps reduce impulsivity and increases behavioral control. Young men described how CBT helps them to slow down their reactions, especially when their emotions are the most intense. As one young man stated, “I actually think before I do something and end up going back to jail, so it helps me calm down because I know where I’ll end up if I don’t do those skills.”

CBT helps improve problem solving skills. Young men reported that thinking more flexibly has helped them improve their problem-solving skills. As one young man reported,

This [CBT skill] could actually like be helpful, because I didn’t think there was more than a few ways to do it [solve the problem]... but she [the youth worker] helped me come up with multiple ways I could have just handled that.

CBT helps reduce anxiety, anger, and depressive symptoms. Young men reported that CBT helps them address uncomfortable (negative) emotions, and to consider their behavioral responses to these emotions. As one young man noted,

…CBT has definitely taught me how to channel my anger… [and] focus on my responses because usually I am very impulsive... I always… find myself in a way where I have to pump the break so to speak, really get a hold of my thoughts and my actions...also my feelings.

CBT helps reduce alcohol and substance use. Young men described an improved ability to differentiate between when they were using substances recreationally versus when they were using substances to manage their emotions. As one young man reported,

I don’t smoke [marijuana] when I get mad anymore...I like, I walk away, I mean I still do [smoke marijuana], but not as excessively as I used to. I don’t try to be mad and like ‘I need a blunt’...I’ll just take a walk and I’ll go flex my thinking.

CBT helps young men express their emotions. Young men described using CBT skills to talk about painful experiences, and to emotionally process the loss. As one young man described it,

My cousin died and [then] I [overdosed] …. After all of that I started flexing my thinking, like, listen… I am just going to keep going through it [my emotions], he is gone. I can feel it in some type of way but that can’t stop me from moving on with my life. …I actually started coming in here more often to do more CBT after my cousin died. I just started using them [CBT skills] more when I got mad, or when I was feeling depressed, or when I didn’t know how to solve a problem. I’d come here and ask for help.

CBT helps young men take on the perspectives of others. Young men described improvements in empathy, and that they were better able to see the perspectives of others through cognitive reappraisal. As one young man explained,

Realizing everybody doesn’t think the same, or they don’t realize how their words affect you, and sometimes you have to take that into consideration when you’re having conversations with people. Sometimes they don’t realize that the actions that they took, they don’t realize the magnitude they affected you. So sometimes you have to kind of step back and see things from their standpoint, and then you can…come back and try to have that conversation with them.

CBT helps young men in their relationships. Young men described using CBT skills with their friends, partners, and families. As one young man stated,

My daughter, every day I use CBT with her, it’s … not literally sitting there and doing CBT [class] with her, [but] I sit here and go through steps of CBT with her…she does something [effective], [I’m] like, ‘That’s CBT, put that into your book.’ This is how you get a process of being a better father with her.

CBT helps young men plan for their future. Young men talked about using CBT to consider the consequences of their actions while working toward long-term goals. One young man described using the Acting in Line with What You Value skill (value-driven behaviors):

I use it all the time when… friends… they might do things that are not my path, in my ways of doing things… Sometimes they might want to go out to a party and it’s late and I might be like, ‘Okay,’ and immediately I start thinking about I got work in the morning, that might not be a good idea because I’d rather have some money than go out tonight and [one] of my goals is paying my bills and all that, so, CBT teaches me to start thinking about the things I need to do, and [that] works better for me.

Young men like CBT. Young men described positive attitudes toward CBT as a part of their programming at Roca. As one young man described,

I get asked, ‘Did you use CBT this week?’ …I’m pretty sure I did because… [youth worker] forces me to go to CBT. He can be driving me home and it can be after hours and he’s like, ‘Let’s go.’ We’re doing a CBT before you get out of this car.’ No, I’ve done them all. ‘No you haven’t, you’re going to do them again.’ I love it. [My youth worker] makes sure I’m on top of my game.

Young men want more interactive learning. Young men described an interest in having youth workers use more real-life examples, and suggested more opportunities for role play or video clips. As one young man noted,

You connect to their own life, I think a lot of guys are really confused about CBT, [they say,] ‘I don’t get it.’ Maybe the scenarios given to us, other people’s [examples] might not be what are they going through so they might not really let it sink in as much as if you’d say, ‘You are going through this, this is why this is happening and you use CBT to help break that down.’

Peer-to-peer teaching may improve engagement. Young men described how peer-to-peer teaching of CBT might help engage young men who are resistant. As one young man explained,

You need more peer-to-peer relationships, because it’s different hearing it from a [crew] supervisor, whether they are one year older or six years older, it’s just different because they are in a different position as opposed to a participant… If a supervisor told me to do something it’s easy to shrug it off. Now if I tell you something it’s like, ‘Wow.’ I don’t feel like there is an ulterior motive…like you are trying to get me do something.

Discussion

This study aimed to provide detail on the development and implementation of a CBT life-skills program that addresses emotion dysregulation and interpersonal problem-solving skills among young men at high risk for (re)incarceration. Our study provides guidance on the process for tailoring interventions for community-based youth development organizations, and for delivery by paraprofessionals who work at these organizations. Main findings support the feasibility and acceptability of the CBT program, and consumer satisfaction. Our data suggest that young men were satisfied with the intervention, as described applying the skills to employment, educational, and relationship contexts.

This study employed a strong academic-community partnership, blended facilitation (internal and external facilitators), a train-the-trainer model, and gathered staff and consumer feedback on design and packaging of the intervention (“toolkit”). These strategies are labor intensive, yet, we argue that this systematic process is necessary to ensure sustainable interventions. As our results indicate, consultation and supervision were the most frequently noted concerns regarding sustainability of the CBT program. At each iteration of training, we designated the roles of external and internal trainers (shifting from psychologist/researcher-led, to co-led, to organization-led).

Feedback from pilot-testing revealed the need for additional visual aids^29,30 to help youth workers deliver the intervention in both classroom (formal) and outreach settings (informal). The toolkit that was developed is a pared down version of the intervention that is transportable to the settings in which youth workers typically practice (e.g., for outreach). The physical toolkit consists of a laminated notecard for each skill, complete with images on the front side, and the main teaching points for the skill on the back of the card. We observed and youth workers reported that having the key ring helped to engage young men in the material, as this spurred curiosity and provided opportunities for young men to access the materials directly themselves or to select a skill that they wanted to learn and practice in that moment. The visibility of the program created a safe environment for young men to talk about their struggles and created a non-pathologizing entry into CBT content. We posit that toolkits may facilitate engagement.

Implementation challenges.

Supervision support for CBT was integrated into Roca’s existing supervision structure. Capacity for supervision is an area that should be considered by other implementation researchers who aim to implement sustainable programs in social service settings. Although youth workers noted overall satisfaction with supervision they received within the organization, sufficient time for CBT-specific supervision was not available. This finding points to the need for organizational changes to fully support the internal supervision and long-term success of the program—an effort that Roca has since embarked upon. This organization is not unique in having this problem, as shortages in training and supervision capacity are well-documented barriers to implementation and sustainability of effective programs.³¹

Another common challenge facing community-based organizations is staff turnover,³² and our trial was no exception. Staff turnover created challenges with maintaining adequate training and supervision in CBT. Given that previous studies have found that implementation of EBTs can reduce burnout and turnover among providers in professional settings, long-term benefits of the program may be evident over time.^32–34 Already, our findings suggest that the CBT program has made Roca, Inc., more attractive to potential employees who see training in CBT as important to professional development. Our findings also indicate that youth workers are applying CBT skills to their own lives, which may decrease their own difficulties with emotions and interpersonal problems that are inherent in this difficult line of work. Previous studies support that self-practice approaches to training can help to reduce employee burnout and turnover.^35–38

Our ongoing community-research partnership has focused on continuous improvement in the implementation of the CBT program. Findings across all phases of this study have provided data-driven suggestions for future refinements to the implementation strategy. Specifically, we are working to implement additional recommendations (noted by asterisks in Table 3) in the areas of: 1) setting clear expectations regarding the dosage of CBT; 2) committing resources to fully support CBT training, supervision, and coaching; 3) enhancing competency and fidelity to the CBT program through implementation of the CBT Supervision Tool that was created at a later phase of the study; and 4) integrating CBT into the formal job description for youth workers.

Limitations.

There are several limitations to this study. As with all qualitative research, there is the potential for discrepancy between what was said (meant) by the participants and what was heard (interpreted) by the data coders. To address these potential threats to validity, our design included recruiting a reasonably-sized sample (in order to triangulate among multiple sources of data), and employed multiple coders familiar with CBT and implementation science, and a consensual data analytic process (in order to triangulate among multiple data interpretations). Young men who anonymously participated in this study provided rich information on how the program had been received. We chose to keep data collected through focus groups anonymous given the vulnerability of this population, and thus we did not collect any potentially identifying information. More background information on the young men who participated in focus groups might have helped us contextualize some of the findings we present here. We hope to conduct future studies through this partnership where we can more rigorously test the intervention, as well as assess the effect of the CBT program on clinical outcomes. Our sample of young men who completed interviews during the feasibility testing had received at least one CBT “dose,” therefore, these young men may not be representative of young men served by Roca.

Future directions.

Our focus on well-being is a strength and likely one explanation for how well the intervention was received by staff and young men. Clinical and functional outcomes were beyond the scope of the feasibility study, but should be systematically assessed in future studies. We learned that some young men who received CBT showed more willingness to seek professional help. Thus, we wonder if CBT life-skills interventions may be a first step in addressing intrapersonal engagement barriers for this population.³⁹ Future studies may examine the effect of paraprofessional-delivered programs on professional health care utilization. Because we learned that staff engaged in self-practice of CBT skills, future studies may wish to examine the potential effect of CBT implementation on employee burnout.

Conclusion.

Our multi-phase study is novel in many ways. Our CBT program (intervention and implementation strategies) expands the reach of CBT beyond clinical settings, and places access to these interventions within organizations that are already serving hard-to-reach populations. Further, our use of youth workers (paraprofessionals) as interventionists may further reduce the threshold for young men to engage in behavioral health supports. Program design and packaging was novel in many ways, as a result of our efforts to increase the availability and mobility of the program across a wide array of settings. We hope that our approach can be a useful example to researchers aiming to improve access to EBTs for hard-to-reach populations.

Abbreviations:

CBT

Cognitive Behavioral Therapy

EBT

Evidence-Based Treatment

CBPR

Community-Based Participatory Research

IM

Intervention Mapping

ADHD

attention-deficit hyperactivity disorder

PTSD

posttraumatic stress disorder