Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 Nov 1.
Published in final edited form as: J Child Fam Stud. 2019 Jun 14;28(11):3110–3120. doi: 10.1007/s10826-019-01488-4

National Survey of Juvenile Community Supervision Agency Practices and Caregiver Involvement in Behavioral Health Treatment

Angela A Robertson 1, Matthew Hiller 2, Richard Dembo 3, Michael Dennis 4, Christy Scott 5, Brandy F Henry 6, Katherine S Elkington 7
PMCID: PMC6865812  NIHMSID: NIHMS1531953  PMID: 31749598

Abstract

Objectives:

This study sought to expand the sparse literature examining the extent to which family engagement interventions and the structural characteristics of juvenile community supervision agencies influence caregiver participation in youths’ behavioral health (i.e., mental health and substance use) treatment.

Methods:

We analyzed data from a national survey of juvenile community supervision agencies, conducted as a part of a Juvenile Justice Translational Research on Interventions for Adolescents in the Legal System (JJTRIALS) Cooperative Agreement funded by NIH/NIDA.

Results:

Findings indicated agencies employ a variety of family engagement strategies, with passive strategies like services referrals and flexible schedules being more common than active strategies like provision of family therapy. Multivariate prediction of caregiver involvement in behavioral health care showed the most consistent effects for rural-urban location of the agency; rural agencies more successfully engaged families in their youth’s behavioral healthcare. Relatedly, the more family engagement services, the greater the involvement of families in behavioral health treatment. Agencies with a juvenile drug treatment court also showed greater involvement.

Conclusions:

Our findings that juvenile justice agencies are using multiple techniques to engage families, and that there is a relationship between use of these techniques and actual family engagement, would benefit from replication over time and in other jurisdictions. Analysis of data from a second wave of the national survey, recently completed, is expected to test the reliability of our findings over time, as well as identify whether and what kind of changes occurred in the two years following the first survey.

Keywords: family engagement, juvenile drug treatment courts, community supervision

Family and juvenile court professionals are well aware of the importance of the family in promoting positive youth outcomes, as well as for offsetting risky behavior trajectories (Center for Juvenile Justice Reform, 2008; Pennell, Shapiro, & Spigner, 2011). This recognition is clearly reflected in the policies of juvenile justice agencies (e.g., Office of Juvenile Justice and Delinquency Prevention), the educational materials of professional organizations (e.g., the National Council for Family and Juvenile Court Judges), “white papers” (Arya, 2013; Shanahan & diZerega, 2016), and supported by individual empirical studies (e.g., Hodges, Martin, Smith, & Cooper, 2011) and meta-analyses of the empirical literature (Wilson, Olaghere, & Kimbrell, 2016). Missing until now, however, was information on the extent to which, nationally, community supervision agencies used systematic efforts to promote family engagement, the types of family engagement strategies used, and the extent to which they, net other contextual and systems-level factors, predict family involvement in behavioral health treatment (i.e., mental health and substance use health treatment) of their youth.

The recognition that parental involvement is essential for successful probation outcomes spurred early efforts to proactively involve parents and guardians in all stages of juvenile justice processing (Davies & Davidson, 2001). Caregiver engagement practices in the juvenile justice system emphasize educating parents, guardians, and caregivers about the juvenile justice system and empowering them to be involved in decision making regarding their children (Shanahan & diZerega, 2016). Specific juvenile justice reform guidelines emphasize families having access to peer support throughout a youth’s involvement in the system, families being involved in decision-making, and use of culturally competent treatment/approaches (Arya, 2013). Indeed, the Vera Institute of Justice and numerous other organizations, including the Annie E. Casey Foundation, the Council of Juvenile Correctional Administrators, and the MacArthur Foundation’s Models for Change Initiative, among others, have prioritized the promotion of family engagement in the juvenile justice system (Arya, 2013) and called for justice reform that reinforces a family-focused culture through practice, policy, and legislation (diZerega & Verdone, 2011). While the adoption of family engagement policies by a juvenile justice agency is the first step toward increasing family engagement, even an agency which employs family engagement practices may still lack involvement and participation by families. Family engagement practices are what the agency does; whereas, family engagement occurs when the family becomes involved in the process. There are several points in the juvenile justice system in which a family member can become involved, such as attending court or supervision sessions, participating in behavioral health treatment planning, and participating in mental health or substance use treatment sessions. Services and family engagement practices to promote family engagement in the justice system include family therapy, parenting classes, and support groups (Mericle, Belenko, Festinger, Fairfax-Columbo, & McCart, 2014) and these have demonstrated improvement in family outcomes when used as part of juvenile drug treatment court (Carey, Allen, Perkins, & Waller, 2013) and within the broader juvenile justice system (Walker, Bishop, Trayler, et al., 2015). However, the adoption of family engagement practices continues to vary significantly across jurisdictions (Burke, Mulvey, Schubert, & Garbin, 2014). Quantifying this variation in the types of stratgeies that juvenile justice agencies use to increase family engagment, therefore, is a primary focus of the current study.

Current evidence supports including the family in behavioral health services within the juvenile justice system as a critical tool for increasing compliance and improving youth outcomes. For example, an observational study of predominantly male African-American youth drug court participants demonstrated that youth who had a family member attend drug court sessions had better attendance at school and in treatment (Salvatore, Henderson, Hiller, White, & Samuelson, 2010; Salvatore, Hiller, Samuelson, Henderson, & White, 2011). Additionally, a recent meta-analysis found strong evidence that youth family support and involvement enhances drug court success, but family problems can be a barrier to success (Wilson et al., 2016). The study also found that family cohesion, communication, and home functioning can improve during a youth’s participation in drug court (Wilson et al., 2016). Within the broader juvenile justice system, family engagement has been described as being linked to increased instrumental and emotional support for youth, improvements in court functions, and improvements in youth behavior (Walker, Bishop, Pullmann, & Bauer, 2015).

Juvenile Drug Treatment Courts (JDTCs) provide a ready example of engaging families and the potential effect of this on behavioral health outcomes, and thus, represent an agency-level predictor of the extent to which caregivers may be involved in a youth’s behavioral health treatment. JDTC programs have long had a mandate to engage families from both professional (National Association of Drug Court Professionals, 1997) and government organizations (Bureau of Justice Assistance, 2003). One need only to review the 16 Strategies (Bureau of Justice Assistance, 2003) and the new JDTC Guidelines to see that family engagement is an integral part of the JTDC model, as family engagement is one of the overarching principles in the revised guidelines and JDTCs are advised to address specific barriers to family engagement (Office of Juvenile Justice and Delinquency Prevention, 2016). In their JDTC guidelines operationalizing family engagement, the Office of Juvenile Justice and Delinquency Prevention recommends that families be involved throughout the drug court process. The guidelines also advocate for training in how to effectively engage families. During needs assessment and service provision, it is also suggested that parental needs be assessed and addressed. Finally, the guidelines encourage JDTC programs to routinely collect data on family related factors (Office of Juvenile Justice and Delinquency Prevention, 2016).

Empirical evidence of the need for integrating family engagement and family therapy specifically into the JTDC process was provided by a randomized-clinical trial that examined traditional family court and a non-enhanced juvenile drug court with two conditions that added an evidence-based family therapy, Multisystemic Therapy (MST; in which as the name suggests the provider assists the family in working effectively with other systems, such as school, to improve youth outcomes ), and/or Contingency Management (CM; an evidence-based apporach that provides external rewards/reinforcement for treatment attendence or diminished substance use) to juvenile drug courts. Findings indicated that, relative to traditional family court, juvenile drug courts reduced in-program delinquency and drug use. However, addition of MST resulted in even greater reduction of delinquency and drug use. CM was associated with a small additional decrease in these outcomes, when combined with MST (Henggeler et al., 2006). Studies of MST in the broader juvenile justice system have also demonstrated success in reducing re-arrest and improving mental health symptoms (Timmons-Mitchell, Bender, Kishna, & Mitchell, 2006). Additional research has demonstrated that JDTCs that employed family engagement practices in conjunction with contingency management were significantly more effective than courts using only contingency management (Henggeler & Sheidow, 2012). However, despite these promising findings and policy or guidelines around family engagement practices, only 72% of juvenile drug court staff reported utilizing referrals of family members for substance use or mental health treatment, 61% reported the use of incentives (part of contingency management), and 41% reported the use of support groups (Harris, Keator, Vincent-Roller, & Keefer, 2017).

Staff practices around family engagement are driven not only by staff attiudes, values and skill, but also by the agency or setting in which they work/practice (Stiffman, Pescosolido, & Cabassa, 2004). Understanading agency-level characteristics is thereofore also necessary when trying to predict caregiver engagement in behavioral health treatment. For example, staff training can provide staff with the skills necessary to successfully engage families in a culturally-competent manner (Gatowski, Miller, Rubin, Thorne, & Barnes, 2016). Additionally, training can enhance staff skills in other areas which also promote family engagement, such as case management (Wilson et al., 2016) and motivational interviewing, an evidence-based approach that seeks to help youth understand discrepancies in how they describe their behavior and its objective adverse impact on their lives (Gatowski et al., 2016). Research also suggests that courts which provided training on the multidimensionality of youth and family problems had more confident staff (Linden, Cohen, Cohen, Bader, & Magnani, 2010). However, the belief that it was the role of the JDTC to train staff in strategies to engage families was not universal (86.0%) among recently surveyed drug court staff (Harris et al., 2017). A less-explored agency characteristic related to family engagement is minority caseload. Ingoldsby (2010) noted that ethnic minority status was associated with less family engagement. Although the reasons for this finding are not fully understood, it is possible that cultural (e.g., distrust of a justice system perceived as being prejudiced or racist) or practical barriers may contribute to less family engagement with the juvenile justice system for ethnic minority families. Staff training may be able to address such barriers to improve family engagement among ethnic minority families.

Additionally, it is important to consider context in which JTDCs exist when attempting to understand staff practices around family engagement. The relationship between rural-urban residence and family engagement has not been closely examined. Although differences have been noted between rural and urban juvenile justice administration for decades (DeJames, 1980; Feld, 1991), little research exists that specifies the impact of geography and population density on family engagement. Differences between rural and urban juvenile justice administration may be related to agency procedures (Feld, 1991). For example, in a study of Minnesota juvenile courts, Feld (1991) concluded that urban juvenile courts followed more formal procedures than rural juvenile courts, possibly due to caseload size. One extrapolation of these findings is that urban juvenile courts may have formal family engagement procedures, and be more likely to have higher family involvement as a result. However, the opposite is also possible: more formality in urban juvenile courts may mean that families are less involved in the process, and family engagement may be lower as a result. Differences between rural and urban jurisdictions may exist independently of agency characteristics, though juvenile justice agencies may be able to address some barriers that limit family engagement once identified. DeJames (1980) described characteristics of rural areas that impact juvenile justice, including limited access to social services, public transportation, and alternative schools. Limited access to neighborhood resources has been identified as a significant barrier to family engagement in the juvenile justice context (Ingoldsby, 2010).

Using data from a nationally representative sample of community supervision agencies, the current paper first presents needed data on the extent to which community supervision agencies actively seek to engage families, as well as the types of strategies used. Next, it explores both agency level characteristics (e.g., having an implemented JDTC, staff training needs) and structural characteristics (i.e., rural-urban location of the jurisdiction) and their association with the extent to which caregivers are involved in the behavioral health treatment of their youth. More specifically, based on the literature reviewed, we hypothesized that agencies with JDTCs, that participated in system-level reform efforts, and that report use of a greater number of family engagement practices, will have higher rates of caregiver involvement in three activities related to behavioral health services, including service need identification, treatment planning, and treatment services.

Method

Procedure

A survey of juvenile justice community supervision (CS) agencies was conducted as part of the JJ-TRIALS cooperative agreement funded by the National Institute on Drug Abuse. The purpose of this survey was to document how these agencies currently address substance use, mental illness, and risk for HIV/STDs for youth who are under community supervision. A nationally representative sample of 20 states was selected for the study based on the number of adolescents between ages 10 to 19 reported in the 2012 Current Population Survey (CPS) (United States Census Bureau, 2012). The five largest states were sampled with certainty, and the other 15 with probabilities proportionate to the number of youths in those states. Within each state, a representative sample of 8 to 14 counties was selected based on the number of youth, again with the largest one or two counties sampled with certainty, and the remaining counties sampled with probabilities proportionate to the number of youths in those counties. Both states and counties were stratified to ensure the proportionate representation of smaller states and counties. With each of the sampled counties, researchers worked with the OJJDP state juvenile justice contact to identify all of the community supervision agencies. Of the 192 sampled counties, 182 had one CS agency, 9 had two agencies, and 1 had three agencies.

The breadth of the survey often required input from multiple agency staff who had access to different information. Given the variation across juvenile justice agencies both within and between states, it could not be predetermined exactly which staff would be best to answer each set of questions. To assist in identifying the appropriate staff to help complete the survey, each state was assigned a survey coach who facilitated a survey overview call with the agency key stakeholder and provided an overview of the survey components. During this meeting, they discussed who at the agency would be best to respond to each set of questions and what data sources were available.

Juvenile Justice Community Supervision Surveys were attempted in all 203 community supervision agencies identified in the 192 counties sampled. A total of 195 surveys (96%) were completed and returned. Data were weighted based on the inverse of the inclusion probability and were adjusted for nonresponse within state. The number of agencies overall and those providing a specific service were estimated by multiplying the weighted average number of agencies per county times the actual number of counties (n=3,143) in the United States. This generated a national estimate of 3,509 CS agencies serving 770,323 youth under community supervision.

Measures

The Juvenile Justice Community Supervision survey covered agency characteristics, youth characteristics, screening, clinical assessment, and referral practices, as well as substance use prevention, substance use treatment, HIV/STI risk prevention, mental health treatment, family engagement, and training needs of staff involved with youth under community supervision. For this study, we focused on agency characteristics, strategies, or practices used by the agency to increase family engagement, and CS agency estimates of caregiver participation in activities related to identifying youth service needs, choosing the type of treatment or level of care, and formal treatment sessions.

Caregiver involvement in behavioral health services.

Survey respondents were asked to estimate caregiver participation (i.e., family member, parent, caregiver, and/or guardian) during the past year in 13 activities related to behavioral health screening and assessment, CS staff management of the youth’s case, including determining services needs and setting incentives and consequences for compliance and noncompliance with the treatment/service plan, and in formal treatment sessions. We limit our analyses to three behavioral health services activities: a.) a formalized treatment staffing or planning meeting to decide what services are needed and set goals; b.) choosing the type of treatment or level of care; and c.) participating in formal treatment sessions. Response options were the percent of youth under community supervision with family involvement, i.e., 0% (coded 0), 1-25% (coded 1), 26-50% (coded 2), 51-75% (coded 3), 76-100% (coded 4), and don’t know/information not available (coded as missing).

County Rural-Urban Continuum Codes.

The counties in which CS agencies were located were classified based on the 2013 Rural-Urban Continuum Codes that distinguishes metropolitan counties by the population size of their metro area, and non-metropolitan counties by degree of urbanization and adjacency to a metro area. Each county in the United States is assigned one of nine codes ranging from 1 (metro – counties in metro areas of 1 million pop or more) to 9 (completely rural or less than 2,500 urban pop, not adjacent to a metro area). We collapsed these codes into three groups: urban (codes 1-3=1), adjacent urban (codes 4, 6, and 8=2) and rural (codes 5, 7, and 9=3).

Specialty Court Programs.

Survey respondents were asked if their agency participated in any specialty court programs during the past year. The majority (68.2%) of CS agencies did not participate in any specialty court programs, 23.6% reported one specialty court program and the remainder reported that their jurisdiction had two or more specialty court programs. Almost two percent (1.9%) reported a mental health court program, 3.9% a teen court program, 4.7% a peer court program, 5.6% a family drug treatment court program, 12.1% a juvenile drug treatment court (JDTC) program, and 16% reported an unspecified specialty court targeting youth. For this study, we only included JDTC (no=0, yes=1) in our analyses. The rationale for this decision is that evidence-based guidelines for JDTCs were recently developed through a specific effort of OJJDP (2016) to improve the potential impact of these programs. This paper serves as a baseline for JDTCs prior to this significant change in the model against which comparison of a second wave of survey data may be compared to infer changes related to use of family and family involvement services.

System Level Reform Efforts.

Survey respondents were asked whether their agency participated in juvenile justice reform efforts during the past year. The majority (67.3%) of CS agencies did not participate in any system-level reform. Participation in the following juvenile system reform grant programs was very low for the MacArthur Foundation’s Model for Change (1.8%) and the Robert Wood Johnson Foundation’s Reclaiming Futures (6.3%). Participation in the Annie E. Casey Foundation’s Juvenile Detention Alternatives Initiative (JDAI, 18.8%) and other juvenile justice system reform grant (19.5%) was more widespread, but still a relatively small percentage of CS agencies. The reform efforts variable was therefore coded as any (1) versus none (0).

Family System Engagement Index (FSEI).

Survey respondents were asked to indicate which of 12 strategies or practices that their agency provided to help increase family member, parent, caregiver, and/or guardian engagement in behavioral health services for youth. Respondents were asked to reply no (coded 0) or yes (coded 1) to each of the Family System Engagement Index items. The practices included: adapted written policies to encourage family engagement; invited family representative to serve on advisory boards; provided family support groups; provided family member (not youth) education groups; provided family therapy (with youth and family); referred to family therapy (with youth and family); provided family behavioral, contingency management or other parenting skills programs; referred to family behavioral, contingency management or other parenting skills programs; provided flexible scheduling to accommodate families; assisted with transportation; assisted with childcare; and addressed the cultural, linguistic, and sexual orientation of families. An index was created by adding the number of checked (coded 1) family engagement practices. Scores ranged from 0 to 10 with higher scores indicating that the CS agency employed more strategies. The internal consistency reliability of the index is good (Cronbach’s alpha = 0.795).

CS Staff Need for Family Engagement Training.

The survey included a section on staff training needs. The respondent for the agency was asked to indicate agreement with the statement “In this county, community supervision staff (e.g., probation, parole) involved with youth under community supervision need additional experience and/or training related to increasing family engagement.” Response options were “strongly agree” (coded 5), “agree” (coded 4), “mixed or unsure” (coded 3), “disagree” (coded 2) and “strongly disagree” (coded 1). Thus, higher scores on this item indicates greater agreement that staff need additional training.

Percentage Minority Youth on Agency Caseload.

Survey respondents were asked to report the percent of youth under community supervision in seven racial/ethnic categories. The percentage of agencies that could not provide this information ranged from a low of 13.6% for White/Caucasian youth to a high of 31.5% for other race. Among the agencies that provided data, 78.1% of youth on the caseload were White/Caucasian, 11.9% were Black/African American, 7.6% were Hispanic, 0.6% were Asian/Hawaiian/Pacific Islander, 2.0% were Native American, 1.1% were categories as other, and 1.3% were mixed or multiple race. Given research indicating that African American and Hispanic youth are disproportionately involved in the juvenile justice system (National Council on Crime and Delinquency, 2007; Rodriguez, 2013) yet minority families have lower rates of engagement in clinical services (Nock & Ferriter, 2005; Snell-Johns, Mendez, & Smith, 2004), we decided to focus on the percentage of African American and Hispanic youth on the agency’s caseload. We created a variable to indicate CS agencies with 10% or higher rates of Black/African American and Hispanic youth (coded 1, otherwise coded 0).

Data Analyses

As noted above, the response options for the three dependent variables were five ordinal categories ranging from zero percent to 76-100% of youth under community supervision with caregiver involvement. Therefore, ordinal logistic regression analyses were completed using Mplus Version 8.0 (Muthén & Muthén, 2017). MLR estimation (maximum likelihood parameter estimates with standard errors and a chi-square statistic that are robust to non-normality and non-independence of observations) was used. The Mplus Direct ML feature for estimation of missing values was used to treat any missing data (Enders, 2010; Muthén & Muthén, 2017).

Missing Data.

In regard to missing data, all independent variables but one had no missing data. That variable, Family System Engagement index, had only 9 cases with missing information (< .002% of weighted cases). On the other hand, each of the three dependent variables had missing information: (1) parent participation in formalized treatment staffing or planning meeting to decide what services are needed and to set goals (n=493, 14% of weighted cases), (2) parent participation in choosing the type of treatment or level of care (n=789, 22% of weighted cases), and (3) parent participation in formal treatment sessions (n=947, 27% of weighted cases). Hence, additional examination was pursued to determine if valid or missing data on the three dependent measures was related to the various predictor variables (Enders, 2010). The magnitude of each of these correlations was low, with the largest, agency participating in juvenile justice reform efforts and parent participation in formalized treatment staffing or planning meeting (r=−.240), accounting for <6% of variance. Each of the other relationships accounted for <3% of the variance. These results allayed concerns of possible systematic bias in these data.

Results

Findings presented in Table 1 show the types of family engagement strategies used by CS agencies, as well as the sizable variation in the use of individual practices across agencies. Making referrals to services was the most common family engagement strategy. For example, nearly 70% of agencies reported they made referrals to family therapy, and 79% made referrals to parenting, family behavioral management, and contingency management skills development programs. Some supervision agencies also provided family therapy (17.8%), and family behavioral management and parenting skills classes (23.3%). Structurally, agencies also tried to foster family involvement by adopting formal policies to promote this (35.7%), and 63.5% of agencies indicated flexible scheduling to accommodate families. Cultural/linguistic /sexual orientation support (37%), transportation assistance (48.5%), childcare (11.2%) and inviting families to serve on advisory boards (15.9%) also were strategies used for fostering family engagement.

Table 1:

Use of Family Engagement Strategies by Community Supervision Agencies

Family System Engagement Index items %
Adapted written policies to encourage family engagement 35.7
Invite family representatives to serve on advisory boards 15.9
Provide family support groups 6.8
Provide family member (not youth) education groups 3.7
Provide family therapy (with youth and family) 17.8
Refer to family therapy (with youth and family) 69.6
Provide family behavioral, contingency management or other parenting skills programs 23.3
Refer to family behavioral, contingency management or other parenting skills programs 79.0
Provide flexible scheduling to accommodate families 63.5
Assist with transportation 48.8
Assist with childcare 11.2
Address the cultural, linguistic, and sexual orientation of families 37.0
M(SD)
Total FSEI score (range 0 – 10) 4.12(2.70)
Open in a new tab

Table 2 presents descriptive statistics for caregiver participation in behavioral health decisions and services for their youth. The most widespread participation was found for caregiver involvement in a formalized treatment staffing or planning meeting to decide what services and goals were needed, with 69% of agencies estimating caregiver involvement in this activity for 76% or more of their cases. Choosing the level of care or type of treatment needed was reported less frequently by agencies. Here, about 42% of agencies indicated caregivers were involved in this for 76% or more of cases. Finally, the least family involvement was noted for participation in formal treatment sessions. About 25% of agencies indicated families were involved in treatment sessions for 76% or more of their youth. Community supervision agencies also differed on key predictors used in subsequent multivariate modelling. That is, only 12% reported having a juvenile treatment court, 32% were involved in justice reform efforts, about one-third were in rural areas, and most (86%) reported that minorities made up less than 10% of their cases.

Table 2:

Summary Statistics of Community Supervision Agency Characteristics and Caregiver Involvement

CS Agency Characteristics, n=3509 % M(SD)
Juvenile Drug Treatment Court (JDTC) program 12.1
Participation in any juvenile justice reform efforts 32.7
CS Staff Need for Family Engagement experience/training 3.46(1.19)
 Strongly disagree (1) 0.7
 Disagree (2) 33.7
 Mixed/unsure (3) 6.2
 Agree (4) 37.0
 Strongly Agree (5) 22.3
Minority Caseload
 10% or greater 13.5
 Less than 10% 86.5
Rural-Urban Codes
 Urban 38.7
 Adjacent Urban 24.4
 Rural 36.9
Estimates of Caregiver Participation in
A formalized treatment staffing or planning meeting to decide what services are needed and set goals
 0% 1.2
 1-25% 5.0
 26-50% 7.7
 51-75% 2.8
 76-100% 69.2
 Missing 14.0
Choosing the type of treatment or level of care
 0% 6.9
 1-25% 10.5
 26-50% 9.5
 51-75% 18.1
 76-100% 32.5
 Missing 22.5
Formal treatment sessions
 0% 1.2
 1-25% 7.0
 26-50% 17.2
 51-75% 23.1
 76-100% 24.6
 Missing 27.0
Open in a new tab

As shown in Table 3, when examining the system and contextual variables and their relationship with caregiver involvement in treatment services, we found a strong association between the number of family engagement strategies that a jurisdiction had and the outcome variables. More specifically, as hypothesized, greater values on the FSEI index were associated with significantly higher proportions of youth with caregivers involved in choosing the level of care (Model 2) and being involved in their youths’ treatment sessions (Model 3).

Table 3.

Ordinal Logistic Regression of Agency Level Predictors of Caregiver Involvement in Youth Treatment – Unstandardized Coefficients – MLR Estimation

Model 1: Treatment Staffing/Planning Model 2: Choosing Treatment Type or Level of Care Model 3: Fonnal Treatment Sessions
Estimate S.E. Est./S.E. Estimate S.E. Est./S.E. Estimate S.E. Est./S.E.
JDTC 0.457 0.668 0.683 1.238 1.135 1.091 1.932 0.755 2.561**
Reform Efforts 1.321 0.546 2.421* 0.222 0.903 0.246 0.694 0.635 1.093
Training/experience 0.088 0.233 0.379 −1.033 0.384 −2.686** −0.522 0.312 −1.674+
Minority −0.101 0.951 −0.107 0.172 0.588 0.293 −1.902 0.952 −1.998*
FSEI 0.089 0.158 0.560 0.558 0.157 3.560*** 0.361 0.151 2.386*
Rural-Urban 2.145 0.558 3.844*** 0.949 0.279 3.402*** 1.176 0.350 3.357***
Dependent Variable Thresholds
 1 −0.261 0.760 −0.343 −2.126 1.146 −1.855+ −2.915 1.291 −2.257*
 2 1.461 0.562 2.598** −0.819 0.980 −0.835 −0.659 0.862 −0.764
 3 2.646 1.223 2.162* 0.070 0.788 0.089 1.319 1.110 1.189
 4 3.008 1.241 2.425* 1.564 1.006 1.555 3.408 1.528 2.230*
Open in a new tab

Two-tailed p-values:

+

.10>p>.05;

*

p<.05;

**

p<.01;

***

p<.001.

Variables coding are discussed in the narrative.

The strongest support for our hypothesis that jurisdictions with JDTCs would show greater involvement of caregivers was found for formal treatment sessions (Model 3), with jurisdictions that had a JDTC showing a significantly larger percentage of youth whose caregivers were involved in their treatment compared to those without JTDCs. There was no significant relationship between JDTC and the proportion of youth for whom a caregiver was involved in treatment/staff planning or the selection of the level of care for the youth.

Finally, a jurisdiction being involved in a reform effort with their juvenile justice system was only related to a higher proportion of youth having caregivers involved in their treatment planning (Model 1). Contrary to hypothesis, associations with choosing treatment type or level of care and involvement in formal treatment sessions were not found.

Other predictors examined were variably associated with caregiver involvement in youth treatment, with some having a more consistent relationship; whereas, other predictors were important in some models, but not others. For example, the degree to which a jurisdiction was situated in terms of rural to urban was significantly related to all three dependent variables. That is, being a more rural jurisdiction was associated with a significant increase in the proportion of youth for whom a caregiver was involved in their treatment planning, selection of their level of care, and participation in treatment with their youth. Jurisdictions where staff expressed a greater need for training for working with families saw significantly lower proportions of youth for whom a caregiver was involved in level of care selection. This trend, although not statistically significant, also was observed for the proportion of youth with care givers attending treatment. Similarly, jurisdictions with higher proportions of minority clients also saw proportionally lower rates of caregiver attendance in treatment.

Discussion

Using data from a nationally-representative survey of juvenile justice community supervision (CS) agencies in the United States, this is the first paper, to our knowledge, that describes the types of strategies used by CS agencies to actively engage parents, guardians, or caregivers, as well as the degree to which use of family engagement strategies along with agency and structural-level characteristics influence a caregiver’s involvement in their child’s BH treatment. Family involvement has been consistently shown to promote positive treatment outcomes, including retention (Alarid, Montemayor, & Dannhaus, 2012; Diamond & Josephson, 2005; Fradella, Fischer, Kleinpeter, & Koob, 2009; Liddle, 2004; McKay, Nudelman, McCadam, & Gonzales, 1996b; Prado, Pantin, Schwartz, Lupei, & Szapocznik, 2005; Staudt, 2007; Szapocznik et al., 1988) and research addressing the prevention of problem behaviors and recidivism in juvenile justice-involved youth highlights the importance of family involvement (Henggeler & Sheidow, 2012). Caregivers have a strong influence on youth behavior and can be key collaborators with probation departments in achieving youth participation in treatment and in turn, reversing youth’s trajectories of offending.

On average, agencies described engaging in approximately 4 (out of 10) different family engagement strategies, of which the most commonly endorsed were passive referral strategies (to family therapy or behavioral parenting program) and providing a flexible schedule to accommodate caregivers. Unfortunately, we are not able to quantify the degree to which these engagement strategies were successfully implemented (e.g., referrals were accepted). Moreover, what is less understood from the current study and requires future examination is the working alliance between the probation officer (PO) and family, and how that may influence the caregiver acceptance of a referral to promote their engagement in their child’s behavioral health treatment. The working alliance is comprised of agreement about tasks, agreement about goals and the bond between provider and client (Bordin, 1979; Horvath & Greenberg, 1994). Given a strong working alliance, we would hypothesize that family engagement strategies would be readily accepted. However, the working alliance and its relationship to service uptake as opposed to treatment outcome has been almost entirely overlooked.

Consistent with our hypothesis, multivariate models showed agencies who employed more family engagement strategies had significantly greater proportions of youth under supervision who had caregivers (i.e., family member, parent, caregiver, and/or guardian) involved in developing choosing the treatment type or attending formal treatment sessions. These findings are consistent with prior work that documents the importance of strategically embedded engagement-focused family interventions that reduce barriers to behavioral health care access and improve treatment outcomes across a range of sites serving youth (McKay, Nudelman, McCadam, & Gonzales, 1996a; McKay et al., 1996b). However, engagement interventions have typically been delivered in clinical settings in which provider capacity to increase engagement is addressed. The current study provides preliminary evidence to suggest that engagement interventions that train probation officers in engagement strategies may have utility in increasing caregiver engagement injustice settings (Schwalbe & Maschi, 2010).

Somewhat counter to our hypotheses, we found that agencies with JDTCs, for which family engagement is a considerable mandate, and those that participated in system-level reform efforts were variably associated with caregiver engagement. Agencies with a JDTC had a greater proportion of caregivers who were involved in formal treatment sessions but not in treatment planning and choice-related activities. This may be explained, in part, by the way juvenile drug courts organize themselves. That is, the day-to-day operations of these programs are overseen by a stakeholder team that included collaborators across both justice and community agencies, including treatment services providers. Team meetings prior to the youth appearing before the juvenile drug court judge (i.e., pre-court staffing meetings, see Salvatore et al., 2011) are used to share information (e.g., performance on supervision, treatment, and other activities) about youth since their most recent meeting with the judge. Perhaps, in jurisdictions with JDTCs, the closer working relationship between justice and community providers may have the unintended effect of limiting caregiver involvement in these types of treatment related decisions. That is, preference for providers with staff on the stakeholder team might reduce the likelihood that caregivers are brought into this decision-making process at this time, because treatment providers plan to involve them after the youth has been referred to their programs.

Agencies that were involved in any system reform efforts had a greater proportion of caregivers involved in treatment and staff planning, but nothing else. This may be the result of organizations (e.g., OJJDP, MacArthur Foundation, Annie E. Casey Foundation) involved in system reform, which promote practice guidelines that emphasize parental involvement in treatment planning (Arya, 2013); greater caregiver involvement, consistent with these guidelines should be expected.

Examining agency and contextual predictors in multivariate models, we observed a negative relationship between caregiver involvement in the two of the behavioral health services activities and ratings of the need for staff training on family engagement. This finding, perhaps as one might expect, indicates that agencies with higher levels of family engagement in treatment are more likely to consider their staff to be sufficiently trained in family engagement.

Interestingly, the most consistent finding related to caregiver involvement in youth behavioral health treatment was the degree of urbanicity of jurisdiction of the agency. Net of all other predictors, agencies that were located in a rural jurisdiction had proportionately greater numbers of caregivers involved in treatment planning, choice of treatment type and participation in formal treatment sessions. Although we were unable to test the causal relationships between urbanicity, percent of minority caseload and our Family System Engagement Index item of “address the cultural, linguistic, and sexual orientation of families,” it is likely that the relationships between these variables help explain our findings. For example, in urban areas Black and Hispanic people often make up a larger percentage of the population. Given our finding that having a high percent of minority caseload was related to lower family engagement, it is possible that the demographics of the jurisdiction may be part of what is driving the relationship between urbanicity and family engagement. Additionally, if these jurisdictions also have low levels of cultural competency (i.e. an ability to “address the cultural, linguistic, and sexual orientation of families”) then it would make sense that family engagement is low. Given that our survey only measures one staff member’s perception of cultural competency, and not actual practices, we are limited in our ability to appropriately examine these relationships. However, future research which directly measures these factors should examine this area.

Limitations and Future Directions

A main limitation of the current study is that our data was collected through agency-completed surveys. Relatedly, the surveys were not completed by the same type of agency staff member (e.g., chief administrator, specific program director, etc.). Hence, it is likely different types and durations of experiences are reflected in the respondents’ replies to the survey questions. These limitations could affect the interpretation of our findings in several ways. First, agencies do not collect detailed information regarding family engagement practices, resulting in a significant portion of missing data; thus, we opted to provide data and analyses that were more descriptive in nature. Second, given that the agencies completed the surveys, the researchers cannot confirm how those surveys were completed (e.g., whether the survey responses were based on systematic data). Third, the data were drawn from a nationally representative sample of 20 states and 198 counties, involving juvenile justice community supervision agencies, juvenile judges, and behavioral health providers, to describe characteristics and needs of youth on community supervision; what behavioral health screening, assessment and treatment are provided to these youths; and collaborative relationships among community supervision and behavioral health care agencies and judges. The purposes of this survey precluded the collection of important youth outcome information in the areas of recidivism, school attendance and performance, mental health, and substance use, as well as data on the relationship of our measure of family system engagement to youth outcomes. Future studies should, if possible, include youth outcome and family engagement data to more fully understand the impact of family engagement practices on youth behavior.

Another important limitation of our study is that our family system engagement index is a count of the number of strategies or practices used by agencies to increase caregiver involvement, rather than a measure with weighted items indicating prediction strength of each item or group of items. For example, assisting with transportation may outweigh referring to family therapy as a strategy to promote family engagement within the decision-making process. Additional research is needed to determine whether certain strategies are more important than others for successful family engagement in juvenile justice programs. Another limitation of our family engagement index is that we were constrained by our data to use a relatively narrow definition of family engagement where we focused only on caregiver involvement and did not include other family members. Given existing evidence that suggests how including a broad array of family can improve outcomes, this is an important limitation. The exclusion of other family members in our measure could impact our findings in several ways. First, we may be undercounting the efforts of juvenile justice agencies which are succeeding at engaging non-caregiver family members but underperforming at engaging caregivers. Second, we may be missing an important contextual element of how juvenile justice agency policies serve as a barrier to non-caregiver engagement. Although we were able to include some measures related to agency reform efforts, data on specific policies relating to non-caregiver engagement were not available. Additional research should strive to use a more inclusive definition of family engagement and directly examine the role that policies play as a barrier or facilitator to non-caregiver engagement.

In conclusion, findings from the current study suggest that juvenile justice agencies are using multiple techniques to engage families, and that there is a relationship between use of these techniques and actual family engagement. The extent to which our findings hold true over time, especially following major changes in administrations and policies, is unclear. Importantly, a second wave of the survey has been mounted, capturing the same information at least two years after the first wave of the survey. Analyses of these data are pending, and it is important to replicate the current study to test the reliability of our findings over time, as well as to examine whether and what kind of changes (and if possible, what precipitated them) occurred in the time that elapsed since the first survey.

Acknowledgments

This study was funded under the JJ-TRIALS cooperative agreement, funded at the National Institute on Drug Abuse (NIDA) by the National Institutes of Health (NIH). The authors gratefully acknowledge the collaborative contributions of NIDA and support from the following grant awards: Chestnut Health Systems (U01DA03622), Columbia University (U01DA036226), Emory University (U01DA036233), Mississippi State University (U01DA036176), Temple University (U01DA036225), Texas Christian University (U01DA036224), University of Kentucky (U01DA036158). The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the NIDA, NIH, or the participating universities or juvenile justice systems.

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

Glossary:

JDTC

Has a juvenile drug treatment court program

Reform Efforts

Any agency juvenile justice reform efforts

Training/experience

CS staff needs training/additional experience in family engagement practices

Minority

Agency has 10% or greater Hispanic and Black youth caseload

FSEI

Family System Engagement Index

Rural-Urban

metro (1), adjacent to metro (2), or rural, not adjacent to metro area (3) county

Footnotes

Publisher's Disclaimer: This Author Accepted Manuscript is a PDF file of an unedited peer-reviewed manuscript that has been accepted for publication but has not been copyedited or corrected. The official version of record that is published in the journal is kept up to date and so may therefore differ from this version.

*

No conflict exists for any of the authors.

Contributor Information

Angela A. Robertson, Social Science Research Center, Mississippi State University, 1 Research Blvd., Suite 103, Starkville, MS 39759

Matthew Hiller, Department of Criminal Justice, Temple University, Philadelphia, PA.

Richard Dembo, Department of Criminology, University of South Florida, Tampa, FL.

Michael Dennis, Chestnut Health Systems, Normal, IL.

Christy Scott, Chestnut Health Systems, Normal, IL.

Brandy F. Henry, The Heller School for Social Policy and Management, Brandeis University, Waltham, MA

Katherine S. Elkington, Columbia University and New York State Psychiatric Institute, New York, NY

References

  1. Alarid LF, Montemayor CD, & Dannhaus S (2012). The effect of parental support on juvenile drug court completion and postprogram recidivism. Youth Violence and Juvenile Justice, 10(4), 354–369. doi: 10.1177/1541204012438422 [DOI] [Google Scholar]
  2. Arya N (2013). Family comes first: A workbook to transform the justice system by partnering with families. Washington, D.C. [Google Scholar]
  3. Bordin ES (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research & Practice, 16(3), 252–260. doi: 10.1037/h0085885 [DOI] [Google Scholar]
  4. Bureau of Justice Assistance. (2003). Juvenile drug courts: Strategies in practice monograph. Washington, D.C.: U.S. Department of Justice, Office of Justice Programs. [Google Scholar]
  5. Burke JD, Mulvey EP, Schubert CA, & Garbin SR (2014). The challenge and opportunity of parental involvement in juvenile justice services. Children and Youth Services Review, 39, 39–47. doi: 10.1016/j.childyouth.2014.01.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Carey SM, Allen TH, Perkins T, & Waller MS (2013). A detailed cost evaluation of a juvenile drug court that follows the juvenile drug court model (16 strategies). Juvenile and Family Court Journal, 64(4), 1–20. doi: 10.1111/jfcj.12009 [DOI] [Google Scholar]
  7. Center for Juvenile Justice Reform. (2008). Juvenile justice professionals certificate program survey. Washington, D.C.: Georgetown University. [Google Scholar]
  8. Davies HJ, & Davidson HA (2001). Parental involvement practices of juvenile courts: Report to the office of juvenile justice and delinquency prevention. Washington, D.C.: Center on Children and the Law. [Google Scholar]
  9. DeJames J (1980). Issues in rural juvenile justice In Jankovic J, Green RK & Cronk SD (Eds.), Juvenile Justice in Rural America. Washington, D.C. [Google Scholar]
  10. Diamond G, & Josephson A (2005). Family-based treatment research: A 10-year update. Journal of the American Academy of Child & Adolescent Psychiatry, 44(9), 872–887. doi: 10.1097/01.chi.0000169010.96783.4e [DOI] [PubMed] [Google Scholar]
  11. diZerega M, & Verdone J (2011). Setting an agenda for family-focused justice reform. New York, NY: Vera Institute of Justice. [Google Scholar]
  12. Enders CK (2010). Applied missing data analysis. New York, NY: Guilford Press. [Google Scholar]
  13. Feld BC (1991). Justice by geography: Urban, suburban, and rural variations in juvenile justice administration. Journal of Criminal Law and Criminology, 82, 156. [Google Scholar]
  14. Fradella HF, Fischer RG, Kleinpeter CH, & Koob JJ (2009). Latino youth in the juvenile drug court of Orange County, California. Journal of Ethnicity in Criminal Justice, 7(4), 271–292. doi: 10.1080/15377930903382142 [DOI] [Google Scholar]
  15. Gatowski S, Miller NB, Rubin SM, Thorne W, & Barnes EW (2016). OJJDP juvenile drug treatment court guidelines project: Juvenile drug treatment court listening sessions. National Criminal Justice Reference Service; Retrieved from https://www.ncjrs.gov/pdffiles1/ojjdp/grants/250491.pdf. [Google Scholar]
  16. Harris B, Keator K, Vincent-Roller N, & Keefer B (2017). Engage, involve, empower: Family engagement in juvenile drug treatment courts. National Center for Mental Health and Juvenile Justice; Retrieved from https://www.ncmhjj.com/wp-content/uploads/2017/02/Engage-Involve-Empower-Family-Engagement-in-Juvenile-Drug-Treatment-Courts.pdf. [Google Scholar]
  17. Henggeler SW, Halliday-Boykins CA, Cunningham PB, Randall J, Shapiro SB, & Chapman JE (2006). Juvenile drug court: Enhancing outcomes by integrating evidence-based treatments. Journal of Consulting and Clinical Psychology, 74(1), 42–54. doi: 10.1037/0022-006x.74.1.42 [DOI] [PubMed] [Google Scholar]
  18. Henggeler SW, & Sheidow AJ (2012). Empirically supported family-based treatments for conduct disorder and delinquency in adolescents. Journal of Marital and Family Therapy, 38(1), 30–58. doi: 10.1111/j.1752-0606.2011.00244.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hodges K, Martin LA, Smith C, & Cooper S (2011). Recidivism, costs, and psychosocial outcomes for a post-arrest juvenile diversion program. Journal of Offender Rehabilitation, 50(7), 447–465. [Google Scholar]
  20. Horvath AO, & Greenberg LS (1994). The working alliance: Theory, research and practice. New York, NY: Wiley. [Google Scholar]
  21. Ingoldsby EM (2010). Review of interventions to improve family engagement and retention in parent and child mental health programs. Journal of Child and Family Studies, 19(5), 629–645. doi: 10.1007/s10826-009-9350-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Liddle HA (2004). Family-based therapies for adolescent alcohol and drug use: Research contributions and future research needs. Addiction, 99, 76–92. doi: 10.1111/j.1360-0443.2004.00856.x [DOI] [PubMed] [Google Scholar]
  23. Linden P, Cohen S, Cohen R, Bader A, & Magnani M (2010). Developing accountability in the lives of youth: Defining the operational features of juvenile treatment courts. Drug Court Review: Special Issue on Juvenile Drug Courts, 7(1), 125–170. [Google Scholar]
  24. McKay MM, Nudelman R, McCadam K, & Gonzales J (1996a). Evaluating a social work engagement approach to involving inner-city children and their families in mental health care. Research on Social Work Practice, 6(4), 462–472. doi: 10.1177/104973159600600404 [DOI] [Google Scholar]
  25. McKay MM, Nudelman R, McCadam K, & Gonzales J (1996b). Involving inner-city families in mental health services: First interview engagement skills. Research on Social Work Practice, 6, 462–472. [Google Scholar]
  26. Mericle AA, Belenko S, Festinger D, Fairfax-Columbo J, & McCart MR (2014). Staff perspectives on juvenile drug court operations:A multi-site qualitative study. 25(5), 614–636. doi: 10.1177/0887403413486342 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Muthén LK, & Muthén BO (2017). Mplus user’s guide (1998-2012) (7th ed.). Los Angeles, CA: Muthén & Muthén. [Google Scholar]
  28. National Association of Drug Court Professionals. (1997). Defining drug courts: The key components In National Association of Drug Court Professionals (Ed.). Washington, DC: Office of Justice Programs, Drug Courts Program Office. [Google Scholar]
  29. National Council on Crime and Delinquency. (2007). And justice for some: Differential treatment of youth of color in the justice system. Oakland, CA: National Council on Crime and Delinquency. [Google Scholar]
  30. Nock MK, & Ferriter C (2005). Parent management of attendance and adherence in child and adolescent therapy: A conceptual and empirical review. Clinical Child and Family Psychology Review, 8(2), 149–166. [DOI] [PubMed] [Google Scholar]
  31. Office of Juvenile Justice and Delinquency Prevention. (2016). Juvenile drug treatment court guidelines. Washington, D.C.: U.S. Department of Justice, Office of Justice Programs; Retrieved from https://www.ojjdp.gov/pubs/250368.pdf. [Google Scholar]
  32. Pennell J, Shapiro C, & Spigner C (2011). Safety, fairness, stability: Repositioning juvenile justice and child welfare to engage families and communities. Washington, D.C. [Google Scholar]
  33. Prado G, Pantin H, Schwartz SJ, Lupei NS, & Szapocznik J (2005). Predictors of engagement and retention into a parent-centered, ecodevelopmental HIV preventive intervention for Hispanic adolescents and their families. Journal of Pediatric Psychology, 31(9), 874–890. [DOI] [PubMed] [Google Scholar]
  34. Rodriguez N (2013). Concentrated disadvantage and the incarceration of youth: Examining how context affects juvenile justice. Journal of Research in Crime and Delinquency, 50(2), 189–215. [Google Scholar]
  35. Salvatore C, Henderson JS, Hiller ML, White E, & Samuelson B (2010). An observational study of team meetings and status hearings in a juvenile drug court. Drug Court Review, 7(1), 95–124. [Google Scholar]
  36. Salvatore C, Hiller ML, Samuelson B, Henderson JS, & White E (2011). A systematic observational study of a juvenile drug court judge. Juvenile and Family Court Journal, 62(4), 19–36. [Google Scholar]
  37. Schwalbe CS, & Maschi T (2010). Patterns of contact and cooperation between juvenile probation officers and parents of youthful offenders. Journal of Offender Rehabilitation, 49(6), 398–416. [Google Scholar]
  38. Shanahan R, & diZerega M (2016). Identifying, engaging, and empowering families: A charge for juvenile justice agencies: Center for Juvenile Justice Reform, McCourt School of Public Policy, Georgetown University. [Google Scholar]
  39. Snell-Johns J, Mendez JL, & Smith BH (2004). Evidence-based solutions for overcoming access barriers, decreasing attrition, and promoting change with underserved families. Journal of Family Psychology, 18(1), 19. [DOI] [PubMed] [Google Scholar]
  40. Staudt M (2007). Treatment engagement with caregivers of at-risk children: Gaps in research and conceptualization. Journal of Child Family Studies, 16(2), 183–196. [Google Scholar]
  41. Stiffman AR, Pescosolido B, & Cabassa LJ (2004). Building a model to understand youth service access: The gateway provider model. Mental Health Services Research, 6(4), 189–198. doi: 10.1023/b:Mhsr.0000044745.09952.33 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Szapocznik J, Perez-Vidal A, Brickman AL, Foote FH, Santisteban D, Hervis O, & Kurtines WM (1988). Engaging adolescent drug abusers and their families in treatment: A strategic structural systems approach. Journal of Consulting and Clinical Psychology, 56(4), 552. [DOI] [PubMed] [Google Scholar]
  43. Timmons-Mitchell J, Bender MB, Kishna MA, & Mitchell CC (2006). An independent effectiveness trial of multisystemic therapy with juvenile justice youth. Journal of Clinical Child & Adolescent Psychology, 35(2), 227–236. doi: 10.1207/s15374424jccp3502_6 [DOI] [PubMed] [Google Scholar]
  44. United States Census Bureau. (2012). Current population survey (CPS). Retrieved October 3, 2017, from https://www.census.gov/programs-surveys/cps.html
  45. Walker SC, Bishop AS, Pullmann MD, & Bauer G (2015). A research framework for understanding the practical impact of family involvement in the juvenile justice system: the juvenile justice family involvement model. American Journal of Community Psychology, 56(3-4), 408–421. [DOI] [PubMed] [Google Scholar]
  46. Walker SC, Bishop AS, Trayler K, Jaeger R, Gustaveson S, & Guthrie AC (2015). Impact of peer partner support on self efficacy for justice-involved parents: A controlled study of juvenile justice 101. Journal of Child and Family Studies, 24(2), 443–454. [Google Scholar]
  47. Wilson D, Olaghere A, & Kimbrell CS (2016). Developing juvenile drug court practices on process standards: A systematic review and qulitative synthesis. National Criminal Justice Reference Service; Retrieved from https://www.ncjrs.gov/pdffiles1/ojjdp/grants/250441.pdf. [Google Scholar]

RESOURCES