Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Jun 5.
Published in final edited form as: J Offender Rehabil. 2023 Jun 5;62(5):315–335. doi: 10.1080/10509674.2023.2213692

Juvenile Probation Officer Perception of Contingency Management to Target Caregiver Engagement and Training Outcomes

Stacy R Ryan-Pettes 1, Meghan Morrison 1, Jeff Randall 2, Colleen Halliday 2, David M Ledgerwood 3, Phillippe B Cunningham 2
PMCID: PMC10688515  NIHMSID: NIHMS1917977  PMID: 38046203

Abstract

Few community-based substance use treatment programs are available or skilled in treating justice-involved youth, highlighting the need to equip juvenile probation officers with the skills to deliver evidence-based substance use treatment. Contingency management (CM) is evidence-based for treating substance use and shows promise for juvenile probation officers’ successful uptake (positive opinions and trainability). However, research has not examined whether probation officers’ positive beliefs and trainability generalize to target behaviors beyond those displayed by youth, but that nevertheless affect youth outcomes. This study examined probation officers’ perceptions of using CM to engage caregivers and assessed probation officers’ CM knowledge and CM delivery after training in a protocol-specific CM program for caregivers of substance-using youth on probation. Results showed probation officers were ambivalent about CM for caregivers. Results also showed that age, training format and how competency is assessed may be essential to consider. Implications for the dissemination of CM and future research are discussed.

Keywords: Acceptability, Attitudes, Perception, Contingency Management, Training, Juvenile Probation

In 2019, more than 31 million youth were under juvenile court jurisdiction, with 65% receiving probation supervision as a sanction (Hockenberry & Puzzanchera, 2020). Of the youth placed on probation supervision, approximately 25% will meet criteria for a substance use disorder (Wasserman et al., 2005, 2010). Unfortunately, prevalence rates increase to about 50–55% as juveniles move deeper (e.g., county detention, state-run detention) into the juvenile justice system (Teplin et al., 2005; Wasserman et al., 2002, 2010). These prevalence rates are two to five times higher than their non-justice-involved peers (Merikangas et al., 2010).

It is challenging for the juvenile justice system to address this high prevalence of substance use. Although juvenile courts typically require any youth who uses substances to enroll in treatment and they use behavioral contingencies (i.e., rewards and sanctions) to motivate compliance, there are often few treatment options accessible in the community (Wilson et al., 2019). This problem is exasperated because many service providers lack sufficient training in evidence-based treatments and working with adolescents (Belenko & Logan, 2003), revealing critical gaps in coordinated service provisions.

Because of these service gaps in the community, there is an urgent need to equip juvenile probation officers (JPOs) with skills to deliver evidence-based substance use treatment. Theories of disruptive innovation and consolidated framework for implementation research suggest that uptake of any new intervention is more likely if the novel intervention is simple, convenient, feasible, effective, and accessible (Breimaier, et al., 2015; Wixom & Todd, 2005). A good fit for the training and experience of JPOs is incentive-based contingency management (CM). Incentive-based CM attempts to modify behaviors such that target behaviors are carefully monitored and reinforcing events (e.g., tangible rewards) occur when the target behavior is achieved. Incentive-based CM may be an appealing intervention for probation departments to adopt because it matches current techniques (i.e., contingency-based consequences) for supervising substance-using youth on probation (Wilson et al., 2019), and it has strong empirical support for treating substance use (Davis et al., 2016; Stanger et al., 2016).

To date, most research on barriers and facilitators of CM implementation and dissemination in any probation setting has focused on probation officers’ perception of CM and ability to be trained in using CM to target the offender’s behavior (e.g., Murphy et al., 2012). This research shows juvenile and adult probation officers generally hold positive perceptions of CM to target drug abstinence (Sheidow et al., 2020), and the use of CM significantly improves JPO attitudes about working with families (Rudes et al., 2021). This research also shows that when probation officers have access to low-cost but high-quality CM training (i.e., a one-day workshop or criterion-based computer assisted training), they incorporate CM in their routine services (Henggeler et al., 2013; Portillo et al., 2016). However, other research suggests that, even with knowledge of CM principles and practices, probation officers may not apply this information when developing new CM programs (Henggeler et al., 2006; Portillo et al., 2016).

As researchers attempt to disseminate CM to juvenile justice settings, it is essential to consider whether JPOs’ perceptions of CM generalize to target behaviors beyond those displayed by youth, but that nevertheless affect youth outcomes, like parental involvement. Research shows that parental involvement in court-mandated substance use treatment and court is related to youth outcomes (Stein et al., 2013). Yet, parent engagement is chronically low in juvenile justice settings (Schwalbe & Maschi, 2010). Incentive-based CM has strong empirical support for increasing treatment attendance, participation, and completion of treatment-related activities (Ledgerwood et al., 2008; Petry et al., 2010a). Taken together, incentive-based CM could help improve youth probation outcomes if also used to increase parent involvement in the process. Juvenile probation officers play an important role in the justice system as they provide probation services that target youth behavior. Therefore, understanding how JPOs perceive CM for parents and trainability in a parent-focused CM program could help to improve comprehensive CM programs within juvenile justice settings (Wixom & Todd, 2005).

We developed a CM program that uses the fishbowl/prize draw method to enhance caregiver engagement in probation services (Petry & Martin, 2002). Caregivers receive escalating chances for tangible reinforcers for 16-weeks, each week for completing up to three verifiable activities that meet the goals of enhancing participation in their youth’s drug treatment and court involvement (e.g., attending probation meetings; attending or engaging in prosocial activities with the youth). Verifiable is defined as documentation that shows the activity was completed. The number of draws earned for each completed activity (up to three) starts at one. It increases weekly by one with each consecutive week of three completed activities, up to five draws per activity on any one occasion. If a caregiver does not complete an activity, they do not receive that session’s prize draw, and the number of draws resets to one for each activity at the next session. The urn is filled with 500 slips of paper: small prizes (worth $1) make up 35% of total slips placed in the urn, large prizes (worth $20) make up 14.8%, and jumbo prizes (worth $100) comprise 0.2%. All activities are agreed upon a priori by the JPO and caregiver.

The current study sought to train JPOs in this caregiver CM program and to (1) explore JPOs’ perceptions of using CM to engage caregivers in juvenile probation services prior to training in the caregiver CM program, (2) examine whether training is related to improvements in general knowledge of CM principles, and (3) examine whether JPOs demonstrate competency in the caregiver CM procedures after training. Further, procedural changes due to the pandemic provided the opportunity to examine these outcomes for in-person training compared to distal training. Results of this study may inform the dissemination of CM in probation settings as research in this setting expands to include comprehensive CM models (Stanger et al., 2016). Data for the current study come from an ongoing randomized controlled trial designed to determine if delivering CM to caregivers by JPOs improves caregiver participation in their youth’s drug treatment and court process. As part of the trial, all participating JPOs receive training in CM and protocol-specific CM before receiving randomized families.

Method

Procedures

Training.

Consistent with prior research (Henggeler et al., 2013; Petry et al., 2012b), JPOs participated in a one-day training workshop delivered by two study authors (JR or DL) that consisted of didactics on CM, instruction in the protocol-specific CM procedures, and practice with the protocol-specific CM procedures via three unique role-plays (in this order). Before the training workshop (pretraining), JPOs completed a measure to assess their perceptions of incentives. After the training workshop (posttraining), JPOs completed a multiple-choice quiz about the protocol-specific CM procedures. At pretraining and posttraining, JPOs completed a general CM knowledge assessment.

Before the COVID-19 pandemic, there were three in-person training sessions held at two study sites, with a total of 27 JPOs participating. The JPOs completed consent, pretraining and posttraining measures and assessments, and posttraining role-plays on the day of training. Due to the pandemic, training shifted to an interactive workshop over Zoom (n =5 trainings with a total of 33 JPOs participating). For these workshops, consent and pretraining surveys were administered a week before the workshop and posttraining assessments were administered immediately after the workshop via Qualtrics. With in-person training, JPOs completed three role-plays on the day of training immediately after the training was complete. When training moved to Zoom, JPOs completed the role-plays when it was convenient for their schedules (M = 31 days after training, SD = 29.20, range = 4 −103). Other than timing, role-plays did not differ between the Zoom and in-person training. However, one JPO completed one role-play, and six completed two role-plays due to time constraints.

For the role-plays, JPOs role-played the part of a probation officer, and one of the trainers assumed the role of a caregiver in three unique scenarios. Each role-play scenario included a prompt which described the focus of each roleplay: (1) describing CM for the first time, (2) probation officer helps caregiver decide on pre-planned activities, and (3) withholding reinforcement from a caregiver who had not completed target activities with their teen. Under each prompt was a brief scenario, which was provided to JPOs verbally and in written form. For example, A caregiver in the third session would be eligible for 6 prize draws (2 per activity) this week, but she only completed two activities. In a role-play, demonstrate what you would do in this situation? After each roleplay, JPOs were praised for on model performance, and provided corrective feedback for off model performances. Audiotaped role-plays were later rated to assess competency.

Juvenile probation officers were considered to have mastered the training content if they achieved the following: (1) a score of 80% or higher on the posttraining CM knowledge assessment, (2) an 80% or higher on the protocol-specific CM quiz, and (3) a mean rating of “4” or higher on all items measuring competency in the three role-plays. Training procedures include remedial training should a JPO score below 80% on the posttraining CM knowledge assessment or protocol-specific CM quiz or receive an average rating less than “4” on any item measuring competency when role-plays were evaluated. Remedial training included discussion of missed items. Twelve JPOs scored below 80% on the CM knowledge assessment or protocol quiz. No JPOs scored less than “4” when competency was assessed via the recorded role-plays. There was no significant association between role-play score and number of days to complete role-plays for JPOs trained on Zoom for CM specific skills (r = −.42, p =.11) or General Skills (r = −.43, p =.10). This study was approved by the Baylor Institutional Review Board.

Participants

All JPOs employed at four participating juvenile probation departments in the southwestern, midwestern, and northwestern parts of the United States between August 2018 and December 2021, were eligible and invited to participate. All agreed to participate at three of the sites (Site One: n = 15; Site Two: n = 31; Site Three: n = 9). However, only five out of the 25 JPOs invited to participate at the fourth site agreed. Notably, the fourth site was recruited when most businesses were working remotely due to COVID-19 and recruitment was limited to email communication.

The final sample at pretraining (n = 60) included JPOs with a caseload of youth assigned to specialty court services (e.g., juvenile drug court or treatment court; n = 29) and non-specialty court services (n = 31). The average age of JPOs was 42.79 (SD =11.40), 58.3% identified as female, and 41.7% identified as male. Seventy percent (n = 42) of JPOs self-identified as white, 40% (n = 24) as Hispanic/Latinx, 18.3% (n = 11) as African American, 5.0% (n = 3) as multi-racial, and 1.7% (n = 1) as American Indian/Alaskan. Ninety percent (n = 54) of JPOs held a bachelor’s degree, 6.7% (n = 4) held a master’s degree, and 3.3% (n = 2) had missing data on this item. At the time of enrollment, JPOs had worked with justice-involved youth for an average of about 9.5 years (SD = 8.9 years). Eighteen percent (n = 11) of JPOs reported they received training in CM for substance use in the past.

Measures

Perception of incentives.

Perception of incentives was assessed using an adapted version of the Provider Survey of Incentives (PSI; Kirby et al., 2006). The PSI is a 44-item, self-report instrument designed to measure the attitudes of treatment providers towards the use of incentive-based CM. Participants rate their agreement with statements representing the limitations, objections, side effects, impracticalities, and positives of using incentives in treatment plans on a 5-point Likert scale. For the current study, the PSI was adapted to include the 28 items representing beliefs about incentives and slightly modified to assess attitudes toward using prizes to incentivize caregivers to engage in their child’s drug treatment and court process. Like the original subscales, Cronbach’s alpha showed the positive opinion subscale (α = .91) and the limitations/objections opinion subscale (α=.86) reached good reliability. Following previous research, results are reported according to the proportion of participants who rated each of the response options for each item (e.g., Ryan-Pettes et al., 2020).

Knowledge of basic CM principles.

Knowledge of basic CM principles was assessed using an adapted version of the Knowledge of Basic CM Principles measure (Petry et al., 2002). The original Knowledge of Basic CM Principles measure is a 20-item multiple-choice questionnaire that assesses one’s understanding of the rationale for CM and the behavioral principles of CM as applied to treating cocaine use disorder. For the current study, items were re-worded to focus on reinforcement of a priori activities, and items specific to cocaine use were removed because they were not relevant to the current study. The removal of these items resulted in a 15 multiple choice item measure. The original measure was used in studies that trained providers on incentive-based CM (e.g., Petry et al., 2012). Percent correct was used in analyses.

Protocol specific CM quiz.

The protocol-specific CM quiz included five multiple-choice questions (e.g., Chances to draw from the prize bowl are contingent on…), six True-or-False questions (e.g., Caregivers/parents must bring in objective verification that activities have been completed), and one fill-in-the-blank item with multiple levels in which the respondent must determine the appropriate number of draws based on session number and number of verified activities completed over 16 weeks. This measure was modeled off previous quizzes used in CM training studies (Petry et al., 2012). Percent correct was used in analyses.

Contingency management competence scale.

Competency was assessed using the Contingency Management Competence Scale (CMCS; Petry et al., 2010) to rate audiotaped role-plays. The CMCS is a 12-item measure with strong psychometric properties that assesses fidelity to prize-based CM procedures. CMCS ratings range from 1 (“poor”) to 7 (“excellent”), with scores of four or greater indicating an “acceptable” level of competency. The CMCS contains two subscales. One subscale consists of items that assess CM implementation procedures (i.e., CMCS CM Specific). For the current study, these items were adapted to rate JPO discussion of completed activities. The second subscale assesses provider skill, session structure, and empathy (i.e., CMCS CM General). Each roleplay was rated, and the average score of the three role plays for CM Specific and CM General skills was used in analyses.

Consistent with other efficacy trials (Henggeler et al., 2013; Petry et al., 2012), role-plays were conducted by one of the expert trainers (JR or DL), and all role-plays (Mminutes= 4.87, SD = 1.80) were rated by JR. JR and DL rated role-play sessions until 80% adjacent agreement (within two scale points) was achieved. This involved JR and DL independently rating 6 roleplays. The first author then calculated the agreement score. Since the trainers achieved a high level of agreement (96%), further coding to establish agreement was no longer necessary.

Data Analysis

The proportion of JPOs indicating agreement, disagreement, or neutral responses to the individual items of the perceptions of incentives questionnaire was calculated for descriptive purposes. Descriptive analyses were also conducted to determine whether JPOs met the training criteria. Paired t-tests were used to compare pretraining and posttraining scores on the CM knowledge assessment. Post hoc analyses were conducted to examine whether training outcomes differed between JPOs who completed posttraining assessments compared to those who did not and between JPOs who completed in-person training compared to those trained online. Additionally, correlation analysis was used to examine associations between age and years of employment within the subgroups. Independent samples t-test and chi-square tests were performed to examine these subgroup differences. The non-parametric alternative to the t-test (i.e., Wilcoxon signed-rank test) was selected when normality or assumptions involving cell size were violated. Finally, to test differences in perception of incentives among the subgroups, positive opinion and objection/limitation items were grouped, and summary scores were calculated for strongly agree/agree, strongly disagree/disagree, and neutral responses.

Results

Perception of incentives.

Table 1 shows the proportion of JPOs that responded with agreement, disagreement, or neutrality to the positive opinion items. Notably, 75% of JPOs indicated they would favor adding a prize program for caregivers to an adolescent treatment program, and 73% would favor adding a prize program for caregivers to adolescent court. The three most highly rated items were: (1) Prizes are useful if they reward caregivers for activities that support their child’s efforts to stay away from drugs or alcohol; (2) Prizes are more likely to have positive effects on parents and caregivers than they are to have negative effects; and (3) Any source of caregiver motivation to get involved in their child’s efforts to stay away from drugs or alcohol not just internal motivation is a good thing for the child’s efforts. Almost no JPO strongly disagreed with positive opinion items, and the proportion of JPOs indicating disagreement across positive opinion items averaged .07. More JPOs indicated a neutral response. The proportion indicating a neutral response ranged from .15 to .42 and averaged .27 across the positive opinion items.

Table 1.

Percentage of Juvenile Probation Officer Responses to Items on the Perception of Incentives Survey

Strongly Disagree Disagree Neutral Agree Strongly Agree Item Mean
Positive Beliefs
Prize are useful if they reward caregivers for activities that support their child’s efforts to stay away from drugs or alcohol. 0% 2% 15% 67% 17% 3.98
Prizes are more likely to have positive effects on parents and caregivers than they are to have negative effects. 3% 0% 15% 43% 38% 4.13
Any source of caregiver motivation to get involved in their child’s efforts to stay away from drugs or alcohol not just internal motivation is a good thing for the child’s efforts. 0% 2% 20% 52% 27% 4.03
Overall, I would be in favor of adding a prize program for caregivers to an adolescent treatment programs.* 0% 3% 20% 53% 22% 3.95
Prize are worthwhile because they can help caregivers take the first steps to get involved in their child’s court and treatment activities.* 0% 3% 22% 57% 17% 3.88
Overall, I would be in favor of adding a prize program for caregivers to adolescent court. 0% 3% 23% 50% 23% 3.93
Giving prizes for participation in court and the child’s drug treatment helps caregivers to become involved in the child’s efforts to stay away from drugs or alcohol. 0% 7% 20% 58% 15% 3.82
Overall prizes are good for the caregivers relationship with court.* 0% 3% 32% 53% 10% 3.71
An advantage of prize programs is that they focus on what is good in the caregiver’s behavior (i.e. their ability to support their child’s efforts to stay away from drugs or alcohol) not what went wrong in court/the child’s drug treatment.* 0% 7% 28% 43% 20% 3.78
Prize programs that cost $50 per caregiver per month are worth it considering how effective they are. 2% 7% 43% 38% 10% 3.48
Overall, prizes are good for the caregiver’s relationship with the child’s counselor. 0% 12% 42% 40% 7% 3.42
Prize programs that cost $150 per caregiver per month are worth it considering how effective they are. 0% 13% 42% 35% 10% 3.42
Prizes can be useful whether or not they address the reasons why some caregivers don’t participate in their child’s efforts to stay away from drugs or alcohol. 2% 25% 28% 42% 3% 3.20
Objections/Limitations
Giving prizes to caregivers doesn’t address the reasons caregivers fail to participate in their child’s efforts to stay away from drugs or alcohol. 2% 18% 42% 33% 5% 3.22
It wouldn’t be right to give a prize to a caregiver for participating in the youth’s drug treatment or court when they aren’t fulfilling other goals like monitoring their child’s drug use.* 2% 30% 28% 23% 15% 3.20
If a caregiver is participating in treatment or court just to get prizes, it could hurt the process. 7% 27% 30% 23% 13% 3.10
A problem with prizes is that caregiver participation in court and treatment will last only for as long as the prizes are given. 0% 37% 35% 23% 5% 2.97
Prizes are not right because they are rewarding caregivers for what they should be doing in the first place. 18% 40% 20% 13% 8% 2.53
If you give prizes to caregivers who’ve earned them, but not to others, it will result in caregivers arguing about rewards.* 5% 43% 30% 15% 5% 2.71
Prize programs require close tracking of caregiver attendance and participation and are too labor intensive to incorporate into court. 8% 43% 30% 15% 3% 2.62
Prizes are a bribe. 17% 37% 32% 10% 5% 2.50
Court sanctions are more effective than prizes in getting caregivers to participate in their child’s efforts to stay away from drugs or alcohol. 7% 48% 32% 10% 3% 2.55
Prizes will stop caregivers from seeing beyond the external reward and prevent them from realizing their internal motivation. 3% 48% 35% 10% 3% 2.62
Prize programs are not consistent with my philosophy of how courts should operate. 20% 35% 33% 5% 7% 2.43
Giving parents/caregivers prizes for treatment or court attendance will have no effect on their attendance. 15% 48% 28% 5% 3% 2.33
Most caregivers would sell the prizes they receive.* 3% 45% 45% 3% 2% 2.53
Overall, prizes have a negative effect on the parent’s/caregiver’s relationship with court. 28% 42% 25% 2% 3% 2.10
Overall prizes have a negative effect on the caregiver’s relationship with the child’s counselor. 15% 58% 25% 2% 0% 2.13
Open in a new tab
*

indicates n=1 item missing so total does not equal 100.

A Wilcoxon signed-rank test showed that JPOs who did not complete the posttraining assessments rated fewer positive opinion items as agree/strongly agree (Median = 7 items) than those who did complete the posttraining assessments (Median = 11 items), z = −2.65, p = 0.008. Additionally, a Wilcoxon signed-rank test showed that JPOs who did not complete the posttraining assessment rated more positive opinion items as neutral (Median = 6 items) than those who did complete the posttraining assessment (Median = 1 item), z = 2.39, p = 0.017.

Table 1 also shows the proportion of JPOs who responded with agreement, disagreement, or neutrality on the objection/limitation items. The response profile to the objection/limitation items suggests a greater level of ambiguity about JPOs’ objections toward incentive-based CM for caregivers. More than half of the JPOs expressed disagreement with more than half (9 of 15; 60%) of the items. The proportion of JPOs indicating neutrality on objection/limitation items ranged from .25 - .42. Those endorsing agreement ranged from 0 - .38, with 20% or more agreeing with 40% of the items (6 of 15). The three items agreed to the most were: (1) Giving prizes to caregivers doesn’t address the reasons caregivers fail to participate in their child’s efforts to stay away from drugs or alcohol (38%); (2) It wouldn’t be right to give a prize to a caregiver for participating in the youth’s drug treatment or court when they aren’t fulfilling other goals such as monitoring their child’s drug use (38%); and (3) If a caregiver is participating in treatment or court just to get prizes, it could hurt the process (37%).

A Wilcoxon signed-rank test showed that JPOs who did not complete the posttraining assessments rated fewer objection/limitation items as disagree/strongly disagree (Median=9 items) than those who did complete the posttraining assessments (Median=4 items), z = −2.58, p = 0.010. Additionally, a Wilcoxon signed-rank test showed a trend toward JPOs who did not complete the posttraining assessment rating more objection/limitation items as neutral (Median =6 items) than those who did complete the posttraining assessment (Median=3 items), z = 1.92, p = 0.055.

Knowledge of basic CM principles.

Knowledge of basic CM principles improved pretraining (M = 63%, SD = .14) to posttraining (M = 70%, SD = .16), t(42) = −2.30, p = .013. These data show that, although scores improved from pretraining to posttraining, JPOs, on average, did not score at or above the cutoff for adequate performance (80%) at the end of training. Additional analyses showed that those who did not complete the posttraining surveys scored lower (M = 44%, SD=.18) on this pretraining survey compared to those who completed the posttraining surveys (M = 63%, SD = .14), t(54) = 4.16, p < .001.

Protocol specific CM quiz.

Most JPOs did not perform at or above the cutoff (80%) for adequate performance on the protocol specific CM quiz. While 28% scored above the cutoff, the majority (71.7%) scored below the cutoff. Of those scoring below the cutoff, 7.5% performed near the cutoff with a score of 78%.

Contingency management competence scale.

The mean CMCS score for the CM specific subscale was 5.54 (SD = .35; Min. = 4.80, Max = 6.39), indicating average competence ranging between “good” and “very good.” The mean CMCS score for the general skillfulness subscale was 5.75 (SD = 5.75; Min = 4.89, Max = 7.00), similarly demonstrating average competence ranging between “good” and “very good.”

Post hoc analyses.

Post hoc analyses examining associations between training format (in-person vs. online) and training outcome were also conducted. Results showed that training format was not associated with whether JPOs completed the post-training measures, χ2 (1) = .53, p = .469. However, training format was related to training outcomes. Table 2 shows that JPOs who completed in-person training demonstrated significant increases in CM knowledge from pretraining to posttraining. However, JPOs who completed training online did not demonstrate a change in CM knowledge pretraining to posttraining. Also shown in Table 2, JPOs who completed in-person training scored significantly higher on the protocol quiz compared to those trained online. Finally, JPOs who were trained in-person showed a trend toward higher CMCS CM specific competency compared to those trained online, and differences between these groups on CMCS general levels of competency was not significant (see Table 2). Notably, among JPOs trained online, there was a trend for a negative association between age and score on the protocol-specific quiz. However, among JPOs trained in person, age was positively associated with the protocol specific quiz and there was a trend for a positive association between years of employment and lower general skills on the CMCS (see Table 3).

Table 2.

Pre- and Post-CM Training Scores of Probation Officers

Training Measure In-Person Online
Paired t-test Paired t-test Independent Samples t-test
M SD t(df) p M SD t(df) p t df p
CM Knowledge
 Pre-test 68% .11 61% .15
 Post-test 81% .09 −4.31 (15) .001 63% .15 −4.75 (26) .64
Protocol Quiz 77% .17 51% .15 6.03 51 .000
Role Play (CMCS)
 CM specific 5.66 .34 5.46 .33 1.85 39 .073
 General 5.64 .46 5.82 .50 −1.14 39 .261
Open in a new tab

CM = Contingency Management; CMCS = Contingency management competence scale.

Table 3.

Correlation Among Age, Years Employed and Variables Related to Training Outcomes

Age Years employed CM Knowledge Pre-score CM Knowledge Post-score Protocol Specific CM Quiz CMCS CM Specific CMCS CM General
Age ---- .62** .13 −.15 −.38 −.08 .07
Years employed .78** ---- .09 −.17 −.36 −.26 .07
CM Knowledge Pre-score −.18 .24 ---- .25 −.00 .33 .44*
CM Knowledge Post-score .12 .13 .17 ---- .17 −.03 −.04
Protocol Specific CM Quiz .47* .27 −.02 .17 ---- .05 −.15
CMCS CM Specific −.03 −.00 .29 .52* .16 ---- .59**
CMCS CM General −.28 −.41 .33 .32 .19 .77** ----
Open in a new tab

Correlations below the diagonal are associations for the subsample of JPOs trained in person and correlations above the diagonal are for the subsample of JPOs trained online; CM = Contingency Management; CMCS = Contingency management competence scale.

Discussion

The current study is the first to examine JPOs’ perception of using CM to address caregiver engagement in juvenile probation services and JPO trainability in a caregiver specific CM protocol for addressing caregiver engagement. Findings revealed JPOs are ambivalent about incentives for addressing caregiver engagement in probation services. Findings also showed that training in CM enhances CM knowledge, though training format (in-person vs. online) and JPO age may be important considerations when assessing mastery of content and competency.

Positive Perceptions of Incentives

Among the positive opinion items, most JPOs reported they would favor adding incentive-based CM for caregivers to an adolescent treatment program and juvenile court proceedings. Most JPOs also believed incentive-based CM could help increase parent activities meant to help keep their teens away from substances and benefit the caregiver-youth relationship. Most JPOs also believed that incentive-based CM could positively impact caregiver motivation to get involved in their teen’s efforts to achieve abstinence and attend court and treatment services. The proportion of JPOs expressing neutrality and disagreement with positive belief items increased when the focus of using CM with caregivers shifted away from youth-centered reasons (e.g., Prize are useful if they reward caregivers for activities that support their child’s efforts to stay away from drugs or alcohol) to caregiver-centered reasons (e.g., Overall, prizes are good for the caregiver’s relationship with the child’s counselor) and practical issues (e.g., Prize programs that cost $50 per caregiver per month are worth it considering how effective they are).

Results of this study are consistent with prior work showing probation officers are in favor of adding CM to existing services and extends previous work by demonstrating JPOs are agreeable to adding CM for caregiver behaviors. Additional comparisons with the existing literature are limited because of measurement differences. Prior research with JPOs measured implementation items from the perceptions survey, (Rudes et al., 2021), omitting nuanced items or dropping items from the measure that are irrelevant for the probation context, (Murphy et al., 2012) instead of modifying items to align the items with the probation context. However, the current results are promising because the frequency of endorsement for positive opinion items is similar to research in other populations. The proportion of JPOs agreeing or strongly agreeing with positive belief items is similar to research with parents of young adult with a history of opioid misuse (Mathis et al., Under Review), patients in substance use treatment (Getty et al., 2021; Leickly et al., 2019), and treatment providers (e.g., Kirby et al., 2006).

Objection/limitation Perceptions of Incentives

Ambivalence concerning CM for caregivers is more pronounced among the objection/limitation items. The items that JPOs objected to the most is consistent with prior research showing CM concerns related to fairness, eroding personal responsibility and contextual processes (Getty et al., 2021; Hoskins et al., 2019; Kirby et al., 2006; Ryan-Pettes et al., 2020). However, results showed JPOs were largely ambivalent in their objection/limitation beliefs, with a quarter to one-third or more of JPOs expressing neutral opinions to all but one item. Comparing the pattern of ambivalence in this study to prior research is somewhat difficult. Results are typically presented as only the percentage of participants who agreed with individual items (Kirby et al., 2006; Ryan-Pettes et al., 2020). Research that presents response frequencies to all response options is needed because this data allows for better discernment of strong opposition vs. neutrality. The results of the current study are consistent with the select few that report the proportion of responses for all response options (Getty et al., 2021; Rudes et al., 2021).

Ambivalence toward the use of CM for caregivers could present a barrier to the dissemination of CM for caregiver engagement in juvenile probation. It is encouraging that prior research shows that using CM positively impacts JPOs’ perceptions of incentives (Rudes et al., 2021). Thus, implementing a CM program may be enough to shift neutral opinions to positive opinions. However, supplemental analyses showed that JPOs who did not complete posttraining surveys agreed with fewer positive opinion items, disagreed with fewer objection items, and expressed more neutral beliefs across both subscales. These results suggest it may be challenging to retain some JPOs through training to use novel CM approaches. Future dissemination research should seek to provide ways to address ambivalence during training. Results of this study could be used to develop an initial CM training protocol that includes content that addresses ambivalence in the areas endorsed by JPOs.

Posttraining Outcomes

First, results showed that knowledge of CM principles improved from pretraining to posttraining but, for the total sample, did not reach 80%. Post hoc analyses showed the training format impacted gains in CM knowledge. Those trained online showed no significant change pretraining to posttraining, achieving scores far below the cutoff. However, those who completed in-person training significantly improved, scoring above the cutoff after training. These results are interesting given nonsignificant differences in pretraining knowledge scores between the two groups. Second, the average performance on the protocol-specific quiz did not reach 80%. However, JPOs trained in-person demonstrated significantly higher scores than those trained online. Interestingly, as JPO age increased, quiz score increased among those trained in person; and there was a trend in this association among those trained online but in the opposite direction. As JPO age increased, quiz score decreased. Finally, competency demonstrated through the role-plays was at or above previous research with therapists (Petry et al., 2012). JPOs trained online and in-person demonstrated competency through role-plays at levels in the “very good” range, with those trained in-person showing slightly higher ratings.

Results are consistent with prior research showing training in CM improves participant knowledge about CM principles (Aletraris et al., 2015; Rash et al., 2013), including for JPOs (Henggeler et al., 2013). Results are inconsistent with research showing that digital-based formats in training lead to significant gains in CM knowledge (Henggeler et al., 2013). There are at least three possible reasons why the CM training delivered online failed to significantly increase JPOs’ knowledge at posttraining. The training was conducted over Zoom and was not designed to be delivered in this format. Thus, essential pedagogy for learning (e.g., self-paced, video to allow replay) was not part of the online training experience and this may have impacted learning (Ali & Miraz, 2020). Two, Henggeler and colleague’s (2013) digital CM training program was developed with probation officer’s feedback. Thus, participants may have been more engaged, which enhanced learning. Three, the online training for the current study was conducted during social distancing due to COVID-19. It is possible that participants were distracted and multitasking during training or unable to fully concentrate due to Zoom fatigue.

Interestingly, although JPOs trained online scored poorly on the protocol-specific quiz compared to those trained in-person, both groups achieved “very good” competency levels in the role-plays. Results suggest that multiple formats for assessing uptake of knowledge may be necessary when training JPOs in the novel use of CM procedures. However, more research is needed to discern best practices for training JPOs using online platforms that involve live training. Moreover, additional research that examines whether age moderates the effects of training format is needed. Preliminary results of this study show a trend toward negative associations between age and the protocol quiz for those trained online, but positive associations for those trained in person. Research is needed to determine whether older JPOs trained online require supplemental support. Notably, prior research shows that skillfulness, measured on the CMCS, is related to client outcomes (Hartzler et al., 2017). Research also shows that compliance with CM principles is associated with the number of training hours (Rash et al., 2013). Taken together, research that examines the predictive value of a survey-style quiz versus role-play demonstration within the context of different training models (in-person vs. remote) with JPOs is needed, especially with older JPOs as there was no associations between age and skillfulness for those trained online or in-person.

The completion of posttraining assessments by JPOs was not related to the training method. Thus, those lost to follow-up are a unique group worth examining. It may be the case that those lost to follow-up were less comfortable or open to the training, considering their lower CM knowledge scores coupled with the tendency to rate perception items neutrally. It may be critical to tailor the training curriculum to JPOs who express ambivalence to retain them through training and implementation. Future research could develop a tailored curriculum that uses pretraining perception scores or offer extended training to those expressing ambivalence and who do not complete prescribed training (Helseth et al., 2018).

Limitations and Strengths

The results of this study should be considered within the context of some limitations. First, this project started pre-pandemic and continued during the pandemic. While use of Zoom during the pandemic helped this project continue, the content was not designed for online learning. We cannot separate the sample into meaningful pre-COVID and during-COVID subgroups because these groups are confounded with additional factors (e.g., site location). Second, sampling was not random. Thus, we cannot be sure about the generalizability of the perception of incentives results. Realistically, however, justice departments have different family experiences and expectations of families. Thus, site differences may be unavoidable in a study examining perceptions of incentives for use with target behaviors other than abstinence (Sloas et al., 2019). Third, this study did not include a post-training measure of JPO’s perceptions of incentives. It is possible that beliefs about CM may have changed after the training, especially given most of the JPOs reported no prior CM training. Fourth, there are some limitations associated with relying solely on self-report measures. The pattern of neutral responses observed could be a result of a response set. Despite these limitations, this study has some strengths. First, this study successfully recruited multiple courts that provide services to a client base with varied demographics. Second, the current study was conducted by experts in CM, which facilitated an easy transition from in-person to online training. Third, the proportion of JPOs responding to all response options on the PIS provides information for future dissemination efforts.

Conclusion

In summary, JPOs hold largely positive beliefs about CM for caregiver engagement, but there is some variation depending on whether items ask JPOs to report on their beliefs that CM could be helpful beyond reasons that are youth centered. Collectively, results suggest the perceptions of incentive measure could be used to tailor CM trainings when novel CM procedures are introduced, including protocols focused on caregiver engagement. Furthermore, results suggested that CM can be successfully implemented with appropriate resources and training, though multiple formats for assessing knowledge may be necessary when using online platforms. Older JPOs trained online might require supplementary support, and further research is warranted to understand the predictive value of survey-style quizzes versus role-play demonstrations within different training models. Ultimately, this study highlights the potential for the dissemination of CM for caregiver engagement in juvenile justice settings.

Funding details:

This work was supported by the National Institute on Minority Health and Health Disparities under Grant R01MD011322.

Footnotes

Disclosure statement: The authors report there are no competing interests to declare. This manuscript has not been published elsewhere and that it has not been submitted simultaneously for publication elsewhere.

REFERENCES

  1. Aletraris L, Shelton JS, & Roman PM (2015). Counselor attitudes toward contingency management for substance use disorder: Effectiveness, acceptability, and endorsement of incentives for treatment attendance and abstinence. Journal of Substance Abuse Treatment, 57, 41–48. 10.1016/j.jsat.2015.04.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ali M, & Miraz MH (2020). Assessment of delivery platforms and online pedagogy requirements. International Conference on Computing, Electronics Communications Engineering, 176–181. 10.1109/iCCECE49321.2020.9231220 [DOI]
  3. Belenko S, & Logan TK (2003). Delivering more effective treatment to adolescents: Improving the juvenile drug court model. Journal of Substance Abuse Treatment, 25(3), 189–211. 10.1016/S0740-5472(03)00123-5 [DOI] [PubMed] [Google Scholar]
  4. Breimaier HE, Heckemann B, Halfens RJ, & Lohrmann C. (2015). The Consolidated Framework for Implementation Research (CFIR): a useful theoretical framework for guiding and evaluating a guideline implementation process in a hospital-based nursing practice. BMC nursing, 14, 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Davis DR, Kurti AN, Skelly JM, Redner R, White TJ, & Higgins ST (2016). A review of the literature on contingency management in the treatment of substance use disorders. Preventive Medicine, 92, 36–46. 10.1016/j.ypmed.2016.08.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Hartzler B, Beadnell B, & Donovan D. (2017). Predictive validity of addiction treatment clinicians’ post-training contingency management skills for subsequent clinical outcomes. Journal of Substance Abuse Treatment, 72, 126–133. 10.1016/j.jsat.2015.11.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Helseth SA, Janssen T, Scott K, Squires DD, & Becker SJ (2018). Training community-based treatment providers to implement contingency management for opioid addiction: Time to and frequency of adoption. Journal of Substance Abuse Treatment, 95, 26–34. 10.1016/j.jsat.2018.09.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Henggeler SW, Chapman JE, Rowland MD, Sheidow AJ, & Cunningham PB (2013). Evaluating training methods for transporting contingency management to therapists. Journal of Substance Abuse Treatment, 45(5), 466–474. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Henggeler SW, Halliday-Boykins CA, Cunningham PB, Randall J, Shapiro SB, & Chapman JE (2006). Juvenile drug court: Enhancing outcomes by integrating evidence-based treatments. Journal of Consulting and Clinical Psychology, 74(1), 42–54. [DOI] [PubMed] [Google Scholar]
  10. Hockenberry S, & Puzzanchera C. (2020). Juvenile Court Statistics (NCJ 254798).
  11. Hoskins K, Ulrich CM, Shinnick J, & Buttenheim AM (2019). Acceptability of financial incentives for health-related behavior change: An updated systematic review. Preventive Medicine, 126, 105762. 10.1016/j.ypmed.2019.105762 [DOI] [PubMed] [Google Scholar]
  12. Kirby KC, Benishek LA, Dugosh KL, & Kerwin ME (2006). Substance abuse treatment providers’ beliefs and objections regarding contingency management: Implications for dissemination. Drug and Alcohol Dependence, 85(1), 19–27. 10.1016/j.drugalcdep.2006.03.010 [DOI] [PubMed] [Google Scholar]
  13. Kirby KC, Dwyer MJ, Burrows C, Fife DA, Bresani E, Tabit M, & Raiff BR (2021). Beliefs related to health care incentives: Comparison of substance use disorder treatment providers, medical treatment providers, and a public sample. Journal of Substance Abuse Treatment, 129, 108383. 10.1016/j.jsat.2021.108383 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Ledgerwood DM, Alessi SM, Hanson T, Godley MD, & Petry NM (2008). Contingency management for attendance to group substance abuse treatment administered by clinicians in community clinics. Journal of Applied Behavior Analysis, 41(4), 517–526. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Leickly E, Skalisky J, Angelo FA, Srebnik D, McPherson S, Roll JM, Ries RK, & McDonell MG (2019). Perspectives on a contingency management intervention for alcohol use among consumers with serious mental illness. Psychiatric Rehabilitation Journal, 42(1), 26–31. 10.1037/prj0000330 [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Merikangas KR, He J, Burstein M, Swanson SA, Avenevoli S, Cui L, Benjet C, Georgiades K, & Swendsen J. (2010). Lifetime prevalence of mental disorders in U.S. adolescents: results from the national comorbidity survey replication–adolescent supplement (NCS-A). Journal of the American Academy of Child & Adolescent Psychiatry, 49(10), 980–989. 10.1016/j.jaac.2010.05.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Murphy A, Rhodes AG, & Taxman FS (2012). Adaptability of contingency management in justice settings: Survey findings on attitudes toward using rewards. Journal of Substance Abuse Treatment, 43(2), 168–177. 10.1016/j.jsat.2011.11.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Petry NM, Alessi SM, & Ledgerwood DM (2012). Contingency management delivered by community therapists in outpatient settings. Drug and Alcohol Dependence, 122(1–2), 86–92. 10.1016/j.drugalcdep.2011.09.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Petry NM, Alessi SM, Ledgerwood DM, & Sierra S. (2010b). Psychometric properties of the contingency management competence scale. Drug and Alcohol Dependence, 109(1–3), 167–174. 10.1016/j.drugalcdep.2009.12.027 [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Petry NM, & Martin B. (2002). Low-cost contingency management for treating cocaine- and opioid-abusing methadone patients. Journal of Consulting and Clinical Psychology, 70(2), 398–405. 10.1037//0022-006x.70.2.398 [DOI] [PubMed] [Google Scholar]
  21. Petry NM, Weinstock J, Alessi SM, Lewis MW, & Dieckhaus K. (2010a). Group-based randomized trial of contingencies for health and abstinence in HIV patients. Journal of Consulting and Clinical Psychology, 78(1), 89–97. 10.1037/a0016778 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Portillo S, Rudes DS, & Taxman FS (2016). The transportability of contingency management in problem-solving courts. JQ: Justice Quarterly, 33(2), 267–290. [Google Scholar]
  23. Mathis P, Beuley G, & Ryan-Pettes S,R (Under Review). Acceptability and willingness to pay for contingency management interventions among parents of adolescents. [DOI] [PubMed]
  24. Rash CJ, DePhilippis D, McKay JR, Drapkin M, & Petry NM (2013). Training workshops positively impact beliefs about contingency management in a nationwide dissemination effort. Journal of Substance Abuse Treatment, 45(3), 306–312. 10.1016/j.jsat.2013.03.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Rudes DS, Viglione J, Sheidow AJ, McCart MR, Chapman JE, & Taxman FS (2021). Juvenile probation officers’ perceptions on youth substance use varies from task-shifting to family-based contingency management. Journal of Substance Abuse Treatment, 120, 108144. 10.1016/j.jsat.2020.108144 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Ryan-Pettes SR, Devoto A, & DeFulio A. (2020). Acceptability and willingness to pay for contingency management interventions among parents of young adults with problematic opioid use. Drug and Alcohol Dependence, 206, 107687. 10.1016/j.drugalcdep.2019.107687 [DOI] [PubMed] [Google Scholar]
  27. Schwalbe CS, & Maschi T. (2010). Patterns of contact and cooperation between juvenile probation officers and parents of youthful offenders. Journal of Offender Rehabilitation, 49(6), 398–416. 10.1080/10509674.2010.499055 [DOI] [Google Scholar]
  28. Sheidow AJ, McCart MR, Chapman JE, & Drazdowski TK (2020). Capacity of juvenile probation officers in low-resourced, rural settings to deliver an evidence-based substance use intervention to adolescents. Psychology of Addictive Behaviors, 34(1), 76–88. 10.1037/adb0000497 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Sloas L, Wooditch A, & Taxman FS (2019). Assessing the use and impact of points and rewards across four federal probation districts: A contingency management approach. Victims & Offenders, 14(7), 811–831. 10.1080/15564886.2019.1656691 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Stanger C, Lansing AH, & Budney AJ (2016). Contingency management approaches for adolescent substance use disorders. Child and Adolescent Psychiatric Clinics of North America, 25(4), 645–659. 10.1016/j.chc.2016.05.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Stein DM, Deberard S, & Homan K. (2013). Predicting success and failure in juvenile drug treatment court: A meta-analytic review. Journal of Substance Abuse Treatment, 44(2), 159–168. 10.1016/j.jsat.2012.07.002 [DOI] [PubMed] [Google Scholar]
  32. Teplin LA, Abram KM, McClelland GM, Washburn JJ, & Pikus AK (2005). Detecting mental disorder in juvenile detainees: Who receives services. American Journal of Public Health, 95(10), 1773–1780. 10.2105/AJPH.2005.067819 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Wasserman GA, McReynolds LS, Ko SJ, Katz LM, & Carpenter JR (2005). Gender differences in psychiatric disorders at juvenile probation intake. American Journal of Public Health, 95(1), 131–137. 10.2105/AJPH.2003.024737 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Wasserman GA, Mcreynolds LS, Lucas CP, Fisher P, & Santos L. (2002). The voice DISC-IV with incarcerated male youths: Prevalence of disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 41(3), 314–321. 10.1097/00004583-200203000-00011 [DOI] [PubMed] [Google Scholar]
  35. Wasserman GA, McReynolds LS, Schwalbe CS, Keating JM, & Jones SA (2010). Psychiatric disorder, comorbidity, and suicidal behavior in juvenile justice youth. Criminal Justice and Behavior, 37(12), 1361–1376. 10.1177/0093854810382751 [DOI] [Google Scholar]
  36. Wilson DB, Olaghere A, & Kimbrell CS (2019). Implementing juvenile drug treatment courts: A meta-aggregation of process evaluations. Journal of Research in Crime and Delinquency, 56(4), 605–645. 10.1177/0022427819826630 [DOI] [Google Scholar]
  37. Wixom BH, & Todd PA (2005). A theoretical integration of user satisfaction and technology acceptance. Information Systems Research, 16(1), 85–102. 10.1287/isre.1050.0042 [DOI] [Google Scholar]

RESOURCES