Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2023 Sep 1.
Published in final edited form as: J Child Psychol Psychiatry. 2021 Dec 9;63(9):992–1001. doi: 10.1111/jcpp.13554

A Randomized Controlled Trial of Technology-Enhanced Behavioral Parent Training: Sustained Parent Skill Use and Child Outcomes at Follow-Up

Justin Parent 1, Margaret T Anton 2, Raelyn Loiselle 3, April Highlander 3, Nicole Breslend 4, Rex Forehand 4, Megan Hare 5, Jennifer K Youngstrom 3, Deborah J Jones 3
PMCID: PMC9177891  NIHMSID: NIHMS1758495  PMID: 34888861

Abstract

Background:

Early-onset (3 to 8 years old) disruptive behavior disorders (DBDs) have been linked to a range of psychosocial sequelae in adolescence and beyond, including delinquency, depression, and substance use. Given that low-income families are overrepresented in statistics on early-onset DBDs, prevention and early-intervention targeting this population is a public health imperative. The efficacy of Behavioral Parent Training (BPT) programs such as Helping the Noncompliant Child (HNC) has been called robust; however, given the additional societal and structural barriers faced by low-income families, family engagement and retention barriers can cause effects to wane with time. The current study extends preliminary work by examining the potential for a Technology-Enhanced HNC (TE-HNC) program to improve and sustain parent skill proficiency and child outcomes among low-income families.

Methods:

A randomized controlled trial with two parallel arms was the design for this study. A total of 101 children (3–8 years old) with clinically significant problem behaviors from low-income households were randomized to HNC (n = 54) or TE-HNC (n = 47). Participants were assessed at pre-treatment, post-treatment, 3-month, and 6-month follow-ups. Primary outcomes were parent-reported and observed child behavior problems. Secondary outcomes included observed parenting skills use (ClinicalTrials.gov Identifier: NCT02191956).

Results:

Primary analyses used latent curve modeling to examine treatment differences in the trajectory of change during treatment, maintenance of treatment gains, and levels of outcomes at the 6-month follow-up. Both programs yielded improvements in parenting skills and child problems at post-treatment. However, TE-HNC families evidenced greater maintenance of parent-reported and observed child behavior and observed positive parenting skills at the 6-month follow-up.

Conclusions:

Our findings contribute to an ongoing line of work suggesting that technology-enhanced treatment models hold promise for increasing markers of engagement in BPT and sustaining long-term outcomes among low-income families.

Keywords: behavioral parent training, technology, low-income families, child behavior disorders

Introduction

Early-onset disruptive behavior disorders (DBDs; oppositional defiant disorder, conduct disorder) co-occurring with attention-deficit/hyperactivity disorder are the second-leading cause of child mental health referrals worldwide (Merikangas et al., 2009). Early-onset DBDs affect an estimated 113 million youth (Polanczyk et al., 2015), increasing their vulnerability for later delinquency and antisocial behavior, depression, substance use, academic underachievement, relationship and employment instability, and chronic illness (e.g., Fergusson et al., 2005; Odgers et al., 2007; Piquero et al., 2016). Therefore, early identification and treatment of DBDs are critical, with estimates suggesting these efforts can save $2.6 to $4.4 million per high-risk child (Cohen & Piquero, 2009).

Behavioral Parent Training (BPT), a family of evidence-based treatments with a common history, theory, and practice elements, is the recommended first-line treatment for children with DBDs (see Comer et al., 2013; Kaminski & Claussen, 2017 for reviews). Research suggests that BPT, which targets the coercive cycle of parent-child interaction implicated in the etiology and maintenance of early-onset DBDs, is more effective than any other treatment approach to date (see Chorpita et al., 2011; Leijten et al., 2013 for reviews). Indeed, the impact of BPT has been called relatively “robust,” yielding moderate effects for improvements in parenting (d = .45) and child behavior (d = .42) (see Lundahl et al., 2006, for a review). Moreover, data suggest that BPT efficacy is equivalent regardless of socioeconomic status and particularly when the child’s behavior is clinically significant (see Leijten et al., 2013, for a meta-analysis). However, BPT effects wane after treatment, likely due to a number of factors including reduction in parent use of skills in the context of families’ daily lives (see Lundahl et al., 2006 for a review).

Maintaining skill use after treatment may be particularly challenging for low-income families, given that financial strain and related stressors are known to increase risk for the coercive cycle and vulnerability for child problem behavior (see Masarik & Conger, 2017; Shaw & Shelleby, 2014 for reviews). Further, evidence suggests that stressors related to economic disadvantage may be confounded or exacerbated by families sociocultural experiences related to their race and ethnicity (e.g., racism, discrimination), and in turn, differentially impact families ability to experience positive outcomes when engaging in BPT. Indeed, studies suggest that in comparison to White, non-Latinx populations, racial and ethnic minority families may not have the opportunity to equally benefit from BPT in part, due to their disproportionate experiences of socioeconomic disadvantage and parental stress, contributing to lower engagement and completion rates (Lavigne et al., 2010; McCabe et al., 2020). Additionally, financial strain also likely competes with caregivers’ ability to effectively use BPT skills after treatment completion when faced with disproportionate adversity and daily stressors (see Jones et al., 2013; Lundahl et al., 2006 for reviews).

Consistent with efforts to increase in- and out-of-session support to low-income families throughout BPT (e.g., Chacko et al., 2012; Williams et al., 2007), the field must also consider how to help low-income families maintain treatment gains at follow-ups. One such approach capitalizes on the rise of telehealth generally and the uptake of mobile phones among low-income consumers in particular (e.g., Georgeson et al., in press; Jones et al., 2013; Lindhiem et al., 2015). Telehealth broadly refers to the use of technology as an adjunct to or replacement for standard health services with the goal of increasing service accessibility, engagement, and/or outcomes. Although telehealth has received increased focus due to the COVID-19 pandemic and associated social distancing mandates, technology ranging from the relatively early and basic (e.g., videotape modeling) to those more recent and sophisticated (e.g., remote coaching, internet-delivered sessions) has long been rooted in the literature on BPT (e.g., Chacko et al., 2016; Ortiz et al., 2020; Jones et al., 2013, for a review). Consistent with national trends, low-income homes are cutting the cord on landlines; however, they are more likely than high-income families to rely on mobile phones as the primary and often only digital device in the home (e.g., Blumberg & Luke, 2018; Pew Research Center, 2019). This trend makes sense, given that mobile phones can cost-effectively increase low-income parents’ ability to use a range of features (e.g., text/email, electronic/shared calendars) essential to parenting children in the 21st century.

Building upon these trends, this study examined parent skill use and child behavior at post-treatment, as well as 3- and 6-month follow-up, in 101 low-income families randomized to Helping the Noncompliant Child (HNC; McMahon & Forehand, 2003) or to Technology-Enhanced HNC (TE-HNC). To date, results across two TE-HNC trials suggest the potential to cost-effectively and efficiently (i.e., less time to master program skills) improve aspects of low-income families’ engagement in BPT without compromising parent satisfaction with services (Jones et al., 2014; 2021). Effect sizes in preliminary work also suggested greater TE-HNC improvement in child behavior at post-treatment (Jones et al., 2014). The current study extended this pilot with a larger sample, longer term follow-up, and more rigorous statistical methods to evaluate post-treatment and follow-up effects for parent skill use and child behavior in this larger randomized controlled trial. We hypothesized that technology enhancements (TE-HNC) would help families maintain treatment gains for parenting and child behaviors resulting in superior long-term outcomes as compared to standard care as usual (HNC).

Methods

Participants

This study included 101 children (3–8 years old) from families with low-income (<250% of the Federal Poverty Guidelines) and their primary caregivers (see Table 1). The majority of parents identified as White (68.3%) or Black/African American (21.8%) with an additional 1.0% as Native American and 8.9% identifying as mixed race. Further, 6.9% of parents identified as Hispanic/Latinx. Similarly, most children were White (63.4%) or Black/African American (20.8%), with 1.0% identifying as Native American and 13.9% identifying as mixed race. Further, 13.9% of children were Hispanic/Latinx. Children with clinically significant problem behaviors (Eyberg Child Behavior Inventory Problem > 15 or Intensity > 131; Eyberg & Pincus, 1999) were included. Children were excluded if they had a significant developmental and/or physical impairment that prohibited use of HNC. Families were also excluded if caregivers had a current mood, psychotic, and/or substance use disorder or a pending and/or prior substantiated child abuse/neglect case. Participants were recruited via advertisements and flyers distributed at nonprofit organizations, local schools, agencies serving low-income families, and word-of-mouth (see Khavjou et al., 2018; Khavjou et al., 2020).

Table 1.

Demographics by group at baseline.

Total Sample
(N = 101)		 HNC
(n = 54)		 TE-HNC
(n = 47)
M (SD) M (SD) M (SD)
Child
Age (years) 4.19 (1.19) 4.28 (1.17) 4.13 (1.19)
Gender (% male) 54.9% 57.41% 53.19%
Race %
 White 63.4% 57.41% 70.21%
 Black/African American 20.8% 25.93% 14.89%
 Native American/Alaskan 1.0% 0.00% 2.13%
 Multiracial 13.9% 14.81% 12.77%
 Not Reported 1.0% 1.85% 0.00%
Hispanic/Latinx 13.9% 14.81% 12.77%
Parent
Age (years) 31.66 (6.72) 32.50 (6.12) 31.34 (5.88)
Gender (% female) 97.06% 98.15% 97.87%
Race %
 White 68.3% 62.96% 74.47%
 Black/African American 21.8% 27.78% 14.89%
 Native American/Alaskan 1.0% 0.00% 2.13%
 Multiracial 7.9% 7.41% 8.51%
 Not Reported 1.0% 1.85% 0.00%
Hispanic/Latinx 6.9% 7.41% 6.38%
Marital Status
 Single 24.51% 27.78% 21.28%
 Married/living together 61.76% 59.26% 63.83%
 Divorced/separated 13.73% 12.96% 14.80%
Employed in Any Capacity 46.08% 50.00% 40.43
Smartphone owner 94.1% 92.6% 95.7%
 iPhone 44.2% 38.0% 51.1%
 Android 54.7% 62.0% 46.7%
 Other 1.1% 0% 2.2%
Open in a new tab

Procedure

Families completed a phone screen and baseline assessment at a community-based clinic to confirm eligibility and provide consent for their family’s participation. Eligible families were then randomized 1:1 to either HNC or TE-HNC. Families were compensated $50 per assessment for completing the baseline, post-, 3-, and 6-month follow-up assessments. All procedures were approved by the university’s institutional review board.

Intervention

All families received HNC, which is a therapist-delivered, criteria-based (i.e., therapists conduct weekly observation and coding of skill use to determine progression through skills and program completion) BPT intervention for children with behavior disorders. HNC included weekly face-to-face therapy sessions (60 minutes), as well as a brief midweek phone check-in. HNC consists of two phases: Differential Attention (e.g., increasing positive attention, ignoring inappropriate behavior) and Compliance Training (e.g., utilizing time outs). When parents progress to Phase II (i.e., Compliance Training), they continue to practice Phase I skills to maintain skill proficiency.

Families in the TE-HNC group received the full HNC protocol (McMahon & Forehand, 2003) augmented with a digital companion, Tantrum Tamers©. Tantrum Tamers is a HIPAA-compliant, interactive system that allowed therapists to monitor caregiver activity on the mobile application, as well as tailor the focus and pace of treatment based on parent practice and progress. The Tantrum Tamers application included: 1) daily surveys of skills practice, 2) weekly video-recorded home practice, 3) daily text reminders for skill practice and appointments; 4) video calls with the family midweek to problem solve obstacles; and 5) skills video series to model new skills and share with other caregivers (see Jones et al., 2021 for more information). Additionally, a homework checklist was added to remind caregivers of daily and weekly assignments. TE-HNC families had access to all app functionality during treatment. After families completed treatment, they had access to a limited range of content, including the skills video series and surveys with automated feedback through their 3-month follow-up assessment when they returned their study phones. From 3- to 6-month follow-ups, TE-HNC parents had access to a program blog, which provided content intended to remind caregivers of program content.

Therapist Training and Fidelity

Master’s-level therapists treated families in both groups. Training included reviewing treatment manuals, establishing reliability with the HNC coding system, role-play and session observations and discussion, weekly observation, and supervision and feedback by two licensed clinical psychologists. Approximately a quarter of sessions (24%) were coded by one Master’s-level coder to ensure treatment fidelity (97% fidelity); 72% of those were double coded (90% reliability between coders). In addition, 35% of sessions were coded for therapist competence by at least one doctoral-level coder and of those 22% were coded by a second doctoral-level coder, yielding an average competence rating of 97%.

Measures

Demographics.

Caregivers reported their and their child’s demographic information at baseline, including, age, race/ethnicity, marital status, education level, and income.

Parent-reported child problem behavior.

At all waves caregivers completed the Eyberg Child Behavior Inventory (ECBI; Eyberg & Robinson, 1983). The ECBI is a 36-item caregiver-report inventory measuring common disruptive behavior problems in youth ages 2–16. The ECBI has two scales: 1) the Intensity Scale, which measures the frequency (1 = never, 7 = always) with which the child engages in each of the 36 behaviors (Range 36–252; ≥ 131 clinically significant), and 2) the Problem Scale, which asks caregivers to indicate whether each of the 36 behaviors is “a problem for you” (yes or no; Range 0–36; ≥ 15 clinically significant). The ECBI is sensitive to BPT interventions (e.g., Nixon et al., 2003; Webster-Stratton & Hammond, 1997) and has demonstrated internal consistency and test-retest reliability (Eyberg & Robinson, 1983; Funderburk et al., 2003). In the current study, the omega reliability coefficients ranged from .88 to .93 for the Intensity Scale and .82 to .92 for the Problem Scale across waves.

Observed parenting and child compliance.

Parent-child observations were conducted at all waves. Coders received approximately 50 hours of training in the Behavioral Observation Coding System (McMahon & Forehand, 2003) and reached at least 80% agreement on one or more of the coded behaviors with expert coders on a series of training videos. Half of the videos were double-coded for fidelity. When two coders failed to reach 80% agreement, they jointly code the observation to resolve discrepancies. Behaviors are reported at a rate per minute during a 5-minute observation period to account for variability in interaction length. Attends was defined as positive attention in which the parent provides an ongoing verbal description of what the child is doing and Rewards was defined as positive attention that is provided following the child’s appropriate behavior) and these were combined for a single average score. Questions (an interrogation to which the only appropriate response is verbal) and Instructions (parent-issued command toward the child), were also measured by the average number of caregiver Questions and Instructions per minute. Finally, child Compliance was measured as the percentage of all parent Clear Instructions the child complied with within 5 seconds after a command was issued.

Data analytic plan

Managing missing data and intent-to-treat analysis approach.

Prior to conducting analyses, the pattern of missingness across follow-ups was examined to determine if data were missing at random. Multiple imputation was conducted for use with descriptive and group mean statistics. Twenty imputed datasets were created by group using Blimp 2.2.3 software (Keller & Enders, 2019) using a fully conditional specification and Gibbs sampler Markov chain Monte Carlo (MCMC) algorithm. MI diagnostics were conducted to ensure that the potential scale reduction values were less than 1.05 (Gelman & Rubin, 1992). For primary group difference analyses, full information maximum likelihood (FIML) estimation techniques were used for inclusion of all available data based on intent-to-treat guidelines.

Treatment effect sizes.

To examine effect sizes, Cohen’s d was used to compare the magnitude of the experimental effects within each treatment and across groups (Cohen, 1988). Within-group effect sizes were used to examine magnitude in change in child problem behaviors and observed parenting comparing pre-post-treatment, pre-3-month follow up, and pre-6-month follow up, controlling for the correlation between each time point, for each group. To test the clinical significance of treatment effects, we utilized the Reliable Change Index (RCI; Jacobson & Truax, 1991), which suggests that effects attributable to treatment are most likely not due to chance, with a value of 1.96 or greater indicating the effect is reliable.

Comparing treatment conditions.

For primary analyses, we employed latent curve modeling (Preacher et al., 2008) to examine differences in the trajectory of change during treatment, maintenance of gains, and outcomes at the 6-month follow-up. Given expected non-linear patterns of change, we used a spline growth model with a treatment slope (pre- to post-treatment), a maintenance slope (from post through 3- and 6-month follow-ups), and intercept coded as the mean level at the 6-month follow-up. Growth curve analyses were conducted using Mplus 8.2 software (Muthen & Muthen, 2017) and ML estimation with robust standard errors. FIML techniques were used for inclusion of all available data. Primary outcomes were child behavior problems and secondary outcomes were parenting skills.

Results

Figure 1 depicts the CONSORT diagram including participant retention at each follow-up by group. No significant condition differences were observed across demographics. Overall, rates of retention by group were similar across follow-ups. Patterns of missingness did not significantly differ by the following: treatment condition; child age, sex, race/ethnicity; caregiver age, ethnicity/race; nor family economic stress, all ps > .05. Further, random patterns of missingness along with a non-significant Little’s missing completely at random (MCAR) test, χ2 (239) = 267.64, p > .05, suggest that the mechanism of missingness was MCAR and support use of multiple imputation and FIML for primary analyses.

Figure 1.

Figure 1

Open in a new tab

CONSORT Diagram.

Treatment effect sizes

Child problem behavior.

Examining within-group improvements, HNC and TE-HNC both demonstrated large effects sizes from pre-post-treatment, pre-3-month follow up, and pre-6-month follow-up (see Table 2). Further, HNC and TE-HNC both demonstrated clinically significant treatment effects (RCIs > 1.96), indicating improvements.

Table 2.

Means, SDs, and effect sizes for parent and child outcomes.

Within Group Between Group
Baseline Post 3-Month 6-Month Pre- Post Pre-3 Month Pre-6 Month Post 3-Month 6-Month
M (SD) M (SD) M (SD) M (SD) Cohen’s d
ECBI Intensity (P)
 HNC 155.4 (27.9) 110.25 (23.6) 115.6 (26.9) 118.35 (35.39) 1.74 1.44 1.16 .27 .56 .48
 TE-HNC 150.7 (29.9) 103.84 (23.7) 101.3 (24.2) 102.37 (30.10) 1.72 1.80 1.60
% below clinical cutoff
 HNC 22.22% 77.78% 70.37% 62.96%
 TE-HNC 23.40% 89.36% 89.36% 85.10%
ECBI Problem (P)
 HNC 23.52 (6.47) 13.65 (6.6) 13.29 (9.6) 13.72 (10.8) 1.50 1.24 1.09 .17 .41 .37
 TE-HNC 22.48 (5.76) 12.39 (7.9) 9.72 (7.6) 10.01 (9.07) 1.42 1.85 1.60
% below clinical cutoff
 HNC 3.70% 50.76% 57.03% 56.01%
 TE-HNC 4.47% 68.81% 76.92% 67.66%
Questions + Instructions
 HNC 3.97 (2.01) .78 (.98) 1.17 (1.58) 1.05 (2.38) 2.00 1.54 1.32 .23 .44 .23
 TE-HNC 3.85 (2.00) .56 (.87) .58 (.93) .59 (1.49) 2.12 2.08 1.83
Attends + Rewards (O)
 HNC 1.09 (.91) 6.69 (3.16) 5.96 (4.43) 4.94 (3.10) 2.39 1.51 1.67 .25 .17 .60
 TE-HNC 1.18 (1.19) 7.56 (3.72) 6.67 (3.77) 7.57 (5.50) 2.29 1.95 1.59
Child Compliance (O)
 HNC .34 (.21) .52 (.26) .51 (.28) .43 (.31) .76 .68 .34 .30 .45 .59
 TE-HNC .33 (.24) .60 (.27) .65 (.34) .63 (.36) 1.05 1.08 .97
Open in a new tab

Note. ECBI = Eyberg Child Behavior Inventory, P = parent report, O = observation

Parenting and child compliance.

Examining within-group effects, both treatments demonstrated large effect sizes from pre-post-treatment, pre-3-month follow up, and pre-6-month follow-up for observed Questions + Instructions and Attends + Rewards. Results indicate that both treatments were successful in reducing the number of questions and instructions, while improving parents’ attending and rewards at the end of treatment with sustained effects over 6 months. Both treatments also improved children’s observed compliance.

Treatment comparisons

Parent-reported child problem behavior.

Parameter estimates for each outcome are shown in Table 3. Model fit for the ECBI Intensity, χ2 (5) = 2.02, p = .846, RMSEA = .000 [90% CI .000, .078], CFI = 1, SRMR = .053, and Problem, χ2 (5) = 7.86, p = .175, RMSEA = .073 [000, .169], CFI = .971, SRMR = .074, models had adequate to excellent fit. As is depicted in Figure 2A, the frequency of parent-reported behavior problems (i.e., intensity scale) demonstrated similar rates of change during the active treatment phase across conditions. However, TE-HNC had superior maintenance of gains resulting in statistically significant group differences at the 6-month follow-up for ECBI Intensity. Further supporting this difference, 85.10% TE-HNC participants were below the clinical cut-off compared to 62.96% of those in HNC at the 6-month follow-up (see Table 2). Results for the ECBI Problem scale (Figure 2B) showed a similar pattern of results, but with group differences only approaching statistical significance. TE-HNC had 10% more children within the normative range, at the 6-month follow-up, compared to HNC. Interpretation of this group difference supports a clinically meaningful difference between conditions for the ECBI Problems outcome.

Table 3.

Model results for parenting and child outcomes.

ECBI Intensity ECBI Problems Compliance Attends + Rewards Questions + Instructions
b (SE) p b (SE) p b (SE) p b (SE) p b (SE) p
Condition - 6M Intercept −15.54 (6.59) .018 −3.58 (2.00) .074 .203 (.063) .001 2.09 (.731) .004 −.484 (.324) .136
Condition - Treatment Slope .575 (8.06) .943 .003 (1.97) .999 .008 (.088) .927 −.10 (.910) .913 −.114 (.483) .813
Condition - Maintenance Slope −3.72 (2.86) .194 −.820 (.822) .319 .066 (.035) .061 .700 (.394) .075 −.081 (.159) .609
6M Mean 117.71 (4.68) .000 13.14 (1.46) .000 .424 (.041) .000 4.91 (.370) .000 1.14 (.259) .000
Treatment Slope Mean −47.14 (4.94) .000 −9.55 (1.39) .000 .246 (.070) .000 6.49 (.609) .000 −3.26 (.317) .000
Maintenance Mean 3.14 (2.12) .139 −.276 (.673) .682 −.052 (.029) .074 −.889 (.24) .000 .144 (.118) .222
Open in a new tab

Note. Condition = HNC (coded 0) vs. TE-HNC (coded 1), 6M = 6-Month Follow-up, ECBI = Eyberg Child Behavior Inventory.

Figure 2.

Figure 2

Open in a new tab

Mean plots from growth curve model results of disruptive behavior and parenting outcomes.

Observed parenting and child compliance.

Parameter estimates for each model are shown in Table 3. Model fit for observed Attends + Rewards, χ2 (5) = 5.21, p = .391, RMSEA = .021 [000, .145], CFI = .995, SRMR = .081, and Questions + Instructions, χ2 (5) = 8.91, p = .113, RMSEA = .090 [.000, .185], CFI = 860, SRMR = .079, demonstrated excellent to adequate fit. Regarding child compliance, we fixed the residual variance of maintenance slope to zero due to a negative residual variance in the initial model. After this adjustment, the model fit was adequate and without errors, χ2 (7) = 7.34, p = .395, RMSEA = .023 [000, .131], CFI = .986, SRMR = .091. Results depicted in Figure 2C support group differences on observed compliance with a statistically significant difference between conditions. TE-HNC demonstrated 20% higher rates of observed compliance compared to HNC. Regarding observed Attends + Rewards (see Figure 2D), a similar pattern of change emerged in initial pre-to-post improvements in parenting, with improved maintenance of positive parenting by TE-HNC (see Table 3). Specifically, TE-HNC showed approximately 50% greater rates of observed positive parenting behaviors during a structured task at the 6-month follow-up. Lastly, Questions + Instructions showed similar rates of initial treatment change and maintenance of gains across groups.

Discussion

This study examined the capacity for TE-HNC to improve parenting skills and child behavior at post-treatment and follow-up relative to standard HNC. Consistent with our pilot, effect sizes suggest families in both groups experience clinically significant gains in parenting skills and child behavior. Between group effect sizes favored TE-HNC, which also resulted in a greater percentage of families below clinical cut-offs. However, primary analyses suggested technology enhancements had similar initial treatment gains as standard care with differences beginning to emerge in the long-term maintenance phase and becoming statistically significant at the 6-month follow-up. Specifically, at the 6-month follow-up, TE-HNC demonstrated lower levels of parent-reported and observed child disruptive behavior and higher levels of observed positive parenting skills relative to standard HNC.

The pattern observed in the current study – similar initial improvements but superior long-term maintenance – may make both empirical and clinical sense. That is, HNC is a criteria-based BPT program, which means parents do not move on with each subsequent skill until they demonstrate proficiency of the previous skill. Further, they do not complete the program until they demonstrate proficiency of all Phase I and II skills. Given that progression through treatment is determined by weekly coding of parent and child behavior, progression and completion of HNC suggests there has been clinically significant change in parenting skills and child behavior. Thus, criteria-based programs like HNC may limit significant initial differences across groups at post-treatment based on floor or ceiling effects. TE-HNC caregivers did, however, sustain greater improvements in parenting skills across 3- and 6-month follow-ups at a higher level than HNC families. Specifically, TE-HNC families had higher rates of observed positive parenting and compliance and caregivers reported a lower frequency and intensity of child behavior problems by the 6- month follow-up compared to HNC.

One possible reason for the maintained gains in TE-HNC is that parents experienced increased support from the program and therapist throughout treatment (e.g., tailored feedback via surveys and mid-week calls, etc.). For example, TE-HNC participants had better mid-week call participation and homework compliance as well as rating the usefulness of the overall program higher than HNC participants (Jones et al., 2021). In addition, TE-HNC parents had ongoing access to program content through the 3-month (i.e., Tantrum Tamers app) and 6-month (skills blog) follow-ups, providing additional, although relatively low-level, support. Thus, if parenting is the primary mechanism of change that allows parents to preserve gains, then ongoing technology-supported practice and use of those skills at home is likely to maintain improvements in child behavior as well. In particular, technology-enhancements may have resulted in a continued and deeper learning of program theory and skills and, in turn, greater maintenance of treatment gains (Anton & Jones, 2019). As low-income families are increasing the use of mobile phones (94.1% in the current study), yet still face barriers accessing mental health care (e.g., transportation), such technology enhancements may represent a viable option to increase dissemination and implementation in community settings. Further, as reported in Jones et al. (2021), families in TE-HNC completed the program in fewer weeks (M = 11.63) than HNC (M = 14.15, p ≤. 05), which offset any additional program costs. Additionally, this increased efficiency did not come at the expense of consumer satisfaction which was equivalent across groups. Together, the current study results and that of Jones et al. (2021) suggest the promise of TE-HNC as a cost-effective approach that increases engagement and delivery efficiency while also achieving improved maintenance of effects for some parenting and child externalizing outcomes.

As with all research this study has limitations. First, we included 3- and 6-month, but not more distal follow-ups. That said, research suggests that BPT outcomes at 6-months are comparable to 12-months (see de Graaf et al., 2008 for meta-analysis). In addition, we provided all families in TE-HNC with phones and service plans through the 3-month follow-up, which would not be feasible in front-line service settings. Given data suggesting 44% of low-income users let service plans lapse due to finances (Pew Research Center, 2015), ongoing work in this area will have to consider strategies to increase the sustainability of this approach. It was also true that the vast majority of our families already had their own mobile phone (94.1%). Therefore, supplying and requiring them to use a program phone may have actually been burdensome. Given that our budget precluded paying for phones and plans beyond the 3-month assessment, we provided families with content via a blog. Although it highlights that effects were maintained even after phones were returned and access to the app ended, it also introduces additional variability between the 3- and 6-month assessments. However, the current study did not systematically track post-treatment access to the app (prior to the 3-month) or other content via the blog, which limited our ability to fully understand the mechanisms underlying greater maintenance of treatment effects for the TE-HNC group. Future studies would benefit from systematically tracking both active treatment and post-treatment engagement in technology-based supports to inform further refinements aimed at long-term skills maintenance. For example, Jent and colleagues (2021) found that an eBook enhanced BPT program did not result in added benefit for parenting outcomes at post-treatment or follow-up. Thus, it may be that technology enhancements that involve therapist interaction during active treatment may be more effective methods for continued and deeper learning of program theory and skills and, in turn, greater maintenance of treatment gains compared to lighter technology-only enhancements (e.g., skill videos). Finally, future research with large sample sizes should explore the potential differential impact of technology-enhanced BPT on long-term maintenance of effects across caregiver race, ethnicity, and education within low-income populations.

Despite such limitations, this is the first study of its kind to show improved follow-up effects with a technology-enhanced BPT model. Given the common elements characterizing this family of interventions, findings should generalize to other BPT programs as well. In addition, we focused on low-income families who are at-risk for early-onset DBDs, yet experience more obstacles to engaging and completing services. Thus, while low-income families are particularly vulnerable, they are often not the focus of intervention research. Lastly, much policy attention and funding has been devoted to the potential for technology to increase engagement in children’s mental health. Empirical research on technology-enhanced treatment models, however, primarily reflects pilot studies limiting conclusions that can be drawn about the efficacy of and next steps in this approach (see Georgeson et al., 2020 for a review). The current study moved beyond earlier pilot studies through a larger sample and long-term follow-ups which enhances confidence in and potential for generalizability of findings.

In summary, our findings contribute to an ongoing line of work suggesting that technology-enhanced treatment models hold promise for increasing markers of engagement in BPT (Jones et al., 2021) and sustaining long-term outcomes. As progress is made in intervention research using artificial intelligence and machine learning (e.g., Cui & Gong, 2018; Timmons et al., 2019), future work must consider if additional just-in-time responsivity in the context of the families’ daily life can further bolster effects. While financial strain is a chronic stressor for low-income families, there is also variability within and between parents in terms of their day-to-day experience of ups (e.g., new job, unexpected savings) and downs (e.g., job loss, unexpected bill). A technology-enhanced treatment model that is more responsive to variability in those emergent stressors (e.g., Guan, Park, & Chorpita, 2019; Lind et al., 2020), as well as its impact on day-to-day parenting and skill use, is likely critical to achieving and preserving treatment gains.

Key points.

  • Behavioral parent training (BPT) is the standard of care for early onset disruptive behavior disorders, but waning treatment effects remain a challenge.

  • An RCT compared a standard BPT program, Helping the Noncompliant Child (HNC), to Technology-Enhanced HNC (TE-HNC) in a clinical sample on parent proficiency of program skills and child behavior at post-treatment and 3- and 6-month follow-up.

  • Both programs yielded improvements in parent skill proficiency and child problem at post-treatment, but TE-HNC families evidenced greater maintenance of some treatment gains at follow-up.

  • The findings suggest that a technology-enhanced treatment model may allow for deeper learning of parent skills and improvement in child behavior.

Acknowledgments

This study was funded by the National Institute of Mental Health (NIMH) R01MH100377 (D.J.J.; ClinicalTrials. gov Identifier: NCT02191956). Other support was provided by NIMH R21MH113887 (D.J.J.; ClinicalTrials.gov Identifier: NCT03597789), the National Institute of Child Health and Human Development (R.L.; T32HD007377), the National Science Foundation (A.H.; DGE-1650116), the National Institute of Minority Health and Health Disparities (J.P., PI; R01MD015401), and the National Institute on Drug Abuse (M.H.; T32DA043449). The authors are grateful to the families, assessors, and therapists who participated in this project for their time and valuable contributions. The authors have declared that they have no competing or potential conflicts of interest.

Abbreviations:

DBD

Disruptive Behavior Disorders

BPT

Behavioral Parent Training

HNC

Helping the Noncompliant Child (HNC)

TE-HNC

Technology-Enhanced HNC (TE-HNC)

Footnotes

Conflict of interest statement: No conflicts declared.

Ethical Considerations - Written consent from the subjects was obtained

References

  1. Anton MT, & Jones DJ (2019). Parent-therapist alliance and technology use in behavioral parent training: A brief report. Psychological services, 16(2), 260–265. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Blumberg SJ, & Luke JV (2018). “Wireless Substitution: Early Release of Estimates From the National Health Interview Survey, January-June 2018.” US Department of Health and Human Services, Centers for Disease Control and Prevention. Available at: https://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201812.pdf [Google Scholar]
  3. Chacko A, Isham A, Cleek AF, & McKay MM (2016). Using mobile health technology to improve behavioral skill implementation through homework in evidence-based parenting intervention for disruptive behavior disorders in youth: study protocol for intervention development and evaluation. Pilot Feasibility Studies, 2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Chacko A, Wymbs BT, Chimiklis A, Wymbs FA, & Pelham WE (2012). Evaluating a comprehensive strategy to improve engagement to group-based behavioral parent training for high-risk families of children with ADHD. Journal of Abnormal Child Psychology, 40, 1351–62. doi: 10.1007/s10802-012-9666-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Chorpita BF, Daleiden EL, Ebesutani C, Young J, Becker KD, Nakamura BJ, & Starace N (2011). Evidence-based treatments for children and adolescents: An updated review of indicators of efficacy and effectiveness. Clinical Psychology: Science and Practice, 18, 154–172. doi: 10.1111/j.1468-2850.2011.01247.x [DOI] [Google Scholar]
  6. Cohen J (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers. [Google Scholar]
  7. Cohen MA, & Piquero AR (2009). New evidence on the monetary value of saving a high risk youth. Journal of Quantitative Criminology, 25, 25–49. [Google Scholar]
  8. Comer JS, Chow C, Chan PT, Cooper-Vince C, & Wilson LA (2013). Psychosocial treatment efficacy for disruptive behavior problems in very young children: a meta-analytic examination. Journal of the American Academy of Child and Adolescent Psychiatry, 52(1), 26–36. 10.1016/j.jaac.2012.10.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cui Z & Gong G, (2018). The effect of machine learning regression algorithms and sample size on individualized behavioral prediction with functional connectivity features. NeuroImage, 178, 622–637. [DOI] [PubMed] [Google Scholar]
  10. de Graaf I, Speetjens P, Smit F, de Wolff M, & Tavecchio L (2008). Effectiveness of the Triple P Positive Parenting Program on behavioral problems in children: a meta-analysis. Behavior Modification, 32, 714–35. doi: 10.1177/0145445508317134. [DOI] [PubMed] [Google Scholar]
  11. Eyberg SM, & Pincus D (1999). Eyberg Child Behavior Inventory and Sutter-Eyberg Behavior Inventory-Revised: Professional Manual. Odessa, FL: Psychological Assessment Resources. [Google Scholar]
  12. Eyberg SM, & Robinson EA (1983). Conduct problem behavior: Standardization of a behavioral rating scale with adolescents. Journal of Clinical Child Psychology, 12, 347–354. [Google Scholar]
  13. Fergusson DM, Horwood LJ, & Ridder EM (2005). Show me the child at seven: the consequences of conduct problems in childhood for psychosocial functioning in adulthood. Journal of Child Psychology and Psychiatry, 46, 837–49. [DOI] [PubMed] [Google Scholar]
  14. Funderburk BW, Eyberg SM, Rich BA, & Behar L (2003). Further psychometric evaluation of the Eyberg and Behar rating scales for parents and teachers of preschoolers. Early Education and Development, 14, 67–81. [Google Scholar]
  15. Gelman A, & Rubin DB (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–511. [Google Scholar]
  16. Georgeson AR, Highlander A, Loiselle R, Zachary C, & Jones DJ (2020). Engagement in technology-enhanced interventions for children and adolescents: Current status and recommendations for moving forward. Clinical Psychology Review, 78:101858. doi: 10.1016/j.cpr.2020.101858. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Guan K, Park AL, & Chorpita BF (2019). Emergent life events during youth evidence-based treatment: Impact on future provider adherence and clinical progress. Journal of clinical child and adolescent psychology, 8(S1), S202–S214 10.1080/15374416.2017.1295382 [DOI] [PubMed] [Google Scholar]
  18. Jacobson NS, & Truax P (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19. [DOI] [PubMed] [Google Scholar]
  19. Jones DJ, Forehand R, Cuellar J, Kincaid C, Parent J, Fenton N, & Goodrum N (2013). Harnessing innovative technologies to advance children’s mental health: Behavioral parent training as an example. Clinical Psychology Review, 33, 241–52. doi: 10.1016/j.cpr.2012.11.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Jones DJ, Forehand R, Cuellar J, Parent J, Honeycutt A, Khavjou O, Gonzalez M, Anton M, & Newey GA (2014). Technology-enhanced program for child disruptive behavior disorders: development and pilot randomized control trial. Journal of Clinical Child and Adolescent Psychology, 43, 88–101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Jones DJ, Forehand R, McKee LG, Cuellar J, & Kincaid C (2010). Behavioral parent training: Is there an “app” for that? Behavior Therapist, 33, 72–77. [PMC free article] [PubMed] [Google Scholar]
  22. Jones DJ, Loiselle R, Zachary C, Georgeson AR, Highlander A, … & Forehand R (2021). Optimizing engagement in behavioral parent training: progress toward a technology-enhanced treatment model. Behavior therapy, 52(2), 508–521. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kaminski JW, & Claussen AH (2017). Evidence base update for psychosocial treatments for disruptive behaviors in children. Journal of Clinical Child & Adolescent Psychology, 46(4), 477–499. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Khavjou O, Forehand R, Loiselle R, Turner P, Buell N, & Jones DJ (2020). Helping the noncompliant child: An updated assessment of program costs and cost-effectiveness. Children and Youth Services Review, 114, [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Khavjou OA, Turner P, & Jones DJ (2018). Cost effectiveness of strategies for recruiting low-income families for behavioral parent training. Journal of Child and Family Studies, 27, 1950–1956. 10.1007/s10826-017-0997-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Keller BT, & Enders CK (2019). Blimp user’s guide (version 2.0). Los Angeles, CA. [Google Scholar]
  27. Lavigne JV, Lebailly SA, Gouze KR, Binns HJ, Keller J, & Pate L (2010). Predictors and correlates of completing behavioral parent training for the treatment of oppositional defiant disorder in pediatric primary care. Behavior Therapy, 41, 198–211. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Leijten P, Raaijmakers MA, de Castro BO, & Matthys W (2013). Does socioeconomic status matter? A meta-analysis on parent training effectiveness for disruptive child behavior. Journal of Clinical Child and Adolescent Psychology, 42, 384–92. doi: [DOI] [PubMed] [Google Scholar]
  29. Lindhiem O, Bennett CB, Rosen D, & Silk J (2015). Mobile technology boosts the effectiveness of psychotherapy and behavioral interventions: a meta-analysis. Behavior Modification, 39, 785–804. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Lind T, Lau AS, Gomez C, Rodriguez A, Guan K, Chlebowski C, Zhang A, Chorpita B & Brookman-Frazee L (2020). Emergent life events in the delivery of a caregiver-mediated evidence-based intervention for children with autism spectrum disorder in publicly funded mental health services. Autism, 24(5), 1286–1299. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Lundahl B, Risser HJ, & Lovejoy MC (2006). A meta-analysis of parent training: moderators and follow-up effects. Clinical Psychology Review, 26, 86–104. doi: [DOI] [PubMed] [Google Scholar]
  32. Masarik AS, & Conger RD (2017). Stress and child development: A review of the family stress model. Current Opinion in Psychology, 13, 85–90. [DOI] [PubMed] [Google Scholar]
  33. McCabe KM, Yeh M, & Zerr AA (2020). Personalizing behavioral parent training interventions to improve treatment engagement and outcomes for culturally diverse families. Psychology Research and Behavior Management, 13, 41–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. McMahon RJ, & Forehand RL (2003). Helping the Noncompliant Child: Family-based Treatment for Oppositional Behavior (2nd ed.). Guilford Press. [Google Scholar]
  35. Merikangas KR, Nakamura EF, & Kessler RC (2009). Epidemiology of mental disorders in children and adolescents. Dialogues in Clinical Neuroscience, 1, 7–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Muthén LK, & Muthén BO (2017). Mplus: Statistical Analysis with Latent Variables: User’s Guide (Version 8). Los Angeles, CA: Authors. [Google Scholar]
  37. Nixon RDV, Sweeney L, Erickson DB, & Touyz SW (2003). Parent-child interaction therapy: A comparison of standard and abbreviated treatments for oppositional defiant preschoolers. Journal of Consulting and Clinical Psychology, 71, 251–260. [DOI] [PubMed] [Google Scholar]
  38. Odgers CL, Caspi A, Broadbent JM, Dickson N, Hancox RJ, Harrington H, … & Moffitt TE (2007). Prediction of differential adult health burden by conduct problem subtypes in males. Archives of General Psychiatry, 64, 476–84. [DOI] [PubMed] [Google Scholar]
  39. Ortiz C, Vidair HB, Acri M, Chacko A, & Kobak K (2020). Pilot study of an online parent-training course for disruptive behavior with live remote coaching for practitioners. Professional Psychology: Research and Practice, 51, 125–133. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Pew Research Center (2019). Mobile Fact Sheet. Available online http://www.pewinternet.org/fact-sheet/mobile/
  41. Piquero AR, Jennings WG, Diamond B, Farrington DP, Tremblay RE, Welsh BC, & Gonzalez JMR (2016). A meta-analysis update on the effects of early family/parent training programs on antisocial behavior and delinquency. Journal of Experimental Criminology, 12, 229–248. 10.1007/s11292-016-9256-0 [DOI] [Google Scholar]
  42. Polanczyk GV, Salum GA, Sugaya LS, Caye A, & Rohde LA (2015). Annual research review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. Journal of Child Psychology and Psychiatry, 56, 345–65. [DOI] [PubMed] [Google Scholar]
  43. Preacher KJ, Wichman AL, MacCallum RC, & Briggs NE (2008). Latent growth curve modeling (No. 157). Sage. [Google Scholar]
  44. Shaw DS, & Shelleby EC (2014). Early-starting conduct problems: Intersection of conduct problems and poverty. Annual Review of Clinical Psychology, 10, 503–528. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Timmons AC, Han SC, Chaspari T, Kim Y, Pettit C, Narayanan S, & Margolin G (2019). Family-of-origin aggression, dating aggression, and physiological stress reactivity in daily life. Physiology & behavior, 206, 85–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Webster-Stratton C, & Hammond M (1997). Treating children with early-onset conduct problems: A comparison of child and parent training interventions. Journal of Consulting and Clinical Psychology, 65, 93–109. 10.1037/0022-006X.65.1.93 [DOI] [PubMed] [Google Scholar]
  47. Williams GC, Lynch M, & Glasgow RE (2007). Computer-assisted intervention improves patient-centered diabetes care by increasing autonomy support. Health Psychology, 26, 728–734. doi: 10.1037/0278-6133.26.6.728. [DOI] [PubMed] [Google Scholar]

RESOURCES