Skip to main content
NCBI home page
Sage Choice logoLink to Sage Choice
. 2023 Apr 22;47(5):1094–1114. doi: 10.1177/01454455231165937

Technology-Enhanced Behavioral Parent Training: The Relationship Between Technology Use and Efficiency of Service Delivery

Madison P McCall 1,✉, Margaret T Anton 2, April Highlander 1, Raelyn Loiselle 1,3, Rex Forehand 4, Olga Khavjou 5, Deborah J Jones 1
PMCID: PMC10403959  PMID: 37086169

Abstract

Behavior disorders (BDs) are common and, without treatment, can have long-term impacts on child and family health. Behavioral Parent Training (BPT) is the standard of care intervention for early-onset BDs; however, structural socioeconomic barriers hinder treatment outcomes for low-income families. While digital technologies have been proposed as a mechanism to improve engagement in BPT, research exploring the relationship between technology use and outcomes is lacking. Thus, this study with 34 low-income families examined the impact of parents’ use of adjunctive mobile app components on treatment efficiency in one technology-enhanced (TE-) BPT program, Helping the Noncompliant Child (HNC). While parent use of the TE-HNC app and its impact on the efficiency of service delivery varied across specific components, increased app use significantly reduced the number of weeks required for families to achieve skill mastery. Implications for the design and development of behavior intervention technologies in general, as well as for BPT in particular, are discussed.

Keywords: Behavioral Parent Training, behavior disorders, technology, efficiency

Approximately 113 million children globally experience clinically elevated externalizing symptoms consistent with a diagnosis of conduct disorder and oppositional defiant disorder (i.e., behavior disorders, BDs), as well as commonly co-occurring attention-deficit/hyperactivity disorder (ADHD; Merikangas et al., 2009; Polanczyk et al., 2015). These externalizing symptoms are often characterized by frequent, long-lasting, dysregulated behavior that occurs across contexts, in conjunction with irritability, impulsivity, or aggression. Further, considerable research has documented an association between untreated symptoms in early childhood and adverse developmental outcomes, including delays in cognitive, social, and emotional functioning, which, in turn, predict long-term risk for later educational, employment, and health problems (Fergusson et al., 2005; Odgers et al., 2007; Piquero et al., 2016).

Behavioral Parent Training (BPT; also called Behavioral Parenting Interventions and Parent Management Training) is an evidence-based intervention for early-onset (3–8 years old) BDs, and its efficacy is well established (e.g., Lundahl et al., 2006; McMahon & Forehand, 2005; Piquero et al., 2009). BPT encompasses a “family” of programs that target the “coercive cycle” of parent-child interactions implicated in early-onset BDs (e.g., Kaehler et al., 2016; Long et al., 2017; Southam-Gerow & Prinstein, 2014). While BPT is the standard of care for early-onset BDs, its effectiveness is hindered by poor engagement, especially for families with low incomes who are overrepresented in statistics on BDs compared to families with relatively higher income (see Chacko, Jensen, et al., 2016; Gardner et al., 2009; Lundahl et al., 2006; Reyno & McGrath, 2006; Shaw & Taraban, 2016 for reviews). With sufficient engagement, outcomes are comparable regardless of family income; however, research suggests that insufficient engagement (e.g., compromises in commitment to and completion of treatment, such as attendance, compliance, or retention) is associated with less beneficial treatment outcomes for families in evidence-based interventions, including BPT (Gardner et al., 2009; Jones et al., 2013; Shaw & Taraban, 2016). Consistent with economic family stress theory (see Conger & Donnellan, 2007, for a review) and the poverty-related stress model (Wadsworth et al., 2011), socioeconomic disadvantage increases the likelihood that parents experience persistent financial (e.g., difficulty paying bills), work (e.g., unemployment and shiftwork), and parenting-related (e.g., lack of or low-quality child care) stressors and cope with those stressors with relatively higher levels of disengagement and withdrawal (Santiago et al., 2011). Thus, socioeconomic disadvantage and associated stressors may both exacerbate the coercive cycle and compromise engagement in BPT interventions, highlighting the critical importance of innovative strategies to decrease the number of sessions required for families to master program skills and complete services more efficiently (Cohen & Piquero, 2009; Jones et al., 2013).

Given technology-enhanced models have been proposed as a potential tool to improve engagement in and efficiency of evidence-based treatments, recent work has extended the long history of technology enhancements in BPT (e.g., video modeling) to include those targeting clinicians (e.g., online therapist training) and families (e.g., telehealth and other behavioral intervention technologies; Bausback & Bunge, 2021; Chacko, Isham, et al., 2016; Comer et al., 2015; Ortiz et al., 2020; Sullivan et al., 2021). Despite well-established disparities in access to and use of information and communication technology (i.e, “digital divide”; Van Dijk, 2020), estimates suggest that the majority (76%) of households with low income own a smartphone and approximately a quarter (27%) are smartphone-dependent (or rely on smartphone devices for online access (Pew Research Center, 2021; Vogels, 2021). Accordingly, families with relatively lower income are more likely than other groups to rely on mobile phones to access health information and as their primary means of communication (Anderson-Lewis et al., 2018; Vangeepuram et al., 2018). Building upon such trends, smartphones have the potential to cost-effectively extend BPT treatment with a range of technical features, especially in the case of mobile applications (apps), which are increasingly influential in how parents consume information about child rearing (Guerra-Reyes et al., 2016; Kubb & Foran, 2020).

By capitalizing on the increased functionality afforded by access to smartphones, Technology-Enhancements (TE-) to one BPT program, Helping the Noncompliant Child (HNC; McMahon & Forehand, 2005), were developed to improve engagement in BPT with families experiencing financial hardship (Jones et al., 2014, 2020). Consisting of a mobile app for parents and web portal for clinicians, TE-HNC was designed, in part, to increase the frequency and reliability of parenting skill use at home and in the context of the family’s daily life. Findings across two randomized control trials (RCT) suggest that TE-HNC has the potential to efficiently (i.e., fewer weeks for families to achieve skill mastery and complete treatment) improve aspects of low-income families’ engagement in BPT while achieving clinically significant gains in parenting skill use and child behavior after treatment (Jones et al., 2014, 2020; Parent et al., 2022). Further, a preliminary within-group analysis of families (n = 9) who used the TE-HNC prototype in the initial pilot RCT evidenced a relatively high level (52%–92%) of overall mobile app use during the active treatment phase and suggested the potential for adjunctive behavioral intervention technologies to improve BPT service delivery models (Anton et al., 2016). As such, this study aimed to replicate and extend those analyses with a larger sample and the updated version of the app by examining patterns in parent use of TE-HNC mobile app components and their differential impact on the efficiency of service delivery, which is critical to understanding opportunities for tailoring treatment. It was hypothesized that parent use of the mobile app components (Skills Videos, Surveys, and Videorecording Skills Practice) during treatment would be associated with the efficiency of service delivery (i.e., weeks to master program skills and complete treatment) in TE-HNC (see Methods for more detail on the mobile app).

Methods

Overview

The current study represents secondary data analysis of a randomized controlled trial with 101 low-income families, who were randomized to HNC (McMahon & Forehand, 2005; n = 54) or TE-HNC (n = 47). To be eligible for the study, families had to meet criteria for low-income (100%–250% of the federal poverty guidelines), as well as have a child between 3 and 8 years old who met clinical cut-offs for significant externalizing behavior as defined by the Problem and/or Intensity Subscales of the Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999), a parent-report measure of common child behavior problems. Exclusion criteria included a pending or prior substantiation of child abuse or neglect; current parental substance abuse/dependence, psychotic disorder, or severe depression/manic episode diagnosis; or child psychotic disorder, mood disorder with psychotic features, or developmental or other disability that would require significant adaptations to the standard treatment protocol and materials to achieve fidelity.

Participants

Participant characteristics are summarized in Table 1. The current analyses focused on families randomized to the TE-HNC group who completed treatment (n = 34). As previously reported, drop-out (27.7%) in TE-HNC was significantly lower than previously reported in BPT work with low-income families (Jones et al., 2020). On average, caregivers in this group were 31.4 years old (SD = 5.2). The majority of caregivers self-identified as female (97.1%) and a fourth self-identified as ethnic or racial minorities (26.5%). According to parent-report, children were 4.6 years old on average (SD = 1.3) and 58.8% were boys. The majority of the participants (94.1%) owned a personal smartphone and reported using Android (55.9%) or Apple iOS (38.2%) operating systems at baseline.

Table 1.

Sample Demographics.

n (%) M (SD) Range
Child age (years) 4.6 (1.3) 3.0–7.8
Child gender
 Female 14 (41.2)
 Male 20 (58.8)
Child race
 White 25 (73.5)
 Black/African American 5 (14.7)
 More than one race 4 (11.8)
Child ethnicity
 Hispanic 3 (8.8)
 Non-Hispanic 31 (91.2)
 Parent age (years) 31.4 (5.2) 24–43
Parent gender
 Female 33 (97.1)
 Male 1 (2.9)
Parent race
 White 26 (76.5)
 Black/African American 5 (14.7)
 More than one race 3 (8.8)
Parent ethnicity
 Hispanic 2 (5.9)
 Non-Hispanic 32 (94.1)
Parent relationship to child
 Biological parent 33 (97.1)
 Legal guardian 1 (2.9)
Smartphone ownership
 No smartphone 2 (5.9)
 iPhone 13 (38.2)
 Android 19 (55.9)
Open in a new tab

Procedures

Families were recruited via word-of-mouth, social media sites, and advertisements targeting local schools and community centers, as well as healthcare, social service, and other agencies that primarily serve low-income families in north central North Carolina (NC). Phone screening was conducted to determine eligibility for the study. After the initial phone screen, interested and eligible caregivers were scheduled for a baseline assessment where parents provided consent for themselves and their child, eligibility was confirmed, and a series of demographic and psychosocial information about the family was collected. Eligible caregiver-child dyads were randomized to HNC or TE-HNC during the first session. Given data that suggests 44% of low-income users let service plans lapse due to finances (Smith, 2015), caregivers randomized to TE-HNC received an iPhone device with the TE-HNC mobile app (i.e., Tantrum Tamers) pre-downloaded for the duration of the study to ensure consistent access to the app. All families received $50 per assessment and TE-HNC families received an additional $100 for returning the project phone. All procedures were approved by the university’s institutional review board.

Intervention

Families randomized to the TE-HNC group in the parent RCT received the standard HNC program, as well as technology enhancements via a HIPAA-compliant technical system, which allowed clinicians (via a web portal) to monitor and reinforce caregiver progress (via native iOS application, Tantrum Tamers©). HNC is a mastery-based BPT program that includes weekly in-person sessions and brief midweek calls with clinicians. Progression through and completion of the HNC program, which consists of two phases, is dependent upon parents meeting criteria for each new skill. Mastery of parenting skills is determined by weekly observations and coding of parent skill use by therapists in a clinic-based setting and some work suggests this type of format may be optimal with low-income families (Lundahl et al., 2006; see Kaminski et al., 2008 for a meta-analysis). In the first phase, Differential Attention, the caregiver is taught methods for (re)establishing a positive and mutually reinforcing parent-child relationship, including attending, rewarding, and active ignoring skills. During treatment, caregivers practice these skills in session in the context of child-directed play (Child’s Game) and are instructed to practice at home for a minimum of 15 minutes per day. Once parents master Phase I skills, they progress to Phase II, Compliance Training. In Phase II, caregivers are taught skills to maximize opportunities for child compliance (i.e., clear instructions) and implement non-physical consequences for noncompliance and inappropriate, nonignorable behaviors (i.e., Time-Out). Phase II skills are taught in the session using caregiver-directed activities (i.e., clean-up task). To maintain mastery of Phase I skills, caregivers are encouraged to practice earlier skills in later sessions and instructed to practice Child’s Game at home throughout both phases.

An adjunct to the standard HNC program (e.g., weekly clinic-based sessions, midweek calls, daily home practice), the Tantrum Tamers mobile app was originally developed by an interdisciplinary team consisting of researchers with expertise in BPT for underserved families, an advisory panel of five clinicians who practiced at least one BPT program, an industry partner with experience developing health-related software apps, and health economists with expertise in health care evaluation, efficiency, and effectiveness (Jones et al., 2014). Building upon the functionality and content tested in the pilot study, the Tantrum Tamers mobile app allowed clinicians to integrate additional behavior change strategies associated with improvements in intervention effectiveness, including three features initiated by caregivers, independently, to promote the acquisition and use of parenting skills at home—Skills Videos, Surveys, and Videorecording Skill Practice (Jones et al., 2020; Webb et al., 2010). The Skills Videos corresponded to each of the program skills (e.g., Attends, Rewards, and Ignoring) and included both psychoeducation and skill demonstrations. Surveys assessed parents’ use of skills outside of sessions, as well as their attitude, mood, and stress management during the day. In response, parents were provided tailored feedback via the app (e.g., positive reinforcement for the completion of home practice and watching Skills Videos) and by clinicians during sessions to adapt and reinforce skills practice. Finally, parents were instructed to upload a 15-minute videorecording of skills practice with their child each week for clinician review and feedback during sessions. Caregivers were only able to access the Tantrum Tamers app through the study phone which was provided to families at randomization.

Therapist Training and Fidelity

In the parent RCT, Master’s-level clinicians treated families in the HNC and TE-HNC groups. Training included reviewing treatment manuals, establishing reliability with the HNC coding system, roleplays, observations and discussion, and supervision and feedback by two licensed clinical psychologists. Approximately 30% of sessions were coded by a Master’s-level or doctoral-level coder to ensure treatment fidelity and 61% of those were double-coded. Analyses yielded a treatment fidelity of 98% in the TE-HNC group.

Measures

Demographics

Participating parents provided demographic information including age, race, ethnicity, and gender of self and the child, as well as family income.

Mobile App Component Use

Parents’ use of mobile app components (Surveys, Skills Videos, and Videorecording Skills Practice) was operationalized as the frequency of component use contextualized by opportunities for use. The frequency of component use was determined via passively collected app data and opportunities for use were defined as the number of times parents’ were instructed to engage with the component at randomization, thus providing an indicator of component use in a manner consistent with treatment specifications. Opportunities for use were defined as weekly for submitting a videorecording of skills practice, and daily for watching a video and completing a survey. For example, Survey use during this phase was defined by the number of surveys completed divided by the total number of days in treatment. An overall mobile app use variable was calculated by averaging use across the three components.

Efficiency of Service Delivery

The efficiency of service delivery was operationalized as the number of weeks required for each family to master program skills and complete the program. Families were considered to have completed the program once the parent met behavioral skill criterion for each program skill, which in HNC and other mastery-based BPT programs determine progress from one skill to the next, Phase I to II, and treatment completion. HNC mastery criteria were assessed by clinician observation and coding of parent and child behavior during weekly sessions.

Plan of Analyses

Descriptive analyses were used to examine demographics, mobile app use, and the efficiency of service delivery. First, given the mobile app was configured for iOS operating systems and may have influenced app use depending on mobile device ownership prior to treatment, an independent samples t-test was conducted to examine whether overall app use differed significantly between participants who owned an Android and iPhone at baseline. Second, correlation and multiple linear regression analyses were conducted to detect if use of the mobile app (Skills Video, Survey, and Videorecording Skills Practice components) during treatment predicted the efficiency of service delivery (i.e., weeks to treatment completion) in TE-HNC. For the latter, a priori sensitivity power analysis determined a minimum detectable effect size of f2 = 0.246 for the two-tailed test given the sample size (n = 34), alpha (.05), power (0.80), and number of predictors (3). Data were analyzed using IBM SPSS version 27.0. Diagnostics were performed on the data to ensure all assumptions of statistical tests were met and power analyses were conducted using G*Power version 3.1 (Faul et al., 2007).

Results

Mobile App Use

All caregivers used the mobile app at least once during treatment. The average overall mobile app use across components was 41.3% (SD = 22.1); in other words, caregivers used the mobile app components, on average, slightly less than half of the number of times they were instructed to during treatment. Overall use did not significantly differ as a function of whether caregivers owned an Android or iPhone device prior to treatment, t(30) = 0.669, p = .509 (Table 2). The Videorecording Skills Practice component was used most frequently on average (62.8%; SD = 33.2), followed by Survey use (35.1%; SD = 23.8) and Skills Video use (26.0%; SD = 23.4). During treatment, 22 (64.7%) parents used a component of the app at least once 25% of the days, with 14 (41.1%) and 3 (8.8%) parents using a component at least once more than 50% and 75% of the days, respectively.

Table 2.

t-Test Comparing Overall Mobile App Use Between iPhone and Android Smartphone Ownership.

Smartphone ownership M (SD) t p-Value
iPhone 38.1 (24.0) 0.669 .509
Android 43.6 (22.0)
Open in a new tab

Relation Between Mobile App Use and Efficiency of Service Delivery

On average, families completed treatment in 11.98 weeks (SD = 4.4). A correlation matrix was initially conducted with each of the main study variables (component use and efficiency of service delivery). While the three components were significantly intercorrelated (.444–.620, Table 3), only use of the Survey and Videorecording Skills Practice components was correlated with the outcome variable.

Table 3.

Correlations Between Mobile App Component Use and Efficiency of Service Delivery.

Variables M (SD) Range 1 2 3 4
1 Skills video use (%) 26.0 (23.4) –
2 Survey use (%) 35.1 (23.8) .449** –
3 Videorecording skills practice use (%) 62.8 (33.2) .444** .620** –
4 Efficiency of service delivery (weeks in treatment) 11.98 (4.4) 7–26 −.250 −.422* −.596** –
Open in a new tab
*

p < .05. **p < .01.

A multiple regression with the three usage variables entered simultaneously was subsequently performed. The overall model for the efficiency of service delivery was statistically significant, F(3, 30) = 7.41, p = .001, with an R2 of .426. Videorecording Skills Practice was significantly associated with efficiency of service delivery (B = 0.70, p = .001), but use of the Survey (B = 1.03, n.s.) and Skills Video (B = 1.02, n.s.) components did not add significantly to the predictive model. Additionally, the model predicted that the number of weeks to achieve sufficient parent skill use and child behavior scores decrease by 7 weeks (~5 days) for every use of the Videorecording Skills Practice component (i.e., ratio of actual component use to opportunities for use). The adjusted R2, which is a corrected goodness-of-fit measure that considers the number of predictors in the model, is .368. Use of the Skills Video and Survey components did not significantly influence the relationship between use of the Videorecording Skills Practice component and the efficiency of service delivery. Results from the model are reported in Table 4.

Table 4.

Multiple Regression Analyses Predicting the Efficiency of Service Delivery.

Efficiency of Service Delivery p-Value R 2 F(df), p
log B B SE B t
.426 7.408 (3, 30), .001
Constant 3.67 39.33 1.31 13.669 <.001
Skills video use 0.02 1.02 1.07 0.325 .747
Survey use 0.03 1.03 1.09 0.369 .714
Videorecording skills practice use −0.36 0.70 1.10 −3.619 .001
Open in a new tab

Note. Model was run with log-transformed variables to meet regression assumptions. Beta from the log-transformed model (Log B) is presented to illustrate the directionality of association. The remaining variables presented in the table have been converted to linear scale via exponentiation for interpretability.

Discussion

Despite the increase in the development of behavior intervention technologies for BPT, few studies to date have examined actual use of technology components in programs (e.g., Breitenstein et al., 2017), especially for technologies designed to augment, rather than substitute, face-to-face treatment (e.g., Cefai et al., 2010; Nixon et al., 2003). Thus, the current study addressed this gap by examining parent use of an adjunct behavioral intervention technology to clarify the distinct and differential effects of engagement with technology components on the treatment efficiency of TE-HNC in low-income families.

Notably, app usage was relatively high during the active treatment phase in comparison to previously documented trends in mHealth and mobile app retention (e.g., 45%–70% reduction in users after 1 week; Bradway et al., 2018; Lattie et al., 2016; Sigg et al., 2016), which may be due, in part, to the additional clinician (e.g., weekly sessions), technology (e.g., automated reminders), and administrative (e.g., technical support) level supports not characteristic of most standalone or commercially available behavioral intervention technologies in prior studies. Consistent with discussions about the caution that is warranted in overinterpreting pilot effect sizes (e.g., Kistin & Silverstein, 2015; Kraemer et al., 2006; Lancaster et al., 2004), overall mobile app use was lower in the present study than with the pilot version in the preliminary study. However, similar trends emerged with Videorecording Skills Practice used most frequently relative to opportunities for use, followed by Survey and Skills Video use (Anton et al., 2016).

Behavioral intervention technologies in BPT have demonstrated efficacy in prior research. Less work has considered if and how parents use these technologies or their impact on treatment efficiency (Bausback & Bunge, 2021), which has been shown to impact implementation in routine practice settings (e.g., Álvarez Lorenzo et al., 2016; Breitenstein et al., 2010; Keyworth et al., 2018). As hypothesized, the current study demonstrates that use of specific mobile app components in TE-HNC predicted the efficiency of service delivery, or how quickly parents met mastery criteria for each of the program skills and completed the program. Further, findings suggest that videorecording parenting skills practice at home and associated feedback, which is not a feature of the majority of behavioral intervention technologies for BPT, may be particularly beneficial for these purposes and additive to traditional and technology-enhanced BPT treatments. One potential explanation for these findings is that the Videorecording Skills Practice component requires that parents engage in an activity (i.e., practicing new skills with the child at home) that is an existing ingredient in traditional BPT programs like HNC, such that the benefits of increased use may simply reflect the effects of greater at-home practice. Although this possibility cannot be ruled out given the study design, open-ended feedback from TE-HNC families after treatment suggests two alternative explanations. For one, the component may serve as a source of accountability for parents to engage in home practice as instructed during treatment, as illustrated by quotes from parents about the specific component during the post-treatment assessment (e.g., “[Videorecording skills practice] forced me to ensure I was having that one-on-one time [with my child]”) and consistent with findings that TE-HNC families completed home practice at significantly higher rates than families who only received the standard treatment in the parent RCT (Jones et al., 2020). This potential benefit may be especially important to explore in future research, as some evidence suggests highly variable caregiver engagement in home practice (39%–89%), which is a common practice element in BPT programs and associated with treatment outcomes (Danko et al., 2016; Berkovits et al., 2010; Högström et al., 2015; Jones et al., 2020; Lyon & Budd, 2010). Additionally, the component may allow clinicians to acquire a more nuanced understanding of parent skill use and child behavior at home and, accordingly, allow parents to receive tailored support on home practice from their clinician (e.g., “The therapist could really see what was going on and [I could] get real feedback on what we needed to improve on”). In other words, Videorecording Skills Practice may facilitate the generalization of the HNC skills from the weekly sessions to the family’s home. Although both possibilities confer an advantage given the importance of consistent practice and skill generalization for behavior change, this component requires a slightly more sophisticated technical system (e.g., video recording and storage capabilities) and use of clinical resources (e.g., clinician time) than the other components. Yet, such functions could be augmented in a range of different formats as technologies and clinical practice evolve, including as a guided behavioral intervention technology with adjunctive human support (e.g., self-directed BPT program delivered online with weekly interactions with a clinician to review video content) or as a standalone technology (e.g., self-directed BPT program with artificial intelligence algorithms to analyze and provide feedback on video content; see Entenberg et al., 2021 for a recent example application of AI chatbox in BPT). Given clinician time and resources may be especially limited in community settings serving low-income communities, future research is necessary to identify specifications (e.g., optimal dosage of the component) and implementation strategies for delivery of TE-HNC in routine practice, as well as evaluate the feasibility, effectiveness, and ethical considerations for the integration of novel technologies (e.g., artificial intelligence) in BPT.

The results of the current study must also be interpreted in the context of its limitations. First, the relatively small sample size limited the opportunity for theoretically-relevant analyses of patterns of mobile app use and treatment efficiency across and within dimensions of socioeconomic status, race and ethnicity, and diverse family compositions. In addition, iPhones with service plans were provided for each family and some families experienced technical difficulties using the mobile app, which may have impacted engagement and real-world implications of the findings.

This study also had several strengths. For one, there has been significant and increasing investment in the development of mental health technologies in recent years across industry and academia, with relatively less attention toward assessing how outcomes improve as a function of engagement with these technologies in ecologically-valid settings (National Institute of Mental Health, 2019; Shah & Berry, 2021;). Thus, this study represented a timely advancement beyond between-group analyses in randomized controlled trials (e.g., HNC vs. TE-HNC) to within-group analyses that further explore the potential mechanisms underlying observed benefits of one adjunct behavioral intervention technology for BPT (Jones et al., 2014, 2020). Additionally, evidence suggests that self-reported measures of technology use have been subject to over- and underreporting (Boase & Ling, 2013; Scharkow, 2016; Short et al., 2018). Thus, this study examined an objective measure of use in analyses. Third, there is a dearth of work on BPT-related behavioral intervention technologies for underserved populations, and this study focused on low-income families who are more likely to have a child with an early-onset behavior disorder and less likely to engage in mental health services (Bausback & Bunge, 2021). Finally, given that HNC is one example in a family of evidence-based BPT programs with similar treatment elements, findings may generalize to and inform research and clinical practice across other interventions as well (Kaehler et al., 2016; Southam-Gerow & Prinstein, 2014).

Conclusion

There is a clear need to cost-effectively improve the efficiency of BPT for low-income families. Despite increased attention to behavior intervention technologies in BPT, very little research has examined actual use of technology components in programs and their differential impact on treatment efficiency. With the impact of technology use varying across specific mobile app components, results demonstrate that parents who used the Videorecording Skills Practice component more frequently required fewer weeks to achieve skill mastery and, in turn, complete treatment. While study limitations should be considered in their interpretation, findings represent a step toward understanding how behavioral intervention technologies may be used to improve BPT service delivery for families facing structural socioeconomic barriers to effective mental health care.

Acknowledgments

The authors would like to thank the families who participated in the study for their contributions to this research.

Author Biographies

Madison P. McCall is a Doctoral Student in Clinical Psychology at the University of North Carolina at Chapel Hill and a Robert Wood Johnson Foundation Health Policy Research Scholar. Her research focuses on the design, dissemination, and evaluation of digital tools that advance mental health care delivery for underserved youth and their families.

Margaret T. Anton is a Senior Clinical Research Scientist at AbleTo, Inc., a virtual behavioral health care company providing technology-enabled services for anxiety and depression. She evaluates AbleTo’s services and provides clinical content development consultation. Her research focuses on designing, testing, and implementing digital technologies to improve access to care.

April Highlander, MA, is a doctoral candidate in the Clinical Psychology program at the University of North Carolina at Chapel Hill. Her research focuses on evidence-based treatments and family-oriented approaches designed to enhance the well-being of young children, parents, and families, particularly those historically underrepresented and underserved in research and clinical practice.

Raelyn Loiselle, PhD, is a Postdoctoral Fellow of clinical psychology at Child Study Center at NYU Langone Health and NYU Grossman School of Medicine. She provides clinical evaluations and interventions for children and adolescents with Autism Spectrum Disorders. Her research focuses on parenting, parent-child relationships, and parental self-regulation variables associated with childhood behavior disorders.

Olga Khavjou, MA, is a Health Economist with more than 20 years of experience. She has extensive experience developing cost collection instruments for public health programs. She has coauthored a Guide to Analyzing the Cost-Effectiveness of Community Prevention Approaches and has conducted a number of economic evaluations for public health programs.

Rex Forehand, PhD, is the Heinz and Rowena Ansbacher Endowed Professor and University Distinguished Professor of Psychological Science at The University of Vermont. His research focuses on family stress (e.g., parent depression and interparental conflict), child psychosocial adjustment, and parenting prevention and intervention efforts. His research has been funded by agencies such as the National Institutes of Health and the Centers for Disease Control and Prevention.

Deborah J. Jones, PhD, is Zachary Smith Distinguished Term Professor of Psychology and Neuroscience at the University of North Carolina at Chapel Hill. Her research explores the mechanisms underlying child, parent, and family functioning and how those factors shape treatment access, process, and outcomes. Her work has been funded by the National Institutes and the Centers for Disease Control and Prevention.

Footnotes

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the NIMH R01MH100377 (ClinicalTrials.gov: NCT01367847) with additional support from the National Institute of Mental Health (R21MH113887; ClinicalTrials.gov Identifier: NCT03597789; 3R21MH113887-02S1), the National Science Foundation (DGE-1650116), and RWJF Health Policy Research Scholars Program.

References

  1. Álvarez Lorenzo M., Rodrigo M., Byrne S. (2016). What implementation components predict positive outcomes in a parenting program? Research on Social Work Practice, 28, 173–187. 10.1177/1049731516640903 [DOI] [Google Scholar]
  2. Anderson-Lewis C., Darville G., Mercado R. E., Howell S., Maggio S. D. (2018). MHealth technology use and implications in historically underserved and minority populations in the United States: Systematic literature review. JMIR MHealth and UHealth, 6(6), e128. 10.2196/mhealth.8383 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Anton M. T., Jones D. J., Cuellar J., Forehand R., Gonzalez M., Honeycutt A., Khavjou O., Newey G., Edwards A., Jacobs M., Pitmman S. (2016). Caregiver use of the core components of technology-enhanced helping the noncompliant child: A case series analysis of low-income families. Cognitive and Behavioral Practice, 23(2), 194–204. 10.1016/j.cbpra.2015.04.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bausback K. B., Bunge E. L. (2021). Meta-analysis of parent training programs utilizing behavior intervention technologies. Social Sciences, 10(10), 367. 10.3390/socsci10100367 [DOI] [Google Scholar]
  5. Berkovits M. D., O’Brien K. A., Carter C. G., Eyberg S. M. (2010). Early identification and intervention for behavior problems in primary care: A comparison of two abbreviated versions of parent-child interaction therapy. Behavior Therapy, 41(3), 375–387. [DOI] [PubMed] [Google Scholar]
  6. Boase J., Ling R. (2013). Measuring mobile phone use: Self-report versus log data. Journal of Computer-Mediated Communication, 18(4), 508–519. 10.1111/jcc4.12021 [DOI] [Google Scholar]
  7. Bradway M., Pfuhl G., Joakimsen R., Ribu L., Grøttland A., Årsand E. (2018). Analysing mHealth usage logs in RCTs: Explaining participants’ interactions with type 2 diabetes self-management tools. PLoS One, 13(8), e0203202. 10.1371/journal.pone.0203202 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Breitenstein S. M., Brager J., Ocampo E. V., Fogg L. (2017). Engagement and adherence with ezPARENT, an mHealth parent-training program promoting child well-being. Child Maltreatment, 22(4), 295–304. 10.1177/1077559517725402 [DOI] [PubMed] [Google Scholar]
  9. Breitenstein S. M., Gross D., Garvey C., Hill C., Fogg L., Resnick B. (2010). Implementation fidelity in community-based interventions. Research in Nursing & Health, 33(2), 164–173. 10.1002/nur.20373 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cefai J., Smith D., Pushak R. E. (2010). Parenting wisely: Parent training via CD-ROM with an Australian sample. Child & Family Behavior Therapy, 32(1), 17–33. 10.1080/07317100903539709 [DOI] [Google Scholar]
  11. Chacko A., Isham A., Cleek A. F., McKay M. M. (2016). Using mobile health technology to improve behavioral skill implementation through homework in evidence-based parenting intervention for disruptive behavior disorders in youth: Study protocol for intervention development and evaluation. Pilot and Feasibility Studies, 2(1), 57. 10.1186/s40814-016-0097-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Chacko A., Jensen S. A., Lowry L. S., Cornwell M., Chimklis A., Chan E., Lee D., Pulgarin B. (2016). Engagement in behavioral parent training: Review of the literature and implications for practice. Clinical Child and Family Psychology Review, 19(3), 204–215. 10.1007/s10567-016-0205-2 [DOI] [PubMed] [Google Scholar]
  13. Cohen M. A., Piquero A. R. (2009). New evidence on the monetary value of saving a high risk youth. Journal of Quantitative Criminology, 25, 25–49. [Google Scholar]
  14. Comer J. S., Furr J. M., Cooper-Vince C., Madigan R. J., Chow C., Chan P., Idrobo F., Chase R. M., McNeil C. B., Eyberg S. M. (2015). Rationale and considerations for the internet-based delivery of parent-child interaction therapy. Cognitive and Behavioral Practice, 22(3), 302–316. 10.1016/j.cbpra.2014.07.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Conger R. D., Donnellan M. B. (2007). An interactionist perspective on the socioeconomic context of human development. Annual Review of Psychology, 58, 175–199. 10.1146/annurev.psych.58.110405.085551 [DOI] [PubMed]
  16. Danko C. M., Brown T., Van Schoick L., Budd K. S. (2016). Predictors and correlates of homework completion and treatment outcomes in parent–child interaction therapy. Child & Youth Care Forum, 45(3), 467–485. [Google Scholar]
  17. Entenberg G. A., Areas M., Roussos A. J., Maglio A. L., Thrall J., Escoredo M., Bunge E. L. (2021). Using an artificial intelligence based chatbot to provide parent training: Results from a feasibility study. Social Sciences, 10(11), 426. 10.3390/socsci10110426 [DOI] [Google Scholar]
  18. Eyberg S., Pincus D. (1999). Eyberg child behavior inventory & Sutter-Eyberg student behavior inventory-revised: Professional manual. Psychological Assessment Resources. [Google Scholar]
  19. Faul F., Erdfelder E., Lang A.-G., Buchner A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. 10.3758/BF03193146 [DOI] [PubMed] [Google Scholar]
  20. Fergusson D. M., Horwood L. J., Ridder E. M. (2005). Show me the child at seven: The consequences of conduct problems in childhood for psychosocial functioning in adulthood. Journal of Child Psychology and Psychiatry, 46(8), 837–849. 10.1111/j.1469-7610.2004.00387.x [DOI] [PubMed] [Google Scholar]
  21. Gardner F., Connell A., Trentacosta C. J., Shaw D. S., Dishion T. J., Wilson M. N. (2009). Moderators of outcome in a brief family-centered intervention for preventing early problem behavior. Journal of Consulting and Clinical Psychology, 77(3), 543–553. 10.1037/a0015622 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Guerra-Reyes L., Christie V. M., Prabhakar A., Harris A. L., Siek K. A. (2016). Postpartum health information seeking using mobile phones: Experiences of low-income mothers. Maternal and Child Health Journal, 20(1), 13–21. 10.1007/s10995-016-2185-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Högström J., Enebrink P., Melin B., Ghaderi A. (2015). Eighteen-month follow-up of internet-based parent management training for children with conduct problems and the relation of homework compliance to outcome. Child Psychiatry & Human Development, 46(4), 577–588. [DOI] [PubMed] [Google Scholar]
  24. Jones D. J., Forehand R., Cuellar J., Kincaid C., Parent J., Fenton N., Goodrum N. (2013). Harnessing innovative technologies to advance children’s mental health: behavioral parent training as an example. Clinical Psychology Review, 33(2), 241–252. 10.1016/j.cpr.2012.11.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Jones D. J., Forehand R., Cuellar J., Parent J., Honeycutt A., Khavjou O., Gonzalez M., Anton M., Newey G. A. (2014). Technology-enhanced program for child disruptive behavior disorders: Development and pilot randomized control trial. Journal of Clinical Child and Adolescent Psychology, 43(1), 88–101. 10.1080/15374416.2013.822308 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Jones D. J., Loiselle R., Zachary C., Georgeson A. R., Highlander A., Turner P., Youngstrom J. K., Khavjou O., Anton M. T., Gonzalez M., Bresland N. L., Forehand R. (2020). Optimizing engagement in behavioral parent training: Progress toward a technology-enhanced treatment model. Behavior Therapy, 52, 508–521. 10.1016/j.beth.2020.07.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Kaehler L. A., Jacobs M., Jones D. J. (2016). Distilling common history and practice elements to inform dissemination: Hanf-model BPT programs as an example. Clinical Child and Family Psychology Review, 19(3), 236–258. 10.1007/s10567-016-0210-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Kaminski J. W., Valle L. A., Filene J. H., Boyle C. L. (2008). A meta-analytic review of components associated with parent training program effectiveness. Database of abstracts of reviews of effects (DARE): Quality-assessed Reviews. Centre for Reviews and Dissemination. https://www.ncbi.nlm.nih.gov/books/NBK75130/ [DOI] [PubMed] [Google Scholar]
  29. Keyworth C., Hart J., Armitage C. J., Tully M. P. (2018). What maximizes the effectiveness and implementation of technology-based interventions to support healthcare professional practice? A systematic literature review. BMC Medical Informatics and Decision Making, 18, 93. 10.1186/s12911-018-0661-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kistin C., Silverstein M. (2015). Pilot studies: A critical but potentially misused component of interventional research. JAMA, 314(15), 1561–1562. 10.1001/jama.2015.10962 [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Kraemer H. C., Mintz J., Noda A., Tinklenberg J., Yesavage J. A. (2006). Caution regarding the use of pilot studies to guide power calculations for study proposals. Archives of General Psychiatry, 63(5), 484–489. 10.1001/archpsyc.63.5.484 [DOI] [PubMed] [Google Scholar]
  32. Kubb C., Foran H. (2020). Online health information seeking by parents for their children: Systematic review and agenda for further research. Journal of Medical Internet Research, 22, e19985. 10.2196/19985 [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Lancaster G. A., Dodd S., Williamson P. R. (2004). Design and analysis of pilot studies: Recommendations for good practice. Journal of Evaluation in Clinical Practice, 10(2), 307–312. 10.1111/j..2002.384.doc.x [DOI] [PubMed] [Google Scholar]
  34. Lattie E. G., Schueller S. M., Sargent E., Stiles-Shields C., Tomasino K. N., Corden M. E., Begale M., Karr C. J., Mohr D. C. (2016). Uptake and usage of IntelliCare: A publicly available suite of mental health and well-being apps. Internet Interventions, 4, 152–158. 10.1016/j.invent.2016.06.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Long N., Edwards M. C., Bellando J. (2017). Parent training interventions. In Matson J. L. (Ed.), Handbook of childhood psychopathology and developmental disabilities treatment (pp. 63–86). Springer International Publishing. 10.1007/978-3-319-71210-9_5 [DOI] [Google Scholar]
  36. Lundahl B., Risser H. J., Lovejoy M. C. (2006). A meta-analysis of parent training: Moderators and follow-up effects. Clinical Psychology Review, 26(1), 86–104. 10.1016/j.cpr.2005.07.004 [DOI] [PubMed] [Google Scholar]
  37. Lyon A. R., Budd K. S. (2010). A community mental health implementation of parent–child interaction therapy (PCIT). Journal of Child and Family Studies, 19(5), 654–668. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. McMahon R. J., Forehand R. L. (2005). Helping the noncompliant child: Family-based treatment for oppositional behavior. Guilford Press. [Google Scholar]
  39. Merikangas K. R., Nakamura E. F., Kessler R. C. (2009). Epidemiology of mental disorders in children and adolescents. Dialogues in Clinical Neuroscience, 11(1), 7–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. National Institute of Mental Health. (2019). Technology and the Future of Mental Health Treatment. National Institute of Mental Health (NIMH). Retrieved January 10, 2022, from https://www.nimh.nih.gov/health/topics/technology-and-the-future-of-mental-health-treatment [Google Scholar]
  41. Nixon R. D. V., Sweeney L., Erickson D. B., Touyz S. W. (2003). Parent-child interaction therapy: A comparison of standard and abbreviated treatments for oppositional defiant preschoolers. Journal of Consulting and Clinical Psychology, 71(2), 251–260. 10.1037/0022-006x.71.2.251 [DOI] [PubMed] [Google Scholar]
  42. Odgers C. L., Caspi A., Broadbent J. M., Dickson N., Hancox R. J., Harrington H., Poulton R., Sears M. R., Thomson W. M., Moffitt T. E. (2007). Prediction of differential adult health burden by conduct problem subtypes in males. Archives of General Psychiatry, 64(4), 476. 10.1001/archpsyc.64.4.476 [DOI] [PubMed] [Google Scholar]
  43. Ortiz C., Vidair H. B., Acri M., Chacko A., Kobak K. (2020). Pilot study of an online parent-training course for disruptive behavior with live remote coaching for practitioners. Professional Psychology: Research and Practice, 51(2), 125–133. 10.1037/pro0000286 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Parent J., Anton M. T., Loiselle R., Highlander A., Breslend N., Forehand R., Hare M., Youngstrom J. K., Jones D. J. (2022). A randomized controlled trial of technology-enhanced behavioral parent training: Sustained parent skill use and child outcomes at follow-up. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 63, 992–1001. 10.1111/jcpp.13554 [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Pew Research Center. (2021, April 7). Demographics of mobile device ownership and adoption in the United States. Pew Research Center: Internet, Science & Tech. Retrieved January 10, 2022, from https://www.pewresearch.org/internet/fact-sheet/mobile/ [Google Scholar]
  46. Piquero A. R., Farrington D. P., Welsh B. C., Tremblay R., Jennings W. G. (2009). Effects of early family/parent training programs on antisocial behavior and delinquency. Journal of Experimental Criminology, 5(2), 83–120. 10.1007/s11292-009-9072-x [DOI] [Google Scholar]
  47. Piquero A. R., Jennings W. G., Diamond B., Farrington D. P., Tremblay R. E., Welsh B. C., Gonzalez J. M. R. (2016). A meta-analysis update on the effects of early family/parent training programs on antisocial behavior and delinquency. Journal of Experimental Criminology, 12(2), 229–248. 10.1007/s11292-016-9256-0 [DOI] [Google Scholar]
  48. Polanczyk G. V., Salum G. A., Sugaya L. S., Caye A., Rohde L. A. (2015). Annual research review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. Journal of Child Psychology and Psychiatry, 56(3), 345–365. 10.1111/jcpp.12381 [DOI] [PubMed] [Google Scholar]
  49. Reyno S. M., McGrath P. J. (2006). Predictors of parent training efficacy for child externalizing behavior problems—A meta-analytic review. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 47(1), 99–111. 10.1111/j.1469-7610.2005.01544.x [DOI] [PubMed] [Google Scholar]
  50. Santiago C. D., Wadsworth M. E., Stump J. (2011). Socioeconomic status, neighborhood disadvantage, and poverty-related stress: Prospective effects on psychological syndromes among diverse low-income families. Journal of Economic Psychology, 32(2), 218–230. 10.1016/j.joep.2009.10.008 [DOI] [Google Scholar]
  51. Scharkow M. (2016). The accuracy of self-reported internet use—A validation study using client log data. Communication Methods and Measures, 10(1), 13–27. 10.1080/19312458.2015.1118446 [DOI] [Google Scholar]
  52. Shah R. N., Berry O. O. (2021). The rise of venture capital investing in mental health. JAMA Psychiatry, 78(4), 351–352. 10.1001/jamapsychiatry.2020.2847 [DOI] [PubMed] [Google Scholar]
  53. Shaw D. S., Taraban L. E. (2016). New directions and challenges in preventing conduct problems in early childhood. Child Development Perspectives, 11(2), 85–89. 10.1111/cdep.12212 [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Short C. E., DeSmet A., Woods C., Williams S. L., Maher C., Middelweerd A., Müller A. M., Wark P. A., Vandelanotte C., Poppe L., Hingle M. D., Crutzen R. (2018). Measuring engagement in ehealth and mhealth behavior change interventions: Viewpoint of Methodologies. Journal of Medical Internet Research, 20(11), e9397. 10.2196/jmir.9397 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Sigg S., Lagerspetz E., Peltonen E., Nurmi P., Tarkoma S. (2016). Sovereignty of the apps: There’s more to relevance than downloads. ArXiv:1611.10161 [Cs]. http://arxiv.org/abs/1611.10161
  56. Smith A. (2015, April 1). U.S. smartphone use in 2015. Pew Research Center: Internet, Science & Tech. https://www.pewresearch.org/internet/2015/04/01/us-smartphone-use-in-2015/ [Google Scholar]
  57. Southam-Gerow M. A., Prinstein M. J. (2014). Evidence base updates: The evolution of the evaluation of psychological treatments for children and adolescents. Journal of Clinical Child and Adolescent Psychology, 43(1), 1–6. 10.1080/15374416.2013.855128 [DOI] [PubMed] [Google Scholar]
  58. Sullivan A. D. W., Forehand R., Acosta J., Parent J., Comer J. S., Loiselle R., Jones D. J. (2021). COVID-19 and the acceleration of behavioral parent training telehealth: Current status and future directions. Cognitive and Behavioral Practice, 28(4), 618–629. 10.1016/j.cbpra.2021.06.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Van Dijk J. (2020). The digital divide. John Wiley & Sons. [Google Scholar]
  60. Vangeepuram N., Mayer V., Fei K., Hanlen-Rosado E., Andrade C., Wright S., Horowitz C. (2018). Smartphone ownership and perspectives on health apps among a vulnerable population in East Harlem, New York. MHealth, 4, 31. 10.21037/mhealth.2018.07.02 [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Vogels E. A. (2021). Digital divide persists even as Americans with lower incomes make gains in tech adoption. Pew Research Center. Retrieved January 10, 2022, from https://www.pewresearch.org/fact-tank/2021/06/22/digital-divide-persists-even-as-americans-with-lower-incomes-make-gains-in-tech-adoption/ [Google Scholar]
  62. Wadsworth M. E., Raviv T., Santiago C. D., Etter E. M. (2011). Testing the adaptation to poverty-related stress model: Predicting psychopathology symptoms in families facing economic hardship. Journal of Clinical Child & Adolescent Psychology, 40(4), 646–657. 10.1080/15374416.2011.581622 [DOI] [PubMed] [Google Scholar]
  63. Webb T. L., Joseph J., Yardley L., Michie S. (2010). Using the internet to promote health behavior change: A systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. Journal of Medical Internet Research, 12(1), e4. 10.2196/jmir.1376 [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Behavior Modification are provided here courtesy of SAGE Publications

RESOURCES