Reducing Therapist Contact in Parenting Programs: Evaluation of Internet-Based Treatments for Child Conduct Problems

Abstract

This study evaluated two Internet-based versions of Parent Management Training (PMT) and the effects of greatly reducing the contact required of a mental health professional on treatment of children referred for conduct problems. We were interested whether reduced contact with a therapist influenced treatment outcome, therapeutic alliance, parent adherence to treatment prescriptions, and parent reactions to and evaluations of the treatment procedures. Sixty children and their caregivers were assigned to receive either Full Contact PMT (with the amount of weekly contact similar to traditional PMT; approximately 50 minutes of direct therapist contact each week) or Reduced Contact PMT (with most information provided through recordings; approximately 10 minutes of therapist contact each week). Children in both groups showed significant and similar reductions in antisocial behaviors specifically, internalizing and externalizing symptoms more generally, and improvements in overall adaptive functioning. Therapeutic alliance also was similar across the two treatment groups. However, parents rated Full Contact treatment as more acceptable than the reduced version. Both treatments were similar in outcomes to in-person treatment as evaluated by a nonrandomized matched sample used as a benchmark in supplementary analyses. Overall, the findings indicate that therapist contact can be reduced while positive treatment outcomes are maintained but that interventions that reduce direct time with a therapist may be viewed less positively by clients.

Providing mental health services to those experiencing clinical dysfunction is a significant challenge facing the healthcare system. Out of a population of approximately 320 million people in the United States (U.S.), approximately 25% (~ 80 million people) meet criteria for a psychiatric diagnosis in a given year (Kessler & Wang, 2008). Unfortunately only about 20-30% of people in need of services actually receive any type of treatment (Kessler et al., 2005). There are many reasons so many people are untreated, including cost of services, stigma associated with treatment, lack of access to treatment resources, and difficulties in identifying dysfunction and then recognizing mental health services as a viable remedy (e.g., Chen et al., 2013; Mojtabai et al., 2011). One major barrier that has received relatively little attention is the model of delivery of psychological services. The dominant treatment model has been one-to-one, in-person individual therapy (Kazdin & Rabbitt, 2013). Currently in the U.S., the majority of treatments are administered in this model (e.g., Hersen, 2005; Kazdin, 2000). The one-to-one, in-person model has been enduring, is in demand, and can deliver many evidence-based interventions. In spite of these obvious strengths, there are not enough available treatment providers in the U.S. or abroad to deliver services in this model, leading to efforts to expand services through nontraditional treatment delivery models (Kazdin & Rabbitt, 2013).

As part of this growing effort, technology is being used to deliver treatment through new and creative ways, including the Internet, virtual reality systems, mobile phone texting, smartphone applications, and socially assistive robots (Bennett-Levy, Richards, Farrand, Christensen, & Griffiths, 2010; Donker et al., 2013; Rabbitt, Kazdin, & Scassellati, 2015). To date, Internet-based treatment programs have garnered the most attention in the research literature. They have been effectively used to treat diverse mental health problems (e.g., mood disorders, anxiety disorders, eating disorders) and have achieved effects comparable to those obtained with in-person treatment (Andersson et al., 2005; Andrews, Cuijpers, Craske, McEvoy, & Titov, 2010; Carlbring & Andersson, 2006; Ljotsson et al., 2007). Unfortunately, Internet based-interventions are not a panacea. Many programs focus on one-to-one treatment delivered by computer (e.g., through videoconference), leaving them open to the same criticism as in-person inventions: ultimately, the workforce needed to provide the interventions is not available and cannot easily be created (Kazdin & Rabbitt, 2013).

Simply adapting existing interventions into programs delivered online (instead of in-person) will not meet the unmet treatment need. In order to effectively bridge the gap in care, new and different ways of intervening are needed (Kazdin, 2015). Reducing the amount of professional time required to treat clinical dysfunction is an opportunity to utilize the work force in ways that would reach more people in need of care. This is critical because the number of individuals in need of treatment far exceeds the number of mental professionals would could treat them (Kazdin & Blase, 2011). Internet-based treatments are particularly well suited for reduction in direct therapist contact because of alternatives to direct contact that are now available thanks to technological innovations. Additionally, multiple Internet-based programs have been designed with minimal or no contact with a mental health professional which provide guidance in how to develop new interventions (e.g., Carroll et al., 2014; Cuijpers, Donker, van Straten, Li, & Andersson, 2010). Reducing contact with a therapist is easy to conceive and execute, but the impact could affect several critical facets of treatment (e.g., therapeutic alliance). Moreover, reducing the contact when treatment is already delivered online may have an impact on other facets of care (e.g., adherence to the intervention). This study evaluated two variations of Internet-based treatment that differ in amount therapist contact and examined whether and how four domains were influenced: treatment outcomes, therapeutic alliance, client adherence to treatment, and client reactions to treatment.

Obviously, the first priority in reducing professional involvement with clients is assuring that there is no commensurate reduction in therapeutic change. Contact with a professional can serve many functions in treatment (e.g., providing information and encouragement, motivating change) and reducing that contact could diminish or eliminate the positive changes associated with an intervention. There are additional aspects of treatment beyond outcomes that are of interest because they contribute to therapeutic change and might readily influence the extent to which treatments with reduced contact (particularly those provided online) are adopted. To begin with, the therapeutic alliance is a core process of treatment that has been investigated in thousands of studies with adults and shown to predict and contribute to therapeutic changes (Horvath & Bedi, 2002; Norcross, 2011; Norcross & Wampold, 2011). Prior work in child and family therapy indicates that both parent and child alliance with the provider influence therapeutic change (Kazdin, Marciano, & Whitley, 2005; Kazdin, Whitley, & Marciano, 2006). Limiting contact might alter the client-therapist alliance (on which treatment outcome may partially depend). Indeed, contact with a therapist is part of the common factors of therapy and could influence therapeutic change (e.g., Lambert & Ogles, 2013; Wampold & Imel, 2015). One goal of this study was to evaluate whether the reduced contact with a therapist influenced therapeutic alliance and the relation between alliance and clinical change.

In addition to therapeutic change and therapeutic alliance, we were interested in the extent to which clients adhered to treatment and showed changes in behavior. The presence of a therapist in sessions might have a motivating influence on participating in therapy and carrying out treatment procedures. This is roughly analogous to exercise, which most people can do independently; the presence of someone else (e.g., personal trainer, friend) helps them carry out the exercise activities. We are not equating treatment administration with exercise but rather underscoring the important of relationships as motivating factors. Consequently, we were interested in evaluating whether reducing contact with a therapist and having clients work more on their own has a trade-off in adherence to treatment prescriptions and, perhaps as a result, diminished therapeutic change.

Finally, we were interested in client reactions to the treatment procedures. Treatment with reduced therapist contact might cause untoward reactions that limit the treatment as a viable option. That is, potential consumers may or may not be inclined to seek or accept treatments independently of the treatments’ effectiveness. Two domains of interest were treatment acceptability and barriers to participation in treatment. First, treatment acceptability refers to the extent to which consumers of treatment view the treatment procedures as reasonable, justified, fair, and palatable (Carter, 2007; Kazdin, 2000). Acceptability influences several critical aspects of the treatment process including the extent to which clients are likely to seek out treatment and adhere to treatment once they begin (Reimers, Wacker, Cooper, & DeRaad, 1992). Even if treatments with different levels of therapist contact were equally effective in therapeutic change, variation in treatment acceptability might limit the utility of reduced contact. Second, barriers to participation in treatment refer to how individuals view the appropriateness and relevance of the treatment to their clinical problem and the extent to which their expectations were met about the treatment (Kazdin, Holland, & Crowley, 1997). Perception of barriers increases the rates of cancelling and not showing up for sessions. Among clients who complete treatment, those who have perceived barriers to participation show less therapeutic change (Kazdin, 2000; Kazdin & Whitley, 2006a). Reducing contact with a trained professional also may be experienced as a barrier by families engaged in treatment, which has major implications for future use of treatments with limited or reduced therapist contact. In addition, Internet-based interventions may reduce or remove some barriers to treatment (e.g., transportation to a clinic) while introducing other barriers (e.g., technological problems).

In the present study, we evaluated variations of Internet-based treatment that differed in the amount of contact with a therapist. Treatment was evaluated for children referred for oppositional, aggressive, and antisocial behaviors, as encompassed by the diagnoses of conduct disorder (CD) and oppositional defiant disorder (ODD). These are among the most frequent bases for clinical referrals of children in the U.S., among the most costly of all mental disorders, and are precursors to many forms of adult psychopathology (e.g., Hill & Maugham, 2001; Nock, Kazdin, Hiripi, & Kessler, 2007). Several effective interventions have been developed and evaluated for the treatment of conduct problems (Kazdin, 2015). Among them is Parent Management Training (PMT), in which parents are trained to alter their children’s behavior in the home. Directly altering parenting practices (e.g., by reducing coercive interactions and increasing use of antecedents and consequences) has been effective in reducing conduct problems, as established in a number of research programs (Weisz & Kazdin, 2010).

Efforts to provide PMT to parents online are emerging (e.g., Comer et al., 2015). Written materials and videos also are available online and are associated with reductions in disruptive behavior (Enebrink, Hogstrom, Forster, & Ghaderi, 2012; Sanders, Baker, & Turner, 2012). Although Internet versions are not as well established as in-person treatment, the evidence is promising. Because Internet-based parenting programs have been delivered successfully, PMT is an ideal candidate treatment for the proposed adaptation (i.e., delivery online and with reduced contact with a therapist). Extending treatment to clinically referred samples may profit from better utilization of the limited mental health professionals who can provide services. We selected a clinically impaired sample to provide a strong test of whether treatments that vary in the amount of direct contact with a therapist are likely to vary in effectiveness. This study evaluated the efficacy of two versions of online PMT and specifically explored whether a version with significantly reduced contact with a therapist could achieve the same level of change as full contact with a therapist. In one version, PMT was administered directly by a therapist using videoconference software, and all treatment sessions included direct contact with the therapist (as is the case in traditionally administered PMT). In the other version of treatment, the amount of direct interaction that parents had with a therapist was dramatically reduced (by approximately 80%), as described later.

The purpose also was to evaluate four broad domains that are central to treatment. First, of course, we were interested in treatment outcomes (i.e., the extent of changes that were achieved by full or reduced treatment), which include both child and parent changes (e.g., stress, depression) evident in clinical trials for the treatment of children with conduct problems (Kazdin, 2010). Second, we evaluated the therapeutic alliance. The goal was to examine whether reducing contact with a therapist diminishes the quality of the alliance and whether the alliance-outcome relation varies as a function of amount of contact with the therapist. Third, we evaluated parent adherence to the treatment prescriptions. Reduced contact with a therapist might readily lead to less fidelity in carrying out procedures that lead to therapeutic change. Finally, we evaluated acceptability of treatment and barriers that parents experience in participating to treatment. These domains greatly influence participation in treatment and therapeutic change and do so independently of the alliance. By evaluating therapy in multiple domains (i.e., outcome, alliance, treatment adherence, and acceptability), we are able to examine comprehensively how variations of Internet-based PMT work and to identify the impact of reducing therapist contact on critical domains of treatment delivery and outcome. In addition to these primary goals, we were also interested in comparing the results from these Internet-based treatments to more traditional (i.e., in-person) treatment options. This study was a randomized control trial with two Internet-based groups that varied in the amount of direct therapist contact. As a complement to it, we evaluated treatment outcome against a matched sample of youth from the same clinical service who received in-person PMT. This later group merely who served as a benchmark for comparisons with the online sample collected in the study to determine if similar patterns of change were observed across treatment platforms.

Methods

Participants

Online treatment groups

In order to be eligible for participation in the study, children and their families were required to meet the following inclusion criteria: a) The identified child was between the ages of 6 and 12 years; (b) The child and his/her primary caregiver lived in the state of Connecticut; b) The family had computer and high-speed Internet access; c) At least one parent was fluent in English; d) The child was referred specifically for oppositional, aggressive, or antisocial behavior; e) The child has no history of cognitive impairment or significant developmental delay; and, f) At least one parent or legal guardian agreed to participate in treatment and provided informed consent. Children were excluded from participation if they met any of the following exclusion criteria: a) The child was enrolled or participating in another form of psychotherapy or counseling directed toward the reduction of social, emotional, or behavioral problems; psychiatric medication (e.g., prescribed stimulants for the treatment of ADHD) were allowed to continue as long as no changes were planned during the course of treatment; b) The child showed evidence of recent suicidality (i.e., within the last four months), had a current or lifetime diagnosis of Pervasive Developmental Disorder, or had a primary diagnosis of a mood or anxiety disorder; or, c) The family was actively in the midst of an acute crisis (e.g., custody battle, bankruptcy, major physical health problem) that would likely interfere with regular treatment participation.

Participants who consented for online treatment included 86 children and their caregivers. Children ranged in age from 6 years to 13 years old (M = 8.48, SD = 1.87). Fifty-eight percent of children were male (n =35), and 41.7% of the children were female (n = 25). The sample included children who were European American (86.7%), African American (5.0%), Asian American (5.0%), and bi- or multi-racial (1.7%). In terms of ethnicity, 8.3% (n = 5) identified as Hispanic or Latino/Latina while 88.3% (n = 53) were not Hispanic or Latino/Latina. Data related to ethnicity were unavailable for two participants.

Diagnoses of the children were obtained from the Research Diagnostic Interview (RDI; Kazdin, Siegel, & Bass, 1992), a structured parental interview that assessed the presence and duration of child symptoms. This measure was derived from the Schedule for Affective Disorders and Schizophrenia for School Aged Children (www.psychiatry.pitt.edu/node/8233) and has been used in prior work on both diagnosis and treatment (e.g., Kazdin & Whitley, 2003, 2006a). The diagnoses were updated based on the diagnostic criteria included in the Diagnostic and Statistical Manual of Mental Disorders, 4^th Edition, Text-Revision (DSM-IV-TR; American Psychiatric Association, 2000). Reliability of psychiatric diagnoses was assessed in the clinic setting by independent observers for over 100 randomly selected children at our service and yielded high agreement (κ = .95) across all diagnoses. Primary child diagnoses were ODD (61.7%), CD (33.3%), Attention–Deficit/Hyperactivity Disorder (ADHD; 3.3%), or did not meet criteria for a DSM-IV despite significant and impairing externalizing symptoms (1.7%). Most children (78.3%) met criteria for more than one disorder (M = 2.75, SD = 1.40, Range = 0 – 6 diagnoses).

The primary caregivers of the children included biological mothers (78.3%), adoptive mothers (13.3%), adoptive fathers (3.3%), or other relatives (e.g., biological father, grandparent; 5.1%), who ranged in age from 29 to 73 years (M = 44.03, SD = 7.45). Because nearly all of the caregivers identified themselves as parents, these two terms (caregivers and parents) will be used interchangeably in this manuscript. Most of the caregivers were married (81.7%) at the time of their participation in this study. Only 8.3% of children were living in single-caregiver households (i.e., without another adult family member or caregiver’s boyfriend/girlfriend). Most of the primary caregivers were employed either full-time (36.7%) or part-time (33.3%). Nearly half of the primary caregivers (48.3%) completed graduate-level education (e.g., doctoral degree), and an additional 28.3% had earned a college degree. Monthly income of the primary caretaker was evaluated on a 12-point scale, and ranged from $0-500 to over $8000 (median monthly income range = $2501-3000). Less than 2% of the primary caregivers in the sample received social assistance.

Because of interest in several process and outcome measures and their relations, only families who completed treatment were included in the study. Of the original sample of 86 enrolled children, 60 children completed treatment. This excluded 30.2% of enrolled children whose caregivers dropped out of treatment before completing 80% of the program. This attrition rate is similar to that noted in in-person intervention programs for children with conduct problems (e.g., Kazdin & Whitley, 2003). The number of families that dropped out of treatment did not differ between the two intervention groups (χ²(1) = 2.12, p = .17). Compared to treatment completers, dropouts were less likely to identify their race as European American (Parents: χ²(1) = 4.97, p = .03; Children: χ²(1) = 5.82, p = .02). They also had more children living in the home (2.68 vs. 2.12, t(82) = 2.71, p = .008). Families who dropped out did not differ from families that completed treatment in age of the identified child (t(84) = 0.46, p = .65), age of the primary caregiver (t(81) = 1.07, p = .29), or the number of adults living in the home (t(81) = 0.00, p = 1.00). The completers and dropouts did not differ in terms of children’s gender (χ²(1) = 2.72, p = .10) or ethnicity (χ²(1) = 0.36, p = .55) or in terms of primary caregivers’ marital status (married vs. not married; χ²(1) = 0.88, p = .35) or education level (college degree or higher vs. college degree not obtained; χ²(1) = 0.11, p = .74). In terms of clinical characteristics, children who completed treatment and who dropped out of treatment were no different with respect to number of symptoms of CD (t(84) = −0.46, p = .64), ODD (t(84) = −0.21, p = .84), ADHD (t(84) = 0.10, p = .92), and depression (t(84) = 0.94, p = .35). They were also similar in overall symptoms of internalizing (t(80) = 1.18, p = .24) and externalizing behaviors (t(80) = 0.51, p = .61) as measured by the Child Behavior Checklist. Please see Figure 1 for the flow of participants through the study. Detailed characteristics of dropouts, including risk factors for dropping out and treatment outcomes, are available elsewhere (see Kazdin, 2010).

Figure 1.

Flow of participants through screening, randomization, and treatment. Note: The first 20 participants to enroll were automatically assigned to a treatment group and not randomly assigned. Twelve of these participants completed treatment, with 6 in Full Contact PMT and 6 in Reduced Contact PMT.

Benchmark comparison group

A sample of families (N = 60) who completed traditional (i.e., in-person) treatment was selected from a large database compiled through several past RCTs (e.g., Kazdin & Durbin, 2012; Kazdin & Whitley, 2006a). This sample served as a benchmark for comparisons to determine if similar changes in relevant child functioning and parenting variables are observed in online and in-person treatment samples. Participants were selected to match the online sample (N = 60) in terms of age and race. Specific gender-based matching was not possible due to the relatively small number of potential female participants in the benchmark pool. Children in the benchmark sample ranged in age from 6 years to 13 years old (M = 8.81, SD = 1.71). Approximately 77% of children were male (n =46), and 23% of the children were female (n = 14). In terms of race and ethnicity, the sample included children who identified as White (73.3%), Hispanic (6.7%), African American (15.0%), Asian American (1.7%), Native American (1.7%), and bi- or multi-racial (1.7%). When compared to the participants in the online sample, children in the online sample did not differ in terms of age (t(118) = < 1, ns). In addition, there were no significant differences between the sample in terms of gender (χ²(1) = 4.60, p = .05) or in terms of race (proportion of the sample who identified as White compared to the proportion of the sample who identified as a different race or ethnic group; χ²(1) = < 1, ns ).

Assessments

The overall goals of assessment were to evaluate four domains: treatment outcome, therapeutic alliance, treatment adherence, and client reactions to treatment. Assessments included multiple assessment formats (e.g., clinical interviews, parent-report questionnaires) and informants (i.e., parents and therapists). Measures were completed at three different time points: before treatment began, halfway through treatment, and after treatment ended (pre-, mid-, and posttreatment, respectively). For each child enrolled in the study, one parent completed all of the assessments (i.e., the pretreatment, midtreatment, and posttreatment assessments). In every case, this was the same parent who was actively engaged in PMT because several of the questions included in the assessments related specifically to the skills learned in treatment and their experiences with and reactions to the treatment program. The therapist responsible for the case administered diagnostic interviews both at intake and at the end of treatment to determine the presence of psychiatric symptoms and diagnoses. At intake (i.e., pretreatment), the therapists administered the interview by telephone for all families. At posttreatment, the therapists administered the interview by either telephone or videoconference (depending on the family’s assigned treatment program). In addition to the pretreatment diagnostic interview, parents completed a General Information Form to gather child and caregiver demographic information and to assess family characteristics. Most parent- and therapist-report measures were collected online using a secure web-based survey program protected for privacy with unique links for each client.

Treatment outcome

Two broad outcome domains were evaluated to assess the impact of the two variations of treatment. Of course, the primary outcome focus was on child functioning given that is the basis for clinically referring children to our clinic. In addition, we evaluated parent outcome domains because prior work with in-person treatment has consistently shown improvements in clinical domains among parents (Kazdin, 2010).

Child treatment outcomes

To evaluate child outcomes, we assessed child symptoms and dysfunction using three specific measures in addition to the diagnostic information provided through the RDI. First, to assess a broad range of both internalizing and externalizing symptoms, parents completed the Child Behavior Checklist (CBCL; Achenbach, 1991). This measure includes 118 items (each rated on a 0- to 2-point scale) that comprise multiple behavior problems. The total problem score was evaluated to assess severity of dysfunction across a broad range of symptom domains. This measure is especially relevant to the present sample because of the high rates of comorbid disorders and symptoms spanning multiple diagnoses. The total competence scale was evaluated to assess participation in activities (e.g., athletics, clubs), social interactions (e.g., number of friends, amount of contact with friends), and academic performance. Multiple forms of reliability and validity of the CBCL have been studied extensively in clinic and non-clinic samples (e.g., Achenbach, 1991). The CBCL was administered before, during, and at the end of treatment.

Second, parents completed the Interview for Antisocial Behavior (IAB; Kazdin & Esveldt-Dawson, 1986; Kazdin et al., 1992), a semi-structured parent-report form that measures multiple overt (e.g., fighting) and covert (e.g., lying) antisocial behaviors of the child. This measure reflects primary symptoms for which children are referred to the clinic (i.e., conduct problems). Each of the 30 items is rated on a 5-point scale for severity of dysfunction (1 = not a problem at all, 5 = very much a problem) and a 3-point scale for duration (1 = recent or new problem [6 months], 3 = always). Total antisocial behavior is obtained by summing severity and duration scores. Internal consistency, convergent and discriminant validity, and overall construct validity have been supported in multiple studies (e.g., Kazdin & Esveldt-Dawson, 1986; Kazdin et al., 1992). Parents in this study completed the IAB before and at the end of treatment.

Third, the therapist responsible for the case completed the Child Global Assessment Scale (CGAS; Shaffer et al., 1983) to provide an overall measure of impairment and functioning. This measure summarizes the child’s psychiatric, adaptive, and social functioning in everyday life and consists of a single number between 1 and 100 (with a higher number indicating better functioning). High inter-rater reliability and concurrent and discriminant validity have been demonstrated for the CGAS (e.g., Bird, Canino, Rubio-Stipec, & Ribera, 1987). Impairment was assessed because it contributes independently to whether a child is referred for treatment (Bird et al., 1990; Sanford, Offord, Boyle, Peace, & Racine, 1992). Therapists completed the CGAS before and after treatment.

Finally, as mentioned previously, the presence of DSM-IV-TR psychiatric symptoms was assessed through the RDI, which provides formal diagnoses. The measure was also used to evaluate treatment outcome. The number of diagnoses in a given child (i.e., comorbidity) has a restricted range (0 to 6 in the present samples) and ignores clinically important information, namely, symptoms a child shows across multiple disorders (whether or not the child meets the full criteria for the other disorders). Consequently, to reflect treatment outcome, we counted the total number of symptoms present across all diagnoses. This number represents DSM-IV-TR symptoms for which there was significant impairment in everyday life and correlates with (but is distinguishable from) number of diagnoses (r(55) = 0.86). Prior research has attested to the utility and validity of number of symptoms as a measure of therapeutic change (Kazdin & Whitley, 2006a).

Parent and family treatment outcomes

Therapeutic changes of the parent and the family were assessed by measures that focused on parent psychopathology, perceived stress, parenting practices, and family relationships. Each of these domains reflects significant improvement after PMT even though none of them is focused on directly during treatment (see Kazdin, 2010). First, parents completed the Beck Depression Inventory (BDI; Beck, Steer, & Garbin, 1996) before and at the end of treatment. For each of 21 items, the parent selected one of several statements that best described their experience of a depression symptom; a higher score on the measure indicates more symptoms and greater severity of depression. The psychometric properties of the BDI have been studied extensively (e.g., Dozois, Dobson, & Ahnberg, 1998).

Second, parent perceptions of stress were assessed through the Parenting Stress Index (PSI; Abidin, 1990; Lloyd & Abidin, 1985), which was completed before and at the end of treatment. The PSI consists of 120 items, most rated on a 5-point scale, that reflect multiple areas of stress related to the parents’ views of their own functioning. The measure assesses perceived sources of stress, delineates perceived stress from life events, and distinguishes sources of stress from the child (e.g., subscales such as adaptability, demandingness, and child mood) and sources of stress related to the parent functioning (e.g., subscales such as restrictions of role, social isolation, and relations with others). Diverse types of reliability and validity for the PSI have been reported (Abidin, 1990).

Third, a shortened version of the Family Environment Scale (FES; Moos & Moos, 1981) was completed by parents before and at the end of treatment. The three subscales of the FES relationship domain (cohesion, expressiveness, and conflict) were used in this study. Parents responded to 27 true-false items that assess quality of interpersonal relationships, support, and family functioning. The FES has been studied extensively, and multiple types of reliability and validity have been demonstrated (Moos, 1990).

Therapeutic alliance

A central feature of the study was evaluation of the alliance and whether this varied between the two treatments and if any outcome differences might be explained by such differences. The Parent-Therapist Relationship Scales (PTRS) assess characteristics of the therapeutic alliance, such as whether the client and therapist are collaborating and working together and whether they are experiencing mutual respect, understanding, and trust in their relationship. The scale was adapted from the Working Alliance Inventory (Horvath & Greenberg, 1989). In keeping with that scale, the version we used includes 36 items rated on a 7-point scale (1 = never; 7 = always). Items focus on the therapist–parent agreement on the tasks and their relevance in therapy; the mutual endorsement of the goals or outcomes of treatment; and the extent to which there is a positive personal attachment, acceptance, and confidence in the relationship. Parent and therapist versions of the scale are identical in content and format. The items on each measure are summed to provide a total score; higher scores reflect a higher quality therapeutic alliance. Prior work has evaluated the therapeutic alliance in separate studies and has shown that alliance is related to changes in parenting practices and therapeutic change (Kazdin & Whitley, 2006b; Kazdin, Whitley, & Marciano, 2006). These studies also provide data on the reliability and validity of this measure. In this study, parents and therapists completed the PTRS during and at the end of treatment.

Parent adherence and execution of parenting procedures

We evaluated whether reduced contact with the therapist would diminish the extent to which parents adhered to critical practices in parenting that are designed to change child behavior in the home. The Treatment Adherence Inventory (TAI; Kazdin & Whitley, 2006a) includes items that reflect the extent to which the parent complies with specific treatment proscriptions related to PMT, parental willingness to try out behavior-change techniques at home, and general receptivity of the parent to treatment. The TAI was completed by therapists (who rated the parents) and by parents (who rated themselves) during and after treatment in case adherence varied significantly at these different points and differentially predicted therapeutic change. A total of 15 items were included at midtreatment assessment, and 18 items were included at posttreatment to reflect additional skills taught in the sessions during the time between mid and the end of treatment.

PMT is designed to change parenting practices in the home. To evaluate changes in practices, parents completed the Management of Children’s Behavior Scale (MCBS; Perepletchikova & Kazdin, 2004) before and at the end of treatment. This measure assesses a broad range of parenting practices that are relevant to conduct problems (e.g., harsh disciplinary practices, inconsistent punishment techniques, praise for prosocial behaviors). The MCBS consists of 46 items that describe parenting practices, and parents select one of three responses to describe their own parenting behavior. The MCBS detects changes in parenting practices after PMT. High levels of internal consistency and diverse types of validity have been demonstrated with this scale (Perepletchikova & Kazdin, 2004).

Treatment acceptability and barriers to treatment

Client reactions to the treatment procedures were evaluated during and at the end of treatment. Perceived acceptability of the treatment (including both specific clinical techniques and the technology used to deliver treatment) was assessed through three measures. Parents and therapists evaluated the acceptability of treatment and the progress made in treatment through the Parent Evaluation Inventory (PEI) and the Therapist Evaluation Inventory (TEI), respectively (Kazdin et al., 1992). These measures each include 19 items and assess the extent to which the treatment was viewed positively by the parents. Parents rated their own perception of the treatment, and therapists rated the parents’ perception of treatment acceptability. Both the PEI and TEI have shown high levels of reliability and validity and are able to discriminate among treatments and reactions to them (e.g., Kazdin, 2000; Kazdin, French, & Sherick, 1981).

To evaluate acceptability of the technology used to deliver treatment, parents completed the Acceptability of Treatment Modality (ATM) scale for their specific treatment condition (i.e., there were two slightly different versions of the ATM, one for each of the two treatment conditions). The ATM was developed specifically for this study as a way to evaluate if clients’ perceptions of therapy are affected by the version of online PMT that they received. The measure includes 13 items that evaluate facets of overall acceptability and barriers that may affect participants receiving online treatment specifically. For each item, parents are asked to select a response on a 5-point scale (from strongly disagree to strongly agree). The measure is composed of several questions adapted from the User’s Perception of Telepsychotherapy (UPT; Bouchard et al., 2000) and several questions generated specifically for this project.

Barriers to participation in treatment were measured by the Barriers to Treatment Participation Scale (BTPS; Kazdin et al., 1997), a questionnaire completed by parents. Parents evaluated the extent to which various stressors and obstacles interfered with treatment as well as perceptions about the relevance of treatment. The measure included 41 items rated on a 5-point scale. Prior studies have shown that barriers predict responsiveness to treatment and that perceived barriers are not explained by other domains (e.g., parent psychopathology, life events; Kazdin et al., 1997; Kazdin, Holland, Crowley, & Breton, 1997). We were interested in whether barriers varied as a function of amount of contact with a therapist and if so whether barriers would portend diminished therapeutic change outcome effects.

Treatment

Two variations of treatment

Different Internet-based treatments were provided to evaluate the effects of greatly reducing the amount of professional therapist time that was provided. Traditionally, PMT is provided in-person, with parents traveling to a clinic for 12 weekly sessions (approximately one hour each; Kazdin, 2005). Practice, feedback, and shaping are used within the sessions to develop parental skills and specific behavior-change programs. The program includes a core set of sessions to convey central content areas, themes, and skills. Occasionally, additional sessions (e.g., 1-3) are provided to address a specific problem or theme that was not conveyed well in a core session or if a complete session could not be provided. In addition, therapists have telephone or email contact with parents to provide specific feedback on implementing the program and to provide support and encouragement. Both of the treatment groups received this intervention presented online instead of the traditional in-person version of the treatment.

Most families (N = 66) were randomly assigned to one of two groups: Full Contact PMT and Reduced Contact PMT. Forty-eight of these families completed treatment and were included in the analyses. The first 20 participants who entered the study were assigned non-randomly to one of the two treatment groups. This occurred because of a technological issue that arose with one of the groups (Reduced Contact PMT), which prevented enrollment in that group for a brief period of time. After the problem was addressed, several participants were non-randomly assigned specifically to Reduced Contact PMT to balance the enrollment across the two groups. This resulted in 6 participants who completed Full Contact PMT without random assignment and 6 participants who completed Reduced Contact PMT without random assignment. Because their assignment was unrelated to characteristics of the child/family and was based exclusively on a technological issue, we have no reason to believe that these families differed from the families who were randomly assigned.

Both Full Contact PMT and Reduced Contact PMT included 8 core treatment sessions. The groups varied in the specific ways technology was used to provide treatment and the intensity of therapist involvement (i.e., time and resources required). Full Contact PMT was structured very similarly to traditional (i.e., in-person) PMT. However, instead of traveling to a treatment center and meeting with a therapist in person, the parent and therapist “met” using a webcam and a web-based videoconference program. This program allowed the parent and therapist to see, hear, and interact with each other in real time over a secure and encrypted Internet connection. Therefore, although treatment was provided through the Internet, parents and therapists engaged in face-to-face interactions. As prescribed by traditional PMT, therapists had regular telephone and email contact with families to address any questions that arose outside of the scheduled meetings and to allow therapists to respond to the individual needs of each family.

Reduced Contact PMT did not involve any face-to-face interactions with a therapist. Each week, parents were emailed a link to a pre-recorded treatment session. This session was recorded using the same videoconference software used for the delivery of Full Contact PMT, resulting in screen images and viewing experiences that were quite similar across the two groups. The recordings consisted of the assigned therapist presenting the same material that was presented in Full Contact PMT sessions. The therapists used scripts based on the specific treatment manual used for Full Contact PMT; the scripts used by each therapist were identical. Because of limitations on interactions with the recorded sessions, professionally taped role-plays were played for parents instead of parents participating in role-plays with the therapists. Parents gained access to the link for a new session on a weekly basis and were able to re-watch previously viewed session if desired (i.e., parents could access information from previous weeks). In addition to the recorded session, parents had regular telephone contact with the assigned therapist. The telephone contact served to directly address questions or concerns that arose as parents implemented the treatment program and to allow therapists to respond to the individual needs of each family. Therapists scheduled and initiated a 15-20 minute telephone call with parents every two weeks. Like Full Contact PMT, parents in Reduced Contact PMT were allowed to call and email their therapists at any time (i.e., outside of the scheduled telephone check-ins) with additional questions. Therefore, parents in the Reduced Contact group had contact with a therapist but this contact was not face-to-face and, instead, occurred through viewing pre-recorded sessions with that therapist, telephone contact, and email contact. Overall, the amount of direct therapist contact time was dramatically reduced in the Reduced Contact PMT group: the scheduled weekly contact with a therapist was approximately 50 minutes for participants in Full Contact and was approximately 10 minutes for participants in Reduced Contact PMT participants in the Full Contact. Therapists documented the length of their sessions (for Full Contact PMT) and telephone check-ins (for Reduced Contact PMT) and monitored their email and telephone contact with families to ensure that they were not unknowingly spending more time with one treatment group.

Therapists and treatment administration

Two clinicians (both women and both European American) served as therapists. Each of these therapists had a master’s degree in a mental health-related field (e.g., psychology, social work) and at least 10 years of experience in delivery of PMT. Their background in PMT included at least 6 – 12 months of intensive training with the principle investigator, which involved extensive role-playing and modeling to master PMT treatment techniques. Since that initial training, the therapists have participated in weekly PMT supervision of cases and ongoing support in the application of the intervention. For this study, all treatment cases were closely supervised using direct observation, review of tapes, and discussion of the case on a session-by-session basis. Throughout the study, treatment sessions for Full Contact PMT were recorded for supervision and review. Sessions for Reduced Contact PMT were pre-recorded and the exact same set of sessions was provided to all families in the group. The same set of therapists provided treatment to both groups (Full Contact PMT and Reduced Contact PMT). The goal of the study was to provide the same treatment skills but to do so through different platforms. Therefore, issues related to contamination of treatment were not concerning as it was an explicit goal for both groups to receive the same treatment content. The therapists monitored the amount of contact they had outside of scheduled telephone check-ins and sessions to ensure that they were not unknowingly providing more contact than intended to either group.

Treatment integrity

To maintain integrity of treatment in Full Contact PMT: (a) therapists followed a treatment manual that delineated the content and focus of each session; (b) materials were provided to foster correct execution of the treatment, including checklists that prescribed the necessary materials (e.g., charts, tasks to be covered) as well as notes and outlines for use within the sessions; (c) documentation of the session summarized what transpired and how the child or parent progressed; (d) all treatment sessions were digitally recorded, and some of them were reviewed weekly to provide feedback to the therapists; (e) all cases were reviewed weekly in supervision to identify the current status of treatment, including what transpired in the previous session, what was planned in the upcoming session, and whether there were any special issues that would influence treatment delivery; and, (f) a subset of the digitally recorded sessions were reviewed by research assistants, naïve to the overall study, to determine the extent to which the specified set of activities and procedures were included in the session (see Results section for more information on this). For families in the Reduced Contact PMT, the use of pre-recorded sessions eliminated the need for many of these procedures. However, specific procedures were put in place to maintain the integrity of the material reviewed in the telephone check-ins. Therapists were given specific materials (including checklists) to ensure that the necessary information was included in the check-ins, and all cases were reviewed weekly in supervision.

Results

Preliminary Analyses

Participant characteristics

Preliminary analyses using t- and chi-square tests for continuous and categorical variables revealed no differences between the two intervention groups in demographic variables and pretreatment clinical characteristics including child or caregiver age, number of children or adults in the home, child or caregiver race or ethnicity, marital or employment status, or educational level. There were no differences between the groups in terms of internalizing or externalizing symptoms, child antisocial behavior, parental depression, family relationships, or stress. The use of parenting techniques differed between the groups at pretreatment (t(58) = −2.00, p = .05). This difference is addressed later. Overall, the groups were not significantly different among key demographic characteristics as well as measures of clinical dysfunction before treatment began.

Treatment administration and treatment integrity

Several procedures were in place to maintain integrity of the treatments. As part of that, sessions in the Full Contact PMT group were recorded and trained research assistants naïve to the hypotheses reviewed the session to ensure whether the key skills and information schedule for the specific session were included and delivered to the parent. A total of 154 treatment sessions were reviewed by 15 raters; 140 of these sessions (90%) were rated independently by two raters. Any discrepancies were resolved by a supervisor who trained the raters. Across both therapists and all raters, mean treatment integrity was 93% (range = 83 – 98%). This reflects the percentage of procedures and activities within the treatment sessions that were carried out as intended.

In both treatments, therapists and parents were in regular contact by email and telephone (e.g., telephone calls, voicemails). These contacts served a variety of purposes, including checking in with participants about implementation of a new skill and rescheduling missed or cancelled appointments. The contacts were a routine part of treatment and are included as part of traditionally administered (i.e., in-person treatment) PMT. Because the extent of contact was a key facet of the study (Reduced Contact vs. Full Contact), we assessed all contacts between therapists and parents. Specifically, we compared the extent to which telephone contact (including both telephone calls and voicemails) varied between the two groups. The number of telephone calls with parents and the number of voicemails left for parents were tracked and combined, and this number was compared between the two groups. There were no differences between the groups in terms of telephone communication (t(58) = 1.52, p = .13). Therapeutic Change

Child treatment outcomes

The primary domain to evaluate the two different versions of treatment was treatment outcome. A series of analyses were conducted to determine if children improved over the course of treatment. Within-group t-tests were computed for the entire sample (N = 60) to evaluate these changes over time (i.e., from pre- to posttreatment). The means for pretreatment and posttreatment scores on child, parent, and family outcome measures are presented in Table 1 for the entire sample. Within-group t-tests indicate that the children improved over the course of treatment (see Table 1) as reflected in significantly lower internalizing and externalizing problems (CBCL), reduced aggressive and antisocial behavior (IAB), improved overall adaptive functioning (CGAS), and fewer endorsed symptoms in the RDI. In short, children in both treatment groups improved.

Table 1.

Measures of Child, Parent, and Family Treatment Outcomes for the Entire Sample (N = 60)

	Pretreatment		Posttreatment
Measure	M	SD	M	SD	t(59)	d
Child Functioning
CBCL Internalizing Symptoms	60.18	12.64	54.23	12.88	6.73*	0.88
CBCL Externalizing Symptoms	64.78	10.07	57.72	10.98	8.05*	1.06
CBCL Total Problem Score	64.12	11.13	57.50	11.41	8.13*	1.04
IAB Total Score	121.16	34.94	104.21	35.29	6.19*	0.78
CGAS	58.08	9.07	70.39	11.85	6.97*	0.92
RDI Total Symptoms	34.40	11.50	26.09	13.23	8.36*	1.14
Parent and Family Functioning
BDI Total Score	7.67	7.72	5.52	6.42	4.31*	0.60
PSI Total Score	255.34	51.25	233.22	53.56	6.36*	0.83
FES Relationship Total Score	7.65	4.63	9.61	4.73	4.16*	0.57

Notes. CBCL = Child Behavior Checklist, IAB = Interview for Antisocial Behavior, CGAS = Child Global Assessment Scale, RDI = Research Diagnostic Interview, BDI = Beck Depression Inventory, PSI = Parenting Stress Index, FES = Family Environment Scale. Improvements in most functioning measures are based on reduction of scores (e.g., lower symptoms). However improvements on the CGAS and FES reflect an increase in scores (e.g., improved functioning, improved relationships). The signs of the t tests for these two measures were reversed so that all tests in the table reflect improvement. The dfs for all analyses were 59 with two exceptions (53 and 58 where complete data were not available for the measures).

^*p < .001

Additional analyses were conducted to determine if there were differences in these outcome variables for the two treatment groups (i.e., Full Contact PMT vs. Reduced Contact PMT). A series of two-way repeated measures analysis of variance (ANOVA) were run for each of the child functioning variables, with treatment group (Full Contact PMT and Reduced Contact PMT) as the between-subjects variable and time (pre- and posttreatment) as the within-subjects variable. The means for pre- and posttreatment scores on child outcome measures are presented in Table 2 for each treatment group along with F-ratios (for the treatment group x time interaction) for each outcome. A notable difference emerged between the groups: children in the Full Contact PMT group had lower levels of internalizing symptoms compared to children in the Reduced Contact PMT group at posttreatment (see Table 2). There were no significant differences on any of the other child outcome measures. Overall, the consistent finding is improvements for both groups and not strong support for one version surpassing the effects of the other.

Table 2.

Measures of Child, Parent, and Family Treatment Outcomes for Each Treatment Group

	Full Contact PMT				Reduced Contact PMT
	Pretreatment		Posttreatment		Pretreatment		Posttreatment
Measure	M	SD	M	SD	M	SD	M	SD	F
Child Functioning
CBCL Internalizing Symptoms	61.10	12.85	53.26	14.26	59.21	12.64	55.23	12.88	5.23‡
CBCL Externalizing Symptoms	66.61	7.73	58.45	8.70	62.83	11.93	56.93	13.10	1.68
CBCL Total Problem Score	65.32	10.63	57.68	10.63	62.83	11.68	57.31	12.38	1.73
IAB Total Score	128.61	35.74	109.70	34.13	113.19	32.82	98.35	35.29	0.55
CGAS	58.00	8.48	71.83	10.05	58.17	9.80	69.90	13.48	0.77
RDI Total Symptoms	33.73	12.30	25.60	12.21	35.15	10.73	26.63	13.23	0.01
Parent and Family Functioning
BDI Total Score	7.97	7.94	5.61	6.03	7.34	7.59	5.41	6.91	0.18
PSI Total Score	254.29	51.97	237.21	52.22	256.46	51.36	228.96	55.56	2.29
FES Relationship Total Score	7.16	4.65	9.30	4.07	8.08	4.65	9.87	5.29	0.13

Notes. CBCL = Child Behavior Checklist, IAB = Interview for Antisocial Behavior, CGAS = Child Global Assessment Scale, RDI = Research Diagnostic Interview, BDI = Beck Depression Inventory, PSI = Parenting Stress Index, FES = Family Environment Scale. Improvements in most functioning measures are based on reduction of scores (e.g., lower symptoms). However improvements on the CGAS and FES reflect an increase in scores (e.g., improved functioning, improved relationships). The signs of the t tests for these two measures were reversed so that all tests in the table reflect improvement. The dfs for all analyses were 1,58, with two exceptions (1,52; 1,57 where complete data were not available for the measures).

^‡Interaction term significant at p = .03.

Clinical impact of treatment on the children

A critical question is the extent to which the treatment produced clinically important changes. Although there is no standardized way to assess clinical significance in outcome research, one means is to evaluate the extent to which treatments brought child behavior within the nonclinical range of functioning (Kazdin, 2001). We drew on data from the well- studied CBCL, mentioned previously, which provides cutoff scores that fall above and below a normative range, as devised from comparisons of clinic and non-referred samples separately for boys and girls. Using that range, we evaluate the extent to which treatment placed children within the normative range at the end of treatment. We examined both the total symptom score and the total competence score. For the total behavioral problems scale, 39% and 48% of children in the Full and Reduced PMT, respectively, fell within the normative range at the beginning of treatment. By the end of treatment, 71% and 65% of children in the Full and Reduced PMT, respectively, fell within the normative range. There were not differences between the two groups in number of participants that fell within the normative range at the end of treatment (χ²(1) = < 1, ns). For the total competence scale, 36% and 18% of children in the Full and Reduced PMT, respectively, fell within the normative range at the beginning of treatment. At the end of treatment, 69% and 78% for these same treatments, respectively, fell within the normative range. There were not differences between the two groups in number of participants that fell within the normative range at the end of treatment (χ²(1) = < 1, ns). By the end of treatment, most children in both treatment groups were functioning within in the typical range with respect to symptoms of psychopathology and social engagement. Moreover, the proportion of children that were characterized as functioning in the normative range was similar for both treatment groups.

Parent and family outcomes

Because PMT also is associated with significant changes in parental dysfunction and stress as well as changes in relationships and interactions among family members, outcomes related to parent and family functioning also were evaluated. As with the child outcome data, within-group t-tests were computed for the entire sample (N = 60) to evaluate these changes over time (i.e., from pre- to posttreatment). These data are summarized in Table 1. After treatment, parents in the sample reported fewer symptoms of depression in themselves, reduced stress levels, and improved relationships among family members.

In order to explore differences in parent and family outcomes based on treatment group, two-way repeated measures ANOVAs were run for each of the parent and family functioning variables, with treatment group (Full Contact PMT and Reduced Contact PMT) as the between-subjects variable and time (pretreatment and posttreatment) as the within-subjects variable. None of the treatment group x time interactions was significant, indicating that patterns of change in parent and family functioning were similar across the two treatment groups. Please see Table 2 for a summary of the results for all variables.

Therapeutic Alliance

A central focus was whether alliance would vary as a function of the amount of contact with the therapist. In this section, we report on the main comparison of whether alliance varied between the two treatment conditions. Parents and therapists independently completed the alliance measure at mid- and posttreatment. Parent ratings of alliance at mid- and posttreatment were highly correlated across the whole sample (r(58) = 0.75, p < .001) and in each treatment condition (Full Contact PMT: r(29) = 0.84, p < .001, Reduced Contact PMT: r(27) = 0.67, p < .001). Similarly, therapist ratings of alliance at midtreatment and posttreatment were strongly correlated across the whole sample (r(58) = 0.86, p < .01) and for each of the treatment conditions (Full Contact PMT: r(29) = 0.91, p < .001, Reduced Contact PMT: r(27) = 0.85, p < .001). Separate analyses showed that mid- and posttreatment assessments for parents and therapists did not yield different results. Consequently mid- and posttreatment alliance scores for parents were combined into one score; the scores for therapists also were combined, yielding two alliance scores (Parent Alliance: M = 445.24, SD = 32.99; Therapist Alliance: M = 439.27, SD = 38.69).

The two treatments were compared to evaluate whether alliance differed for the two interventions. Parents and therapists did not differ in their ratings of the therapeutic alliance for the Reduced and Full Contact treatment conditions (Parent Alliance: t(58) < 1, ns; Therapist Alliance: t(58) < 1, ns). Interestingly, the alliance between the therapist and client did not vary as a function of frequent versus reduced contact with the therapist.

Parent Adherence and Execution of Parenting Procedures

A key interest in the study was whether amount of contact with the therapist would influence how well parents carried out the intervention learned and practiced in the sessions were carried out at home. Both parents and therapists rated parental use of the techniques taught in session. These ratings were provided at time points: mid- and posttreatment. Parent ratings of adherence at mid- and posttreatment were strongly correlated across the whole sample (r(57) = 0.60, p < .001) and for each treatment condition (Full Contact PMT: r(29) = 0.76, p < .001, Reduced Contact PMT: r(26) = 0.41, p = .03). Similarly, therapist ratings of adherence at mid- and posttreatment were correlated across the whole sample (r(57) = 0.53, p < .001) and for each of the treatment conditions (Full Contact PMT: r(28) = 0.44, p = .02, Reduced Contact PMT: r(27) = 0.52, p = .004). In light of these correlations, we combined mid- and posttreatment measures of adherence to have the total parent adherence score separately for parents and therapists. The total parent adherence scores and total therapist adherence scores were moderately correlated (r(57) = 0.58, p < .001). Parents in the two treatment groups did not differ in their self-reported adherence to treatment (t(57) = 1.61, p = .11). However, therapists rated parents in the Full Contact PMT as more adherent to treatment than parents in the Reduced Contact group (t(57) = 3.06, p = .003, d = 0.77), indicating that therapists perceived parents in the Full Contact group as using the skills more frequently and more consistently than parents in the Reduced Contact group. Parent adherence scores were not correlated with changes in child behavior problems (as measured by the CBCL total problem score; (r(57) = −0.03, p = .82). Therapist adherence scores were not correlated with changes in child behavior problems (as measured by the CBCL total problem score; (r(57) = 0.12, p = .38).

Parents also rated their use of a broad range of treatment-related parenting practices through the MCBS. This measure includes parenting behaviors that often change over the course of PMT (e.g., praise, harsh discipline, use of corporal punishment) and differences between the two treatment groups may suggest different degrees of treatment adherence. An analysis of covariance (ANCOVA) was run with treatment group (Full Contact PMT and Reduced Contact PMT) as the fixed variable and posttreatment MCBS score at the dependent variable. Because of the pretreatment group differences noted earlier, pretreatment MCBS score was entered as a covariate in the ANCOVA. There was no significant difference between the groups on their posttreatment MCBS score after controlling for pretreatment score (F(1,57) < 1, ns), suggesting that the pattern of change over time did not differ between the two groups. These MCBC findings are consistent with the parent-reported adherence data reported earlier. Regardless of the measure, parents in both groups perceived themselves as similar in their implementation of PMT skills and techniques.

Treatment Acceptability and Barriers to Treatment

We evaluated whether parents would view the treatment differently and along lines known to influence adherence and participation in treatment as well as therapeutic change. Measures of acceptability and barriers to treatment participation were evaluated. A total of three measures focused on perceived acceptability of the treatments. Two measures assessed the acceptability of the specific PMT skills taught in each of the treatment groups; one of these measures was completed by the parent (PEI), and one was completed by the therapist (TEI). Parent and therapist evaluations of treatment acceptability were moderately correlated across the entire sample, r(58) = 0.49, p < .001. The correlation of parent and therapist evaluations were similar for Full Contact PMT (r(29) = 0.35, p = .05) and Reduced Contact PMT (r(27) = 0.50, p = .005). Parents in the Full Contact PMT rated treatment to be more acceptable than parents in the Reduced Contact PMT group (PEI: t(58) = 2.06, p = .04, d = 0.53). Therapists’ evaluations of acceptability reflected the same pattern; they rated parents in the Full Contact PMT group as having more positive views of treatment and being more engaged in the treatment program than parents in the Reduced Contact PMT group (TEI: t(58) = 2.44, p = .02, d = 0.63). The overall finding and consistencies between parents and therapists are noteworthy by suggesting, independently of therapeutic change and outcome, more in-person contact made the treatments in the present study more acceptable.

A third measure of acceptability (ATM) focused on parental reactions to the use of technology to deliver the interventions. Data were collected from parents at mid- and posttreatment. At both time points, parents rated the treatments as highly acceptable. The range of ATM is 13 to 65, with a higher score indicating a more acceptable treatment. At midtreatment, acceptability ratings ranged from 33 to 64 for the whole sample (Full Contact PMT: Range = 44 – 64, M = 54.64; Reduced Contact PMT: Range = 33 – 62, M = 54.09). At posttreatment, acceptability ratings ranged from 39 to 64 for the whole sample (Full Contact PMT: Range = 39 – 64, Mean = 54.46; Reduced Contact PMT: Range = 41 – 63, Mean = 54.63). The mean scores for both groups at mid- and posttreatment indicate that the treatment was viewed as acceptable for both of the intervention groups. None of the participants (0%) in either Full Contact PMT or Reduced Contact PMT rated the treatment as low in acceptability. At both mid- and posttreatment, parents in the two treatment groups provided similar acceptability scores, suggesting that the different delivery formats were both viewed as acceptable (Midtreatment: t(54) = 0.36, p = .72, ns; Posttreatment: t(54) = − 0.10, p = .92). At posttreatment, the sample (N = 60) was divided into two groups based on their acceptability ratings. Participants with scores below the median (Median = 54.00) were classified as moderate acceptability raters (n = 29) while participants with scores equal or greater than the median were classified as high acceptability raters (n = 31). The proportion of moderate and high acceptability raters did not differ for Full Contact PMT and Reduced Contact PMT (χ²(1) = 0.00, p = .99). Overall, these results suggest that participants across both treatment groups experienced the interventions as highly acceptable and that these rating did not differ between the two groups.

Parents completed a measure to assess their perceptions of barriers that may interfere with the treatment process. Reduced contact with the therapist might well increase the perceived barriers to treatment. However, total scores on the barriers measure were similar for the two treatment groups at the end of treatment (t(58) = −1.03, p = .31). Participants in the groups did not differ reliably in their experiences of obstacles (e.g., scheduling difficulties, childcare problems) that might interfere with treatment engagement.

Supplementary Analyses

Relations among critical domains

Apart from evaluations of treatment outcome, participation, acceptability, and emergent processes, we used this opportunity to explore several relations among these domains, many of which are well studied in traditional (i.e., in-person) psychotherapy research.

The therapeutic alliance and treatment outcome often are related, even though the relation is relatively small (e.g., r = .27; Horvath et al., 2011). To explore the relation with online treatment, we correlated therapeutic change scores on internalizing and externalizing behavior problems (as measured by the CBCL’s total problem score) with parent-rated therapeutic alliance score (i.e., the combined mid- and posttreatment alliance scores mentioned previously) separately for each treatment group. The correlations between child behavior problems and parent-rated therapeutic alliance were low for both the Full Contact PMT group (r(29) = 0.05, p = .79) and the Reduced Contact PMT group (r(27)= 0.27, p = .15). These correlations were not significantly different from each other (Fisher’s direct test, z = 0.83, ns). Similarly the correlation between change in child behavior problems and parent-rated therapeutic alliance was also low (r(60) = 0.13, ns). These findings indicate that therapeutic alliance and therapeutic change and therapeutic were not strongly related. Interestingly, substantially different amount of contact with a therapist did not lead to differences in quality of the alliance or the alliance-change relation.

We have known for some time that symptom change and improvement in therapy are not highly related to how acceptable patients find their experience in treatment (e.g., Ankuta & Ables, 1993; Lambert, Salzer, & Bickman, 1998). With two variations of online therapy, this is an opportunity to evaluate the extent to which there is a relation between therapeutic change and perceived acceptability of the interventions. Therapeutic change scores on child behavior problems (as measured by the CBCL’s total problem score) were correlated with parent-rated (as measured by the PEI) and therapist-rated acceptability of treatment (as measured by the TEI) separately for each intervention group. The correlations between total problem change scores and parent-rated treatment acceptability were low (r(29) = −0.15, p = .42 and r(27)= 0.08, p = .69 for the Full Contact PMT and Reduced Contact groups, respectively) and did not vary for the two groups (Fisher’s direct test, z = −0.05, ns). A similar pattern emerged with therapist-rated therapeutic acceptability. The correlations between antisocial behavior change scores and therapist-rated treatment acceptability were low (r(29) = −0.23, p = .22 and r(27) = 0.14, p = .48 for the Full Contact PMT and Reduced Contact groups, respectively) and were not different from each other (Fisher’s direct test, z = −1.38, ns). The key finding is that both groups found treatment as highly acceptable, were not different in perceived acceptability, and that acceptability and treatment outcome were not differentially affected by the amount of therapist contact.

Comparisons to in-person PMT

One of the natural questions in the development of Internet-based interventions is whether they work as well as more traditional treatment formats. While this question was not a primary focus of the current study, a subset of variables was compared between the online sample and the benchmark sample described previously. Three variables, for which data were available for the online treatment groups and the benchmark group, were selected for these comparisons. The CBCL (total problem score) and the IAB (total score across a range of aggressive and antisocial behaviors) were selected as measures that reflect change in child functioning. These measures reflect over symptom domains (CBCL) and also the primary target focus of the intervention (IAB). In addition, the PSI (total score) also was included because of its parental stress has been shown to be related to a positive outcome of in-person PMT (Kazdin, 2010). As noted previously, a matched sample was drawn from participants seen at the same clinical service and who received the treatment in person.

Before running analyses to compare change in these variables for in-person and online treatment, within-group t-tests were computed for the entire benchmark sample to evaluate these changes over time (i.e., from pre- to posttreatment). Participants’ scores on both of the child functioning measures changed significantly from pretreatment to posttreatment with moderate effect sizes (CBCL: t(59) = 6.04 p < .001, d = 0.79; IAB: t(59) = 5.07 p < .001, d = 0.67). Furthermore, parenting stress was reduced significantly during the course of treatment (PSI: t(59) = 5.27, p < .001, d = 0.71). In keeping with prior research and with the results for the two online groups in the present study, children improved significantly from pre- to posttreatment on these critical outcomes.

A second set of analyses was conducted to determine if there were differences in these child functioning and parenting variables for participants who completed online treatment and participants who completed in-person treatment. A two-way repeated measures analysis of variance (ANOVA) was run for each of the variables of interest, with treatment group (online and in-person) as the between-subjects variable and time (pre- and posttreatment) as the within-subjects variable. For the CBCL total problem score, the main effect for time was significant (F(1,118) = 96.77, p < .001, d = 1.80), indicating that children displayed fewer behavior problems at the end of treatment compared to the beginning of treatment. The treatment x time interaction was not significant (F(1,118) = 0.27, p = .60), suggesting that the pattern of change over time did not differ between the two groups. The results from the IAB total score were similar. As with the CBCL, the main effect for time was significant (F(1,118) = 60.03, p < .001, d = 1.43), but the treatment x time interaction was not significant (F(1,118) = 0.18, p = .68), indicating that antisocial behavior changed for both treatment groups over the course of treatment. In terms of the PSI, the main effect for time with was significant (F(1,118) = 61.45, p < .001, d = 1.40), indicating that parents reported lower levels of stress after treatment relative to before treatment. However, the treatment x time interaction was not significant (F(1,118) = 1.18, p = .39). Overall, these findings suggest that the patterns of change in child behavior and parenting were similar for participants in both online and in-person treatment groups.

Discussion

Children and families in both intervention groups improved significantly in multiple domains over the course of treatment, whether they received traditional amounts of contact with a therapist or significantly reduced contact. By the end of treatment, children showed less antisocial behavior, lower internalizing and externalizing problems more generally, and positive changes in prosocial functioning. Similar and significant improvements were also noted in their families: their primary caregivers reported fewer symptoms of depression and reduced stress and improved family relationships. Parenting practices, the primary focus of training, also improved with both treatments. These findings are in keeping with our prior work on in-person treatment.

The full or reduced contact treatments were similar in how they performed and in the treatment outcomes. Parents in both groups were equally adherent to the interventions regardless of which treatment they received and were no different in the degree of improvements in their parenting practices. In addition, there were no differences between treatment groups in therapeutic alliance, regardless of whether the relationship was rated by the parent or by the therapist. However, a noteworthy difference was observed between the two groups in terms of perceived acceptability of the treatment. Parents who received the treatment with more therapist contact rated their experience as more acceptable than parents who received treatment with less therapist contact. A similar pattern emerged when therapists rated the treatments. Like the parents, therapists rated the treatment with more therapist contact as more acceptable.

We focused on four domains relevant to evaluations of mental health resources: therapeutic change, therapeutic alliance, parent adherence to treatment, and treatment acceptability. In terms of therapeutic change, one of our interests was to identify differences, if any exist, in clinical outcomes for families in each of the two treatment groups. Participants in both groups experienced similar patterns of improvements in externalizing symptoms and antisocial behaviors. However, families direct therapist contact (i.e., the group with more direct therapist contact Full Contact PMT group) experienced a greater improvement in internalizing symptoms. This finding raises the prospect that treatment with full (and much more) contact with a therapist may produce broader effects. We consider this one significant difference between treatments in the larger context of no other differences and therefore constrain speculation about its significance (clinically) or replicability. At the same time, spread of effects to problems beyond a referral domain itself is an outcome in need of further research and whether that relates to characteristics of treatment (e.g., amount of contact, therapeutic alliance) is of interest beyond the scope of this study. The main conclusion for child treatment outcome is that participants showed significant improvements on several outcome measures regardless of whether they received full or reduced contact with a therapist. In terms of parent and family outcomes, participants in both treatment groups displayed similar patterns of improvements across relevant domains (e.g., parental depression, parental stress, family relationships). These findings indicate that, in terms of clinical symptoms and functioning, children and parents in the two interventions experience significant benefit from online parent training, even when the treatment dramatically reduces time with a therapist.

In addition to therapeutic change, we evaluated whether differences would be observed in the therapeutic alliance for participants depending on the amount of contact with their therapist. Participants in both groups reported similar ratings of the therapeutic alliance, in spite of the dramatic differences in the amount of contact with therapists between the treatment groups. This pattern held whether parent ratings of therapeutic alliance or therapist ratings of therapeutic alliance were analyzed. Thus, alliance remained unchanged even with the significant reduction in time spent with the therapist.

Parent ratings of adherence to treatment did not differ across the two treatment groups. Parents in the reduced contact treatment group rated themselves as using the skills with the same frequency and consistency as parents who had more intense contact with their therapist. The similarity in clinical outcomes is consistent with this finding. We would expect parents with similar outcomes to be similar in their adherence to treatment. However, therapists rated parents in the reduced contact group as less adherent to treatment than parents in the full contact condition. This discrepancy between parent and therapist perceptions of adherence is a finding that requires additional research in the future. One possible explanation for this pattern is that the therapists were influenced by their knowledge of study hypotheses and group assignments. Yet, ratings of therapeutic change did not appear to be biased by this information. Even so, replication of these findings with treatment providers who were masked to study hypotheses would be helpful in clarifying whether parental adherence to treatment truly is influenced by the amount of contact with a therapist. The addition of assessment tools not completed by parents or therapists (e.g., teacher-report forms) also would be helpful in clarifying if or how therapist bias may have influenced these results. An alternate explanation may be that the therapists, because of their reduced contact with the families, were not able to accurately estimate parental adherence to treatment. Again, replication is needed in order to better understand these findings.

Finally, parents in both groups rating the treatments as highly acceptable. Even though parents perceived the treatments positively across both groups, parents with full contact with their therapist rated treatment as more acceptable than parents who engaged with their therapists less frequently and less intensely. This pattern of findings was observed when parents rated their own perceptions of treatment and when therapist provided these ratings. The consistency in this finding indicates that, regardless of the clinical improvements noted in children and families, parents view the reduced contact intervention as less acceptable. Acceptability has enormous implications, as we noted previously, in relation to seeking treatment and recommending a given treatment. Three comments provide an important context for evaluating the differential acceptability of treatment and whether reduced contact with a therapist beyond the confines of this study would lead to lower acceptability. First, participants across both groups rated the treatment as highly acceptable. So, while there was a statistically significant difference noted, it is unclear whether this would translate to a meaningful different in the real-world experience of treatment. Second, there might well be a cohort effect on technology-based interventions and varying degrees of contact with a professional, mental or physical health professional. Each new birth cohort seems to be moving more toward a robotic, smart watch/phone, app, and virtual reality life. Reduced face-to-face contact may have less impact on acceptability over time. Third, researchers have argued for the need for a portfolio of interventions to expand the modalities of delivering treatment in order to reach the majority of people in need of services who receive no interventions at all (Kazdin & Blase, 2011; Kazdin & Rabbitt, 2013). Simply stated, more treatment delivery options are necessary, and these options are likely to vary in all sorts of characteristics (e.g., scalability, convenience, cost, acceptability). We want diverse options with the hope that more people in need will be able to access care. The present study shows that a reduce contact Internet-based treatment has many of the advantages of treatment that requires full time therapist involvement. These advantages include therapeutic change in children and their parents.

In spite of the observed differences related to acceptability, the similarity in outcomes in this online sample (including those who received both full contact with a therapist and reduced contact with a therapist) and participants who received traditional, in-person PMT is encouraging. Regardless of whether the intervention was provided online or in-person, the same pattern of significant change was observed: children engaged in fewer behavioral problems, including disruptive and antisocial behavior, over the course of treatment and their caregivers reported significantly reduced stress levels after completing treatment. While only three measures were used for these comparisons, the lack of differences in the findings provide further evidence that reduced-contact adaptations of parenting interventions are a viable treatment alternative.

Important limitations of the study are worth noting. First, the participants were primarily White, well educated, and supported by a partner (e.g., spouse, boyfriend/girlfriend). They may represent a more protected sample than is typical for studies of child conduct problems. Special efforts were made in the course of the study to recruit an ethnically diverse sample (e.g., targeting geographical regions of the state based on U.S. Census data), but our difficulty in enrolling these participants is noteworthy. Based on this study, special outreach efforts (above and beyond those implemented here) may be needed to engage a diverse sample of families. PMT has worked across different ethnic groups both in prior research (e.g., Kazdin, 2010) and more broadly across a range of studies (e.g., Miranda et al., 2005); based on this past work, we would expect similar patterns of improvement in a more diverse sample of children and families. Nevertheless, technology-based variations of treatment have been less well studied and their applicability across different groups still warrants direct evaluation. After all, an effective treatment that is Third, a small number of supplementary analyses were completed to compare Internet-based treatment to the traditional in-person treatment format. Even though results were encouraging, only a subset of the critical domains of interest was included in these analyses. Information about key topics, including therapeutic alliance and treatment acceptability, were not compared in this study, and it is not known what differences (if any) would be evident between online and in-person treatment groups in these domains.

Looking ahead, there are multiple specific ways that this work can be expanded in future studies. Full and reduced contact with a therapist raises an obvious question that was omitted from the design but very relevant to our goal of expanding treatment options. Specifically, what would be the effects of a completely self-help (i.e., no therapist) contact condition? The clinical sample with which we work is significantly impaired; it is promising that statistically and clinically significant levels can be achieved with greatly reduced contact. It is possible that self-help (i.e., no therapist contact) might be an effective treatment option for some families. This may be hard for some to imagine for CD given the levels of stress in the home and parent and family dysfunction with which the problem is associated. Yet, we omitted this critical condition and question. As a next step of this line of work, it would be meaningful to evaluate a range of therapist contact levels, from a traditional level (i.e., the Full Contact group used in the present study) to a mid-range level (i.e., the Reduced Contact group used in the present study) to self-help version of treatment with no scheduled therapist contact. In addition, evaluating these interventions in a racially, ethnically, and socio-economically diverse sample and documenting the efforts needed to recruit and retain families from a variety of backgrounds and circumstances is crucial for the goal of implementation of programs like these on a large-scale basis. Finally, the assessment battery must be expanded to include ratings of individuals who are not directly involved in treatment (e.g., the parent, the therapist) and are less vulnerable to biases in their ratings (e.g., teachers, independent observers, staff interviewers masked to group assignment).

Taken together, these findings suggest that Internet-based parenting treatments are effective for improving both child and family functioning for youth with conduct problems. The similar pattern of clinical improvement observed, even when amount of contact with a therapist is dramatically reduced, indicates that the traditional treatment delivery model (i.e., one-on-one services with a trained professional) for parenting interventions is not necessary for clinical change. Indeed, the implications of this work extend beyond treatment for conduct problems. The idea that very similar (in fact, almost the same!) clinical changes were observed when the amount of direct therapist contact was slashed by 80% could and should change how mental health professionals deliver clinical services to people of all ages and for a variety of clinical problems. From the perspective of meeting unmet mental healthcare needs, providing pre-recorded treatment sessions may be a way to expand substantially the reach of services without changing the demand on mental health providers. In our efforts to ensure that more people receive the treatment that they need, our field must adapt other evidence-supported treatment programs to be delivered with reduced therapist contact and consider technologically creative ways to minimize therapist contact while maintaining client engagement.