Computer-Based Intervention with Coaching: An Example Using the Incredible Years Program

Abstract

Increases in computer ownership and Internet use patterns provide a potential avenue for dissemination of evidence-based prevention and treatment interventions. This paper describes the use of a hybrid delivery model combining computer and web-based delivery of content with assistance of a coach through phone calls, electronic messages and home visits, to deliver a psychoeducational intervention (the Incredible Years parenting program) designed to promote behavioral change in parents and children. The model attempted to simulate many of the parent training methods shown to be successful in the original program. The intervention was implemented with 90 Head Start families who reported elevated levels of child behavior problems. Of the 45 families offered the intervention in the final year of the project, using procedures refined in light of the initial year's experience, 82% completed at least half the program and 76% completed the entire intervention. These participants reported high achievement of their self-determined goals and were highly satisfied with the intervention. The combination of technology with professional coaching represents a potential model for adapting and disseminating evidence-based interventions.

Introduction

Computer-Based Intervention with Coaching: An Example Using the Incredible Years Program

There is now widespread acceptance that evidence-based interventions exist to treat a range of mental health and behavioral problems (Baucom, Shoham, Mueser, Daiuto, & Stickle, 1998; DeRubeis & Crits-Christoph, 1998; Kazdin & Weisz, 1998). Yet these evidence-based practices are often not adopted in applied settings (Weisz, Weiss, & Donenberg, 1992). As a result, strategies need to be developed to overcome potential barriers to the widespread adoption of evidence-based interventions (Hoagwood, Burns, Kiser, Ringeisen, & Schoenwald, 2001; Schoenwald & Hoagwood, 2001).

There are a number of potential reasons for this slow adoption of evidence-based approaches. One potential barrier is that service providers do not have the time or financial resources to receive training in and purchase materials to implement a variety of separate evidence-based interventions. Providers may also lack training in or comfort with the underlying behavioral or cognitive-behavioral theories upon which so many evidence-based practices rest.

Benefits and Barriers to Group Psychoeducational Interventions

Group-based interventions frequently rank high on lists of evidence-based practices (e.g., Webster-Stratton, 2001a; Lewinsohn, Antonuccio, Steinmetz, & Teri, 1984), with studies demonstrating that well-designed group-based models are often at least as effective as the best alternative one-on-one approaches (e.g., Webster-Stratton, 1984). In addition to being effective, group-based approaches are more acceptable than one-on-one therapy for some individuals. For example, one outcome study randomly assigned individuals to be invited to either group-based parent training or single family office-based parent training (Cunningham, Bremner, & Boyle, 1995). This study demonstrated that parents whose children exhibited the most behavior problems were more willing to attend a group-based parenting course than to attend office-based parent training individually (50% vs. 32%) (Cunningham et al., 1995). These authors also report that parents for whom English was a second language had an even greater aversion to attending office-based therapy, with only 19% willing to attend office-based parent training, although 63% were willing to attend a parenting group. This suggests that the format and the delivery of evidence-based interventions may strongly influence the number of people who could be served by an intervention. This is important because the societal impact of an intervention is determined not only by its effectiveness but also by its reach, i.e., the number of individuals it can potentially serve, as well as its ease of adoption, implementation, and maintenance (Glasgow, Vogt, & Boles, 1999).

There are several possible reasons for the effectiveness and acceptability of group-based psychoeducational interventions to many participants. The course-like format may be less stigmatized than attending “therapy.” Additionally, in a group, participants may spend less time dwelling on the details of the problems and challenges they are facing than they would alone with a therapist, and more time on learning solutions for these problems. They may also benefit from learning how others are coping with similar problems, learning from their example, and gaining support from them. It is possible that in a group, participants have a greater desire to share positive experiences with their peers and thus are more motivated to put what they've learned into practice.

Although group-based services have a number of potential advantages, including cost-effectiveness and acceptability to participants, there are unique barriers that often inhibit their wider dissemination. Often service providers have too few clients with a specific presenting problem to be able to organize groups. To ensure individuals don't have to wait too long for assistance, individual services are offered instead. This can be especially true in small communities and agencies or for private practitioners. Additionally, groups are often offered in the evenings since this is the most likely time that all participants can meet. This is often less convenient for service providers, and they may limit the number of such groups they are willing to facilitate. Many service providers have little training or experience with group-based approaches and may not be comfortable with the greater structure imposed by manualized group-based interventions, even though participants enjoy them. As a result, efficacious group-based services may either be ignored completely by service providers or be implemented less often than they could. To facilitate the spread of the benefits of these interventions, it may be beneficial to develop alternative delivery formats that retain the aspects of these interventions that make them effective, while eliminating some of the potential barriers to their dissemination.

Potential Benefits of Computer and Web-based Technology

The use of computer and web-based technology holds some promise for adapting evidence-based skills training interventions. These formats allow not only printed media but audio and video-based media to be presented in a structured manner. The web also makes it possible to allow individuals to connect with others experiencing the same problems somewhere else in the world. However, on their own, such interventions minimize the potential beneficial role that a professional can play to manage change. Skills training approaches to change entrenched and complicated habits often involve individualizing the intervention with a chance to problem-solve and consult with an experienced professional as part of the experience. Although purely technological approaches may be effective for mild problems where simple information is sufficient to promote behavioral change, to modify more significant behavioral habits, the personal assistance of a professional may also be necessary.

In this paper, we describe a hybrid model that takes advantage of many of the benefits of computer and web-based technology, while incorporating the regular attention and assistance of a professional “coach.” This intervention retains many of the potential benefits of the original group-based psychoeducational program, while simultaneously overcoming many of the barriers. It is hoped that this case study of adapting an evidence-based practice to a new format will serve as a stimulus and a model for similar adaptations of other evidence-based skills training interventions.

Incredible Years Group Parenting Program

Behavioral parent training is widely recognized as one of the most effective approaches to reducing early conduct problems (Taylor & Biglan, 1998), and the group-based Incredible Years videotape modeling group parenting program has been recognized as the most carefully evaluated intervention available for parents of preschool and young school-aged children exhibiting aggression and conduct problems. A series of randomized controlled trials conducted by the program developer (Reid, Webster-Stratton, & Baydar, 2004; Webster-Stratton, 1984, 1998a; Webster-Stratton & Hammond, 1997; Webster-Stratton, Kolpacoff, & Hollinsworth, 1988) and others (Brotman et al., 2005; Gross et al., 2003; Scott, Spender, Doolan, Jacobs, & Aspland, 2001; Spaccarelli, Cotler, & Penman, 1992; Taylor, Schmidt, Pepler, & Hodgins, 1998) have demonstrated that the program is effective at reducing behavior problems in children, improving parenting practices and reducing parent depression.

The Incredible Years group parenting program is described in great detail in the book for parents (Webster-Stratton, 2006), the treatment manual (Webster-Stratton, 2001b) as well as in several published articles and book chapters (Webster-Stratton, 1998b, 2000; Webster-Stratton & Hebert, 1993, 1994; Webster-Stratton & Reid, 2006). The following brief summary highlights the major elements of this intervention.

One of the most important aspects of the Incredible Years preschool basic parenting program is its attention to creating an effective instructional process. Over 12–14 weeks parents watch approximately 250 carefully selected videotape vignettes of other parents interacting with children, sometimes effectively, sometimes less skillfully. Skills are taught in a careful sequence, with each step building upon each other in small increments. Following each vignette the group facilitator asks open-ended questions about the vignettes to stimulate thought and discussion about key principles of parenting. Instruction is further reinforced through the use of a book for parents (in print and on audio CD) and handouts. Participants engage in role-plays to practice the skills and engage in home assignments to practice the skills between sessions. Facilitators also monitor parents' participation in the program by calling and encouraging them to do make-up sessions if they miss a class, and by checking in each week on parents' experience putting the new strategies into practice. In addition to careful attention to monitoring the learning process, the group facilitators encourage parents to set goals for both their children's behavior and their own behavior, and make manageable short-term plans for achieving those goals. All of this occurs in a highly collaborative atmosphere that is designed to motivate, reinforce and encourage parents to put new learning into practice and facilitate interaction with other parents about parenting.

Although the Incredible Years group parenting program has been shown to be highly effective, and to be able to engage a high percentage of both clinical (e.g., Webster-Stratton et al., 1988) and highly stressed low-income families (e.g., Webster-Stratton, 1998), there are still potentially some families for whom a group-based model is not ideal. Applied settings are often reluctant to address barriers such as transportation and childcare, or ensure that meals are offered for an early evening group. Even with these issues addressed, the logistical barriers of work schedules and of organizing an entire household to get out to the group may be overwhelming for some families. For others it is difficult or impossible to participate in a group, either for health or personal reasons.

The Self-Administered Parenting Program

To be able to reach a broader audience, Webster-Stratton developed a self-administered and self-paced version of the Incredible Years parenting program that includes the same content videotapes, handouts, and home activities as those used in the group-based program. Two randomized controlled trials were conducted in which parents came into a clinic at their convenience 10 times over a period of approximately 10 weeks, and watched the videotapes accompanied by a self-administered manual (Webster-Stratton et al., 1988; Webster-Stratton, 1992). These studies demonstrated that the self-administered program achieved most of the benefits achieved by the group-based parenting program in the short term. However, by three years follow-up the families who received the self-administered intervention had lost some of the gains they made relative to families who attended a group (Webster-Stratton, 1990). A third randomized controlled trial evaluating the program demonstrated that the addition of two visits with a therapist resulted in increased parents' satisfaction with the self-administered intervention (Webster-Stratton, 1990).

Although allowing a more flexible schedule, this model eliminated some of the potentially important strategies (e.g., role plays), and also had potential barriers for implementation. It still required families to schedule and make repeated visits to the clinic. The clinic was required to have a receptionist available to greet parents and supply them with the next videotape to watch. A room with a TV/VCR was also necessary for families to use. This implementation format also offers little control over how the program is viewed. Parents might fast-forward through some vignettes or choose not to watch others, or simply fail to pause the videotape after each vignette and read the appropriate sections of the self-administered manual.

The Theory that Guided the Development of the Computer and Web-Based Intervention with Parent Coaching

The goal of this project was the development of a new format for delivering the Incredible Years content that would allow the self-administered program to be viewed in the parents' own home, while incorporating many of the beneficial elements of the group-based format. An analysis of the collaborative process used by group leaders to facilitate groups identified that most of the strategies served one of six objectives or functions. These were 1) enhancing the instructional process, 2) arranging role-plays to rehearse skills, 3) monitoring parents' participation in the program as well as their use of skills learned each week, 4) engaging parents in goal setting and making commitments, 5) motivating, reinforcing and encouraging parents to put their new learning into practice, and 6) facilitating interaction with other parents about parenting. Efforts were made to achieve these same objectives with the new format of the intervention, either by implementing the same strategies or newly developed strategies to achieve the same objective. Below is a description of how technology and parent coaches were used to achieve each of these objectives.

Methods

This paper reports in detail on an intervention implemented as part of a randomized controlled trial prevention study. Analyses comparing the intervention and control groups will be described in other papers. The purpose of this paper is to offer a detailed description of the intervention and to report on attendance rates and self-reported goal completion.

Participants

Four-year-old children attending Head Start classrooms in seven rural and urban communities in Oregon were recruited based on elevated scores on a screening questionnaire, given to their primary caregiver/parent (majority being mothers), administered to identify those with elevated levels of oppositional behaviors. A total of 1,510 Head Start parents received the 16-item oppositional behavior scale from the Child and Adolescent Disruptive Behavior Inventory (CADBI v. 2.3; Burns, Taylor, & Rusby, 2001) to complete, and 1,167 (77%) did so. Parents who rated a child in the top 33% of behavior problems among Head Start parents (a score of 40 or above), who had a contact phone, and were English speaking were eligible for inclusion in the study. This ensured that all families were reporting elevated behavior problems, the primary target of the intervention. From the 380 families eligible for inclusion in the study, 178 (47%) agreed to be in the study and were enrolled, 90 (24%) refused, 97 (26%) were unable to be contacted, 9 (2%) had only cell phones, and 6 (1%) had moved. Of this group, 87 served in a no-treatment control group, and 90 received the intervention in this study. As this paper focuses only on the families in the intervention condition, demographic information is reported only on these 90 families.

Forty-three of the target children were females (46%). According to parent reports, 18% of children were Hispanic/Latino and 82% non-Hispanic. The racial breakdown was 81% White, 4% Indian/Alaska Native, 2% Asian, 1% Native Hawaiian, 3% Black/African American, 7% multiracial, and 2% not reported. Forty-nine percent of parents were unemployed, 14% had less than a high school education, 27% completed high school or GED, 51% had taken some college classes and 8% had graduated college. Forty-six percent of parents were married or living together as if married. The mean age of the parent/caregiver was 34 years, with a range of 23–54 years of age. The median reported income was between $10,000 and $14,999. Computer familiarity ranged from 5% who reported no familiarity with computers, 36% somewhat familiar, and 59% moderately or very familiar with computers. Computers were loaned to all participants and dial-up internet service was supplied.

Coaches

To ensure maximum similarity with the group-based model, experienced professionals were hired to serve as coaches, most of whom had been trained in and led Incredible Years parenting groups prior to the project. A total of 16 people served as coaches over the 2 years of the intervention. Those not trained and experienced in leading the Incredible Years parenting program had experience with behavioral parent training or cognitive-behavioral therapy. These individuals also attended the authorized 3-day training in the Incredible Years group program by a certified trainer in addition to the training offered to all coaches. The majority of coaches had other full-time jobs (e.g., child mental health therapists, school counselors). Some were at a stage in life (retired, home with young children, graduate student) in which a part-time position with a high degree of flexibility was of benefit. All coaches then received specialized training in how to serve as a coach for this project, and began serving a “pilot” family about 6 weeks prior to beginning with their first project family.

Instruction

The role of technology

A large part of the instructional content of the intervention was presented through technology. The video vignettes, sound files, and pictures were on a password-protected location on the hard drive of the computer loaned to families, so that only simple graphics and text had to be downloaded. This hybrid system of combining Internet delivery of some content and accessing media files on the computer allowed for videos at a higher resolution than is typical for web-based video (448 X 336), and allowed participants to access all media elements of the program quickly while still using the dialup internet service provided for them during the project.

After logging in to a secure website, participants were able to watch the same 250 video vignettes that are shown in the group-based parenting program in the same order. After each vignette, a picture of the last frame was frozen on the screen, offering a visual reminder of what was just seen while an audio recording posed questions to participants similar to those that a group facilitator would pose. These questions were designed to get participants to think about what they just observed and why it was effective or ineffective. A written summary of key points also appeared after most vignettes, with the option of audio presentation as well to minimize literacy requirements of the program. This served as an alternative to information that typically would be shared by group facilitators. To watch the next vignette, the participant was required to click “Next,” ensuring participants didn't simply start the program and then leave the computer while it ran. Technical support was provided through a toll-free number for both hardware and software problems.

In addition to the computer-based presentation of the program, participants received the same supplemental materials that are used in the original Incredible Years parenting groups. This included a copy of the book “The Incredible Years: A Troubleshooting Guide for Parents of Children 3–8 years” (Webster-Stratton, 1992). On request participants received this book on compact disc as well, further reducing literacy demands. Participants also received numerous handouts summarizing key points covered in each program topic, as well as suggested home activities to complete after viewing each topic.

When a participant completed a topic, the program would not let the participant continue for a period of time (several days at the project outset, reduced to one day for the second cohort). This created the opportunity for participants to take time to practice the skills before moving on to the next topic. In addition, at two points in the program at which home visits were to be scheduled (after topic 4 and after topic 7), participants could not continue until the coach turned a “switch” on the web that allowed the parent to resume. This break provided coaches with an opportunity for a home visit, keeping the participant's focus on the content just covered that would be reviewed during that visit. The program was available 24 hours a day to fit into the hectic schedule of parents of preschool children.

The role of coaches

Although much of the instruction of the Incredible Years program occurs through technology and media, the coach still had an important role in enhancing instruction. Coaches attempted to schedule five home visits with each family, one before participants began the program, three during the process, and one at the end. Coaches averaged four scheduled visits per family. In their first home visit, coaches gave an overview of the program, instructed participants how to access and use the web-based program and took them through the first few vignettes. When they subsequently visited participants in their homes, they reviewed the content of the topics covered and highlighting key principles. Coaches offered additional instruction in an informal manner, especially if participants did not appear to understand some important concept. A detailed manual for each visit described each of these strategies. These strategies were designed to assist participants to understand and implement the strategies taught.

Role Plays and Rehearsal

During the second through fifth home visits, coaches were responsible for getting participants to practice or rehearse how they would try to implement the skills learned in the most recently covered topics. Coaches invited parents to participate in planned role-plays during each visit. If one or more children were available, the practice of playing was done with the child with the coach observing, joining in to model if necessary, and afterwards describing specifically what was done well. The manual outlined potential role-plays, although coaches were encouraged to adapt or add additional practice as needed. Coaches answered questions and listened to how participants had put the skills into practice already, and encouraged them to think about how they would put the skills into practice in the coming days. Coaches also helped participants to generalize the skills learned to new situations and to problem-solve.

Monitoring Participant's Implementation

Perhaps the most unique aspect of the web-based version of the Incredible Years program is the ease with which coaches could monitor the progress of the participants through the program. On a special website accessible only to coaches and supervisors, coaches could view daily updates on each individual. Information available included when a participant logged onto the website, when the participant watched the most recent new vignette and which vignette it was, how many times the participant logged on in the current week and previous weeks, whether they had read or posted any messages in the bulletin board, and how much time the participant spent logged on. The coach could also see a chart that graphed the participant's progress through the program.

Coaches used this information to shape their contact with participants. If a participant watched some vignettes, especially after being stuck for a while, the coach would often leave an electronic message praising their progress and would mention it during their next weekly phone call. If a participant failed to make progress for a period of time, the coach would initiate a contact with the participant to check in. If reminders and brief problem-solving by phone was not sufficient to resolve the problem, then the coach would arrange a special home visit with the goal of problem-solving difficulties, watching some vignettes together, and getting a specific commitment from the participant to use the program. The coach would monitor progress on the web, following up with praise and encouragement if the participant made progress and gentle reminders if the participant did not. Participants responded positively to these strategies with several commenting that this made it clear the coaches genuinely cared about them.

Goal Setting & Commitments

Goal setting

Another important role of the coach was to help participants to identify and set specific goals, and to make commitments of specific strategies they would attempt to achieve those goals. Similar to the group-based model, this began in the first interview between the coach and the participant(s). After getting to know the participants, and learning what life was like with their children, the coach assisted the participant to identify specific goals. One way coaches began this process was to ask, “Now imagine that I have a magic wand, and it is going to make two to four things better--but just those four things and we have to be really specific--what would those be?” Participants set an average of three goals at the initial visit. Most goals involved specific behaviors they wanted to see more or less often from their target child (“I want him to mind me more”), as well as goals related to their own behavior as a parent (“I don't want to get angry so easily”). After each goal was identified, the coach followed up by learning and recording how often the behavior was presently occurring. Participants were assisted to respond in a measurable way (e.g., “3 of 10 times I ask him to do something,” “twice a day”). They then solicited from the participants how often the behavior would occur if things were better. If the participant set an unrealistic goal (e.g., “He should do what I tell him every time without reminders”), the coach would gently help set a more realistic goal (e.g., “I'm not sure my wand will be strong enough for that. What would be good enough, that you would say this is a whole lot better? I can live with this.”). This process of documenting how often each targeted behavior was occurring now, and how often the participant wanted it to occur, was repeated for each goal. At each subsequent meeting participants were asked to report how often each of their target behaviors was happening, and this was recorded. This helped to keep participants focused on their own goals and to recognize what progress was being made. Participants could add additional goals at subsequent meetings if they wished. However, only goals set in the initial visit were used to evaluate the percentage of goals achieved (reported later).

Commitments

Another important aspect of the intervention was for participants to make specific commitments of what they planned to do over a short period of time. This process of getting commitments began in the first visit. After setting specific goals, participants were asked to decide how many days or weeks they needed to watch all of the vignettes in the first topic (approximately 1 to 1.5 hours in front of the computer). The date for the next visit was typically scheduled at this time to occur typically within 2 to 3 weeks. In this way, if participants followed through on their commitment, they would finish at least the first topic before the second visit. If they did not meet this goal, at the second visit the coach problem-solved and sometimes watched the remaining videos with the participant to help them finish that topic.

Participants were also encouraged to commit to practice the skills that they had been learning. Participants and coaches reviewed together the home activity sheets and coaches helped participants to make specific commitments, such as daily play with their child for 10 minutes. The coaches would then ask about these commitments during brief phone calls between visits, as well as at subsequent visits. Coaches offered praise and encouragement for progress made and problem-solved any difficulties that occurred.

Motivating, Reinforcing, and Encouraging Participants

A fourth important role of the coaches was to motivate and encourage participants. Coaches built relationships by learning about the family and actively listening to any concerns. Coaches encouraged participants to talk about their use of the skills learned, how they felt and how their child responded, and praised whatever they were able to do. Coaches also encouraged participants to praise themselves. Participants realized how much they enjoyed playing with their child or focusing on noticing their child being good. When participants described barriers that interfered with putting the skills into practice, the coaches showed understanding and acceptance, helped them to problem-solve, as well as encouraged them to keep trying. All of these strategies were designed to keep participants motivated to continue through the program.

Motivation to complete the program was also promoted through a flexible time schedule. Although participants were encouraged to set a schedule that would allow them to complete the program in 3 months (similar to the group-based model), they were informed they would have up to 6 months to complete the program. This way if delays occurred, participants still had plenty of time to complete the program. Several extensions of a few extra weeks were granted for participants that had made no progress for months, but were then re-engaged toward the end. On average participants took 24 weeks to complete the program, within a range of 16–34 weeks.

In between the five regular home visits, coaches made regular phone calls to participants to reinforce them for what they were doing, and to encourage them to continue. These calls tended to be brief (average 5 minutes), with the main purpose being to check in on how things were going and offer encouragement to continue to use of the program. Coaches were very accepting and understanding if the participants had not kept up with their use of the program, and asked them for an estimate of what they could do over the coming week. Coaches attempted to call participants at least once per week, and were successful in speaking directly to the participant an average of 14 times and left messages on a machine or with another person an average of 10 times over an average of the 24 weeks that participants used the program. The impression of coaches was that these phone calls were vital to the success of the program. Initially, our system clearly recorded an increase in use of the program immediately following such reminder phone calls. As time went on, it also became clear that participants increased their viewing of the program immediately before a scheduled check-in phone call, so they could tell their coach that they had made progress. Although participants rarely initiated calls to their coaches, many nevertheless commented on how much they appreciated these phone calls because it showed them that the coach really cared how they were doing.

Coaches also left electronic messages for participants regularly in a special secure bulletin board conference area called a webboard. Each family could access only their own private conference, and only they and their coach would post messages there. While most participants accessed the webboard rarely, for others this became a more reliable method for the coach to communicate with the participant than by phone. On average coaches left 17 private messages for participants in the webboard, participants read messages in their personal webboard 35 times, and participants left messages for their coaches 11 times.

Another strategy used to motivate participants was to use tangible rewards. Planned rewards were scheduled for home visits ($10 gift certificate at a local store for the 2^nd visit, $15 gift certificate after the 3^rd visit (approximately halfway), none for the 4^th visit, and $25 gift certificate after completing the program). The impression of coaches was that the Head Start parents were clearly motivated by this relatively modest amount of money. Often participants informed the coaches of their plans for the money, such as buying a birthday present for a child or family member. Our recommendation for others would be to have the same amount of reward at every visit to avoid confusion.

In addition to the planned rewards, coaches brought small surprise rewards for the participants (e.g., bubble bath) at least once during the intervention. (This is also a strategy used by group facilitators.) These gifts were given in recognition of the effort and accomplishment that participants made. The emotional impact of such personal gifts far exceeded the $1–$3 that these gifts typically cost. Several coaches reported that participants cried when they were given these gifts, and many participants commented that no one had ever done something like that for them before.

At the end of the program, similar to the group, coaches reviewed participants' accomplishments and invited them to plan how they would deal with common problems in the future. Typically participants reported that they felt capable of managing new challenges that might arise. Coaches also gave certificates of accomplishments for completing the program. Even participants who completed at least three topics received certificates of completion listing the topics they had completed. All of these strategies were designed to reinforce accomplishments and motivate and encourage them to continue to practice positive parenting.

Facilitating Interaction with Other Participants

Computer and Internet technology made it possible for participants to interact with each other, in a special secure Internet-based bulletin board system located within the website. The goal was to approximate the opportunities for interaction that are offered by the group experience. Participants had access to secure public conferences where they could post questions and information for other participants or coaches to read and to respond. Messages posted would remain for others to read when they logged into the Internet-based bulletin board system. Participants were assisted in choosing a nonidentifying username, and were advised not to give their real names or contact information on the system. They could post messages and read messages from others at their convenience. Coaches assisted participants in posting their first message during the first home visit, and encouraged them to read the webboard regularly and consider posting themselves.

Project staff monitored postings to ensure no inappropriate postings were made and responded to questions if other participants did not. A staff monitored the webboard regularly, and if a participant posted a highly personal or significant posting, informed their coach so the coach would be aware of what the participant shared. Although it was possible to do so, it was never deemed necessary to remove a message posted by more than 80 participants over the course of the project.

Experience during the first cohort was that after their initial introductory post, most participants would read posts but would not post themselves. In the second cohort, coaches made substantial efforts to encourage participants to post, in hopes of building up a critical mass of interest, but for most this service was not very meaningful. It appears that a cohort of 45 participants going through the program at the same time is not enough to create a critical mass for a forum. This parallels the experience of public discussion forums on the web, where several hundred readers are typically necessary before enough people contribute sufficiently to create enough activity (Feil, Noell, Lichtenstein, Boles, & McKay, 2003; McKay, Glasgow, Boles, Feil, & Barrerra, 2002).

For a small minority of participants, however, the webboard did become an important part of the service. In each of the two cohorts a small number of participants ended up leaving messages for each other, often several times per day. In several cases the participants who used the service were socially isolated parents who had difficulty getting out, so this became an important social outlet. In two cases, participants asked coaches if they could learn the identity of the other participant. After checking with both parties and obtaining their permission, the coach gave each the first name and phone number of their “virtual friend.” This allowed them to continue the contact if they wished.

Supervision

Supervision in this project was offered by the first author, a psychologist and certified trainer of the Incredible Years group parenting program. Initial training of coaches in this project relied primarily upon small group workshops to review and role play protocols from the written manual developed for this project. Similar small group supervision meetings were held approximately every 6–8 weeks. However, supervision was significantly enhanced through the use of technology. Staff were required to enter when and how they contacted families, whether in person or by phone, on a special secure website. Their electronic messages to families were automatically viewable by the supervisor. As a result, the supervisor could monitor family progress and coach efforts on a daily basis without having to wait until the next individual supervision session. If it appeared that a family was stuck, the supervisor would often e-mail the coach and request an update on the family, including efforts to contact and encourage them, and suggest that there be a brief supervision phone call about that case in a few days. It was amazing how often by the time supervision occurred, the coach had been successful in contacting the family and the family had proceeded to make progress through the program, as documented by their viewing additional videos. In supervision the coaches shared the often heroic efforts they made to reach the family who wouldn't answer phone calls. Their efforts included mailing letters, warm messages on answering machines, and dropping by the home. Often they shared how they problem-solved with the participant about some particular crisis or difficulty. As a result, much of the phone and in-person supervision involved praising coaches for their efforts and creativity in solving the problems. Often the supervisor sent e-mails to the other coaches informing them about successful strategies, giving them an opportunity to learn from the successes of others.

The ability to monitor coaches' contacts also allowed the supervisor to identify that many coaches were not leaving the number of electronic messages for families that were specified in the treatment protocols. The supervisor selected a couple of electronic messages left by a coach that were good examples of warm encouragement and relationship building. After deleting identifying information, these messages were forwarded by e-mail to all coaches as a positive example of “what many of you are doing.” This prompted some coaches to leave additional messages, from which additional examples were forwarded to coaches. Next, e-mails were sent that included information such as “I see that more than half of you have posted an electronic message to your families this week. Keep up the good work!” These regular e-mails updating coaches on the positive examples, combined with the implied reminder of the expectation, was sufficient to produce dramatic change. We learned that although a gentle individual reminder by e-mail could be effective, any direct requests to individual coaches to engage in some action for which they were delinquent were best made in individual supervision.

Measures

Goal setting

In the initial home visit, following procedures described above, coaches elicited for each goal the participant set the “initial reported frequency” and “goal frequency.” For example, they might report that their child does what he is told without arguing 3 of 10 times, and the goal is at least 7 of 10 times. At subsequent visits, coaches elicited from parents the reported frequency of the behavior targeted. The percentage of each goal achieved was measured by the formula: 100 × (final reported frequency minus initial reported frequency) / (initial goal frequency minus initially reported frequency). Although part of the data collection protocol for coaches in both cohorts, during the first cohort, there were not sufficient monitoring procedures in place, and as a result considerable data was missing. By the second cohort coaches entered the data on goal setting online after each visit, and the supervisor reviewed this as part of supervision. Only data from the second cohort is reported. (Goal achievement was likely higher in the second cohort because of the more systematic attention it received.)

Satisfaction

Parents rated their overall satisfaction with the intervention on four questions each on a 7-point scale. Parents rated “My overall feeling about the parenting program for my child and family is” on a scale from “very positive” to “very negative,” “Would you recommend the parent program to a friend or relative” on a scale from “strongly recommend” to “strongly not recommend,” and two questions asking about their confidence “managing current child behavior problems in the home” and “future behavior problems in the home” on a scale from “very confident” to “very unconfident.”

Results

Participation Results

Participation in the program was high for families in the first cohort and even higher in the second cohort, once reporting and tracking features had been fully implemented and coaching procedures refined through experience. Table 1 lists the number and percent of participants who completed all of the program, “almost all” of the program (all of the positive parenting strategies, limit setting, ignoring, and time out), more than half the program (all the positive parenting strategies and limit setting), and the core relationship-enhancing strategies in the program (i.e., the first three topics–playing with children, helping children learn through play, and praise).

Table 1.

Program Completion Percentages

Participation rates of parents	Cohort 1(45)	Cohort 2(45)	Cohort 1&2(90)
Failed to complete Topic 1	18%(8)	4%(2)	11%(10)
Completed Play & Praise (3 topics)	77%(35)	84%(38)	81%(73)
Completed more than half	69%(31)	82%(37)	76%(68)
Completed 100%	56%(25)	76%(34)	66%(59)

Goal Attainment

Goal setting allowed us to determine participants' progress on initial goals. Among the 45 families in the second cohort, 30 participants reported at least one goal was 100% achieved, 30 reported at least 50% progress on all goals set, and 17 reported all goals were achieved. Forty-one families made at least 50% progress on at least one goal. All four who did not report progress on at least one goal were families who failed to complete the program, three of whom stopped on or before the third of nine topics. Of a total of 128 goals set by the 45 participants, at least 50% progress was made on 102 goals and 100% progress was made on 68 goals. Thus, overall, we have considerable evidence that participants who participated in the intervention made major progress on their goals.

Satisfaction

Satisfaction measures were completed by 83 of the 90 families participating in the study. Of these, 72 (89%) of the participants reported they felt “very positive” or “positive” about the program for their child and family (6 or 7 on a 7-point scale); 77 (93%) would recommend or strongly recommend the program to a friend or relative (6 or 7 on 7-point scale), 63 (76%) felt confident or very confident in managing current child behavior problems (6 or 7 on 7-point scale) and 66 (80%) felt confident or very confident managing future child behavior problems (6 or 7 on 7-point scale).

Discussion

This study demonstrates that a computer and web-based delivery of a skills training and behavioral change intervention, combined with support from a professional coach through phone calls, electronic messages and home visits, was successful in achieving high participation rates and self-reports of goal attainment among an at-risk population that had not initiated a request for assistance. Participants watched instructional videos, read handouts, consulted with and role-played with a trained professional, and engaged in putting skills learned into practice over a period of several months. This project raises the possibility that combining technology with direct involvement of professionals may assist in disseminating key elements of other evidence-based interventions to populations who would not otherwise be able to receive them.

However, participation and self-reported goal achievement are not proof of efficacy. For that, controlled studies with objective outcomes are necessary. Findings from carefully controlled studies with different populations are necessary to determine whether this model is efficacious and for whom. Although this model incorporates many elements of the original model, it offers less opportunity for participants to interact with or receive support from other parents, a possible key element of the original intervention. Additionally, their face-to-face contact with the professional was less frequent than the group-based model, which may also have affected their implementation.

It is worth noting that the participation rates achieved in this study are comparable to the high participation rates achieved in the group-based Incredible Years program. In cohort two of the current study, 82% (37/45) completed half the program and 76% (34/45) completed 100% of the program. In contrast, in an independent replication evaluating this program in a children's mental health center as part of a randomized controlled trial, 35 of 46 parents (76%) completed at least six sessions of the program—the equivalent content to half of the program (Taylor, Schmidt, Pepler, & Hodgins, 1998). In a prevention study conducted by the program developer in Head Start (where, like this study, parents weren't seeking help in the first place), 97/191 (51%) completed at least six sessions of the program (Webster-Stratton, Reid, & Hammond, 2001). Thus, for exposure to the content of the program, this delivery format at least matches, if not exceeds what is achieved in groups in real world settings.

It was the clinical impression of our coaches that a number of the families who were served successfully by this model may not have been as successful in groups. For example, one mother had colon cancer and was unable to sit for very long. Another mother was so obese that she could not go up the stairs to get to her bedroom and slept in the living room in her home. One mother stopped doing the program for a while when dealing with spousal abuse and arranging restraining orders against a former partner. She commented she wanted to resume the program because this was one way she was trying to do the right thing for her child. Several other families in the project were evicted from their homes or moved suddenly during the project and lost touch with project staff. When we were able to reach them after several months, typically through contact information of friends or family participants had voluntarily supplied for just such an occurrence, most resumed the program, often completing it. Many coaches commented that they felt the flexibility of the intervention format made it possible for these and other families to participate in the intervention.

One potential advantage of the implementation model described in this paper is the ability to serve participants over a significant geographic area with a modest number of part-time professionals as coaches. In this project we were able to hire very highly qualified and experienced professionals, most of whom had other primary employment, were retired, were graduate students, or were parents of young children re-entering the labor market gradually. For these staff, the highly flexible self-determined hours were important. (For example, one coach checked in with families and the supervisor by phone while visiting the Grand Canyon, and another sent electronic messages to parents while visiting Israel.) We were also able to offer coaches part-time work for 6 months, and then have most of them return after a 6-month hiatus to work part-time for another 6 months. When one coach moved unexpectedly, we were able to reassign her cases to other coaches with little difficulty. Such a staffing model may serve as a model, not only for research projects, but for serving a geographically disperse population with a specialized intervention.

Limitations of this Study

As noted earlier, this study demonstrates the feasibility of the approach, but is not, in itself, proof of the effectiveness of the approach. For that, results of multiple carefully controlled studies—preferably randomized controlled trials—are necessary. The intervention described in this study was conducted as part of a randomized controlled trial, the results of which are currently being analyzed and will be reported in future papers.

Another limitation of this study was that computers were loaned to all participants and dial-up internet service was supplied. Without this, it would not have been possible to serve families who did not have computers. With the number of homes with computers and internet access continuing to increase, it would be possible to serve many individuals without such efforts. However, there will always be a percentage of the population that could only be served with an intervention of this type by incurring such costs.

Future Directions

This project demonstrates the feasibility of adapting existing interventions to be offered through computer and web-based presentation with professional coaching. This holds promise for nearly any evidence-based skills training and/or behavioral change intervention. In addition to the obvious possibilities in the field of mental health, interventions for chronic health problems may be appropriate targets. Lifestyle and behavior plays an important role in managing many chronic health problems, including diabetes, heart disease and obesity to name a few. Parents of children with chronic diseases may also benefit from the ability to easily seek advice from professionals or other parents over an extended period of time. Such individuals may be more likely to participate in a long-term intervention that can be conducted at their convenience in their own home. The flexibility and ease of accessing such an intervention may have important benefits for reaching more at-risk individuals.

Follow-up to in-person interventions might also be successfully offered using strategies described above. The ability to have simple questions answered by simply posting them electronically to a coach or bulletin board for other participants may help to avoid more serious problems from developing. For group-based interventions that continue on even if an individual misses a session, it may be possible to use a format similar to this one to allow participants to make up sessions. It is possible that by combining the strengths of existing interventions and knowledge with new technology for information exchange, it will be possible to impact the many individuals who experience problems for which evidence-based practices exist, but who are never offered the opportunity to receive such service.